Most breast cancers begin in the cells that line the ducts. The ducts are the passages that connect the lobules (the milk-producing glands) to the nipples. Some cancers begin in the cells of the lobules.