The milk let-down reflex may be controlled by a factor that affects the hypothalamus. Anxiety and stress can prevent the flow of milk. Some mothers learn to associate a baby’s crying with suckling. These women may experience the milk let-down reflex as soon as they hear their baby crying.