On May 4, 1863, Hooker examined his situation and found it difficult but far from terrible. The Confederate surprise attack had dented his right flank, but he still had plenty of men, many of whom had yet to fire a shot. As he prepared for battle, Hooker moved with his staff to the Chancellor House, for which Chancellorsville is named, then—much like the evening before—fate intervened. This time it was in the shape of a cannonball that hit the chimney of the Chancellor House.

Hit by the recoil from the chimney, rather than the actual cannonball, Hooker was stunned, even knocked out, for as much as half an hour. When he came to, he was groggy and ill-tempered, but he refused to allow anyone else to take over command of the Army of the Potomac. At that moment the battle began in earnest.