James Monroe, the country’s fifth president, served as James Madison’s secretary of state for Madison’s full two terms in office. For a brief time, Monroe also served as Madison’s secretary of war after the resignation of John Armstrong.