Information theory is a branch of the mathematical theory of probability and statistics, allowing it to quantify concepts of information. It was formulated primarily by American scientist Claude E. Shannon (1916-2001; who was also called “the father of information theory”) to explain the aspects and problems inherent in information and communication. In particular, it involves efficient and accurate storage, transmission, and representation of information, such as the engineering requirements—and limitations—of communication systems. (Note: Information theory has nothing to do with library and information science or with information technology.)

In information theory, the term “information” is not used in the traditional sense. Here it is used to mean a measure of the freedom of choice with which a message is selected from the set of all possible messages. Because it is possible for a string of nonsense words and a meaningful sentence to be equivalent with respect to information content, “information” in this sense takes on a different meaning.