Information Theory

Information theory holds that information can be measured mathematically regardless of what it means or how it is carried, using entropy to quantify a message's uncertainty and the bit as its basic unit. It sets hard limits on two practical questions: how far a message can be compressed without losing content, and how reliably it can be sent over a channel that introduces noise or error. Claude Shannon founded the theory in a single 1948 paper written at Bell Labs, and its framework underlies digital compression, error-correcting codes, and modern telecommunications and computing generally.