← Hall of Foundations Exhibit 151 · H = −Σ p log p
The Measure of Surprise

Counting what
you don't know

Shannon gave uncertainty a number. Entropy measures how surprising a source is — how many bits of genuine unknowing it holds. It's why the one-time pad is provably unbreakable, and why a weak password falls in seconds regardless of how the cipher is built.

Secrecy is bounded by surprise. No cipher can protect a key that wasn't surprising to begin with. Entropy is the ceiling on security — everything else is just not wasting it.
§1

Bits of the unknown

A fair coin holds exactly one bit of entropy: two outcomes, equally likely. Bias the coin and the entropy drops — a coin that's 90% heads barely surprises you. Drag the bias and watch the entropy fall from its peak.

—
Entropy peaks at 1 bit when the coin is fair and collapses toward 0 as it becomes predictable. Maximum uncertainty means maximum secrecy-carrying capacity.
§2

Key length under a uniform model

These bars assume each character is selected independently and uniformly from the stated alphabet. Under that model, entropy is length × log₂(alphabet size). Human-chosen passwords do not generally follow this model: length and character variety alone do not determine resistance to guessing.

—
Bars show total entropy in bits for three alphabets. As a rule of thumb, 128 bits is the modern floor for a symmetric key — the dotted line.
§3

The one-time pad's perfect secret

A pad chosen uniformly and independently of the message, kept secret, as long as the message and used once provides perfect secrecy for fixed-length messages. Entropy is part of that argument, but a high-entropy string that is public or related to the message is not a secure pad.

Perfect secrecy exists. It just costs one bit of true randomness for every bit you want to hide.
Where this touches your keys

The ceiling on everything

The One-Time Pad’s perfect secrecy is this measure made physical. A cipher fed a low-entropy key is broken before the algorithm runs — the attacker just searches the small space of likely keys. Entropy is why the random number generator matters as much as the cipher.

Security bits estimate attack cost under a specified model; they are not the same quantity as Shannon entropy. AES-128 has a 128-bit key space and is intended to provide roughly 128-bit classical security. A suitable 256-bit elliptic-curve group has generic discrete-log attacks costing roughly 2¹²⁸ group operations. A source's Shannon entropy alone does not specify its password-guessing cost or a scheme's computational security.

Where the ceiling is reached: the One-Time Pad spends exactly as much key entropy as it has message, which is what buys perfect secrecy — and Vernam's 1917 machine is that idea built in hardware, decades before Shannon proved why it works.