Chris Spackman's NeoCities Page

Hitt, Friedman, and the Birth of Scientific Cryptanalysis

For centuries, codebreaking was an art. Skilled cryptanalysts relied on intuition, pattern recognition, and deep experience with languages. In the early 20th century, two Americans changed that — first Parker Hitt, who systematized the practical craft of military codebreaking, and then William Friedman, who introduced statistical and mathematical methods that could determine cipher type, find key lengths, and break encryption systematically. Together, they transformed cryptanalysis from a craft into a branch of applied mathematics.

Parker Hitt: The First American Textbook

Before Friedman, there was Parker Hitt (1878–1971), a U.S. Army officer stationed at Fort Leavenworth, Kansas, and later on the Mexican border. Hitt was fascinated by codes and ciphers, and he realized that the Army had no systematic training for codebreaking. Officers were expected to figure it out on their own.

In 1916, Hitt published Manual for the Solution of Military Ciphers — the first American military textbook on cryptanalysis. It was a slim, practical book, but it covered techniques that would remain relevant for decades:

• Frequency analysis — systematic methods for breaking substitution ciphers by counting letter frequencies and comparing them to known English patterns.
• Digraph and trigraph analysis — looking at pairs and triples of letters (like TH, THE, ING) rather than just individual letters, which gives much stronger clues.
• Breaking Vigenère ciphers — techniques for finding the key length and then breaking each alphabet separately.
• Solving transposition ciphers — methods for reconstructing the original letter order when the message has been scrambled.

The Manual was not a theoretical work — it was written for field officers who needed to break enemy messages quickly and correctly. Hitt filled it with worked examples and practical tips. It became the training manual for a generation of American military cryptanalysts, including many who later joined Yardley's MI-8 and Friedman's SIS.

The Strip Cipher

Hitt was also an inventor. He designed a strip cipher device: a wooden cylinder with 25 removable alphabet strips. Each strip had the 26 letters printed in a different scrambled order. To encrypt a message, you arranged the strips on the cylinder so that one row spelled out your plaintext, then read off a different row as the ciphertext.

The strip cipher was a significant innovation. It was portable, fast to use in the field, and much harder to break than pencil-and-paper ciphers. Most importantly, it was a cipher device — a physical machine that performed encryption mechanically rather than relying on manual letter-by-letter work. Hitt's strip cipher was a direct ancestor of the U.S. Army's M-138 strip cipher, which remained in service through World War II. It also foreshadowed the principle behind rotor machines like Enigma: using interchangeable, rearrangeable components to create complex substitutions.

Hitt's Legacy

Hitt's direct contributions were practical rather than theoretical. He did not develop new mathematics — he organized and systematized existing techniques and made them teachable. But that was exactly what the field needed. Before Hitt, American military cryptanalysis barely existed. After Hitt, there was a trained community of practitioners, a standard reference text, and a cipher device good enough for field use. When war came in 1917, the United States had something to build on.

Friedman himself acknowledged Hitt's foundational role. The Manual was among the texts Friedman studied at Riverbank, and several of the techniques Friedman later formalized mathematically were first described in practical terms by Hitt.

Friedman: From Genetics to Codebreaking

Friedman's path to cryptanalysis was unusual. Born in Moldova in 1891, he came to the United States as an infant. He studied genetics at Cornell and took a job at Riverbank Laboratories in Illinois, working for the eccentric millionaire George Fabyan. Riverbank had a cryptology department run by Elizabeth Smith (whom Friedman married in 1917), and Friedman was drawn into the work.

His scientific training proved invaluable. Where other cryptanalysts saw patterns intuitively, Friedman asked: can we measure these patterns? Can we express them as numbers? Can we build a systematic method that works even when intuition fails?

The Index of Coincidence

Friedman's most important contribution was the Index of Coincidence (IC), published around 1920. The idea is elegantly simple: if you pick two letters at random from a text, what is the probability that they are the same letter?

Why Does This Matter?

In English text, some letters are very common (E, T, A) and others are rare (Z, Q, X). If you pick two random letters from English text, there is a relatively high chance they will match, because common letters show up often. The IC of English is about 0.0667 (roughly 1 in 15).

In completely random text (all 26 letters equally likely), the probability of a match is much lower: 0.0385 (exactly 1/26, roughly 1 in 26).

This difference is the key insight. Different types of encryption affect the IC in predictable ways:

• Monoalphabetic ciphers (like Caesar or substitution) just rearrange which letter is which. They do not change the underlying frequency distribution, so the IC stays near 0.0667.
• Polyalphabetic ciphers (like Vigenère) flatten the frequency distribution by using multiple alphabets. The IC drops toward 0.0385, depending on the key length.
• Perfectly random text has an IC of exactly 0.0385.

The Formula

For a text of N letters where letter i appears n_i times:

IC = Σ n_i(n_i − 1) / N(N − 1)

The sum is over all 26 letters. Each term n_i(n_i − 1) counts the number of ways to pick two copies of letter i, and N(N − 1) is the total number of ways to pick any two letters.

Try It: Index of Coincidence Calculator

Frequency Distribution

The Kasiski Examination

The Index of Coincidence tells you what kind of cipher was used. But for a polyalphabetic cipher like Vigenère, you also need to find the key length. The Kasiski examination, developed by Friedrich Kasiski in 1863, does this by looking for repeated sequences in the ciphertext.

How It Works

In a Vigenère cipher, if the same plaintext sequence happens to align with the same part of the key, it produces the same ciphertext sequence. The distance between these repeated sequences is always a multiple of the key length.

By finding all repeated sequences of three or more letters (trigrams), measuring the distances between them, and finding the greatest common divisor (GCD) of those distances, you can determine the likely key length. Once you know the key length, you can split the ciphertext into groups (one per key letter) and attack each group as a simple Caesar cipher.

Try It: Kasiski Examination

Chi-Squared Test

Friedman also applied the chi-squared (χ²) test to cryptanalysis. This statistical test measures how well an observed distribution matches an expected distribution. In cryptanalysis, you compare the letter frequencies in a ciphertext to the expected frequencies of English.

The formula is:

χ² = Σ (O_i − E_i)² / E_i

Where O_i is the observed count for each letter and E_i is the expected count (based on English frequencies and the total text length). A low χ² value means the text's frequency distribution closely matches English. A high value means it does not.

Using Chi-Squared to Break Caesar Ciphers

For a Caesar cipher, you can try all 26 shifts and calculate χ² for each one. The shift that produces the lowest χ² value is most likely to be correct, because that shift makes the frequencies best match English.

Try It: Chi-Squared Shift Finder

Friedman's Legacy

In 1929, when the Black Chamber was shut down, the Army established the Signal Intelligence Service (SIS) and put Friedman in charge. He hired a team of mathematicians — including Abraham Sinkov, Solomon Kullback, Frank Rowlett, and John Hurt — and trained them using the statistical techniques he had developed.

This team went on to break the Japanese PURPLE cipher machine in 1940, one of the greatest cryptanalytic achievements in history. They did it without ever seeing the machine — they deduced its structure entirely from analysis of intercepted messages, then built a replica.

Friedman's key innovations:

• Index of Coincidence — a universal measure for classifying ciphers and determining key lengths.
• Statistical testing — applying chi-squared and other statistical tests to systematically evaluate cryptanalytic hypotheses.
• Formalization — defining precise terminology and mathematical notation for cryptanalysis, replacing informal, ad-hoc methods with reproducible techniques.
• Training — building a team and an institution (SIS, later NSA) that carried his methods forward.

Things to Try

• Calculate the IC of the English sample and compare it to the random sample. How different are they?
• Load the Caesar ciphertext into the IC calculator. Is the IC close to English or close to random? Why?
• Load the Vigenère ciphertext into the IC calculator. How does its IC compare? What does this tell you about the cipher type?
• Use the Kasiski examination on the Vigenère sample. What key length does it suggest?
• Use the chi-squared test on the Caesar sample. Which shift produces the lowest χ²? Does the decoded text make sense?
• Go to the Vigenère cipher page, encrypt a message with a keyword, then come back here and use the IC and Kasiski tools to figure out the key length.