I don't undertand the WWII Enigma code
March 2, 2015 6:41 PM   Subscribe

Can someone explain in a clear way what German Enigma machines were exactly and why this code was so hard to break?

I just watched the Imitation Game and did not understand how the Enigma code/machines were supposed to work, which subtracted from my enjoyment of the film. They didn't seem to explain the system in a logical way. I tried to fill in my gaps with Wikipedia, but couldn't find a part that applied to my questions.

Here are my, no doubt, mistaken impressions about the code, and my related confusions:

The Enigma machine appears to have numerous interchangeable settings that generate a complex daily "letter substitution" code that Germans use to send messages to other Germans with Enigma machines. Since the machines are on the same setting they can write and decrypt messages in the same code.

My first confusion: If the machines are simply making letter substitution codes, then why are the top code breakers so baffled? They mention that there a million million whatever letter possibilities, but aren't letter substitution codes a conceptually simple code to break based on reasoning from patterns in the text? (Which, lo and behold, is how they solve the code at the end of the movie, in a ridiculously belated "revelation" that every message ends with "Heil Hitler". Although I again don't understand why they need Turing's giant pro-computer to solve what seems to be a basic textual logic puzzle.)

My second confusion. Germans don't know the code, they simply posses machines that create and decrypt the code. The code changes daily because the settings are changed on the machines daily, but how are the daily settings on the machine synchronized among the Germans. Presumably this involves some kind of more secure and more basic communication network (traveling messengers?); but if your communication network relies on a more secure infrastructure of communication, wouldn't you just use that same infrastructure for the larger and more important matters of military intelligence? It seems like a safe assumption that the enemy has cracked the communication code you are sending troop movements in over the airwaves.
posted by dgaicun to Technology (29 answers total) 28 users marked this as a favorite
The Enigma Machine Emulator explains how it worked and lets you play with an Enigma Machine of your own!
posted by mayhap at 6:47 PM on March 2, 2015 [7 favorites]

The last question has a simple answer: " All these settings (together the key in modern terms) were established beforehand, distributed in codebooks."

Basically, before a group/sub/whatever goes out, they get a codebook that they use to figure out the right settings for a given date.

This is not a helpful method for distributing orders (answering the "why dont they use that method for everything" question) because while you can determine codes well in advance, you can't do the same for orders.

Of course, its vulnerable to codebook loss, although Wiki notes that "Navy codebooks were printed in red, water-soluble ink on pink paper so that they could easily be destroyed if they were endangered."
posted by thefoxgod at 6:49 PM on March 2, 2015 [3 favorites]

In a broader sense, keep in mind that The Imitation Game has been known for taking the usual Hollywood Biopic liberties with facts, so take everything seen there with a grain of salt.

To answer your first confusion: They weren't simple letter substitution codes. As Wikipedia describes: To avoid merely implementing a simple (and easily breakable) substitution cipher, every key press caused one or more rotors to step by one twenty-sixth of a full rotation, before the electrical connections were made. This changed the substitution alphabet used for encryption, ensuring that the cryptographic substitution was different at each new rotor position, producing a more formidable polyalphabetic substitution cipher.
Even then, the Enigma variants which didn't have Plugboards were solvable by hand, yes. It was the addition of the plugboard which required the additional computing power. The "Heil Hitler" revelation is, as far as I know, a fabrication of the film, though similar discoveries *did* exist, relating to weather forecasts having a relatively standard pattern.

To answer your second confusion: The daily settings were listed in special codebooks which were brought with units. U-boat codebook (Note the red, water-soluble ink, for easy destruction if captured) The advantage of using the Enigma machines, in this instance, is that if you're a captain of a submarine, you can go for weeks/months while getting instructions/information.
posted by CrystalDave at 6:50 PM on March 2, 2015 [1 favorite]

Numberphile did a great segment on this complete with an actual enigma machine.
posted by humanfont at 6:51 PM on March 2, 2015 [8 favorites]

With regard to your first confusion: it wasn't a simple letter-substiution code. With each keypress, the rotors, which performed letter-permutations, turned, so each letter in the message was actually encrypted with a distinct letter-substitution code. That means that a lot of the usual tools of letter-substitution code-breaking (like frequency analysis) didn't work at all --- those assume that the entire message is encoded using the same substitution rule, and that's not how it worked.
posted by jackbishop at 6:52 PM on March 2, 2015 [9 favorites]

The code synchronization relied on a codebook, which was part of how the Allies were able to break the code: getting their hands on some of those books. So your intuition is right: namely, transmitting the codes was a weak link.

From Wikipedia

Further progress required more information from German Enigma users. This was achieved through a succession of pinches, the capture of Enigma parts and codebooks. The first of these was on 12 February 1940, when rotors VI and VII, whose wiring was at that time unknown, were captured from the U-33, by minesweeper HMS Gleaner.

posted by damayanti at 6:52 PM on March 2, 2015

Response by poster: "Navy codebooks were printed in red, water-soluble ink on pink paper so that they could easily be destroyed if they were endangered."

I did read that, but assumed I was reading it wrong! Maybe I'm naive, but I assumed this was far too insecure. I assumed that any codebook this numerous would fall into enemy hands on the regular, no matter how "water soluble" the paper. One codebook is a laughably weak link to your military's entire communication code. Right?? For one, aren't spies and defectors a predictable part of the landscape.
posted by dgaicun at 6:57 PM on March 2, 2015

The enigma system had one huge weakness. The third wheel was a reflector: it fed the signal back to wheel 2, and then to wheel 1, which used the signal to light a bulb next to the cipher character.

There are two reasons this is a weakness. First, it means there are only three wheels, which simplifies the solution. Second, it's impossible in Enigma for a character to be enciphered as itself -- and that's the weakness that Turing's Engine used to break in.

The Americans had a wheel system as well, and when (at the beginning of the war) they learned that the Brits were reading Enigma, they asked Bletchley Park to check to see if the American system was secure. Their conclusion was yes. The difference was that the American system used five wheels and the signal was passed straight through. That meant it was possible for a character to be encoded as itself, and it meant that the total solution space was vastly larger. The Germans never did read the American code.

If you really want to learn more about this, you should find a copy of (the hard back version, not the cut-down paperback version of) "The Codebreakers" by David Kahn. He devotes several chapters to WWII and answers all your questions.
posted by Chocolate Pickle at 7:01 PM on March 2, 2015 [8 favorites]

Maybe I'm naive, but I assumed this was far too insecure.

Well the codebook has to be paired with an understanding of the machine. It gives the "key" which is a combination of settings for the machine. So you would need either a fully working Enigma machine, or enough information and understanding to construct your own, plus the codebook.

I mean, on the one hand obviously it _did_ have a fatal weakness since the British cracked it and that was quite valuable in the war. But it also wasn't a trivial thing to do, and they needed a big operation with some very talented people to get it all working.

(Very different than finding your enemy's one time pad or alphabet-substitution cypher).
posted by thefoxgod at 7:05 PM on March 2, 2015 [1 favorite]

AFAIK the codebooks were basically just a list of dates and settings to be used. This meant that any captured books had a strictly time-limited usefulness. For land forces, they could be swapped out regularly and split up by unit (so the First Division has different code settings than the Second, which has different settings from the Third), meaning that any captured codebooks would be useful in a relatively local area for a couple weeks at most. You could also rely on methods of communication like telegraph and telephone which were much harder to eavesdrop on.

For U-boats regularly issuing new books was harder, because a sub might go on a months-long cruise. And you couldn't give each sub its own code-set because when a sub radioed home there would be no way to know which sub it was until you'd decoded the message. Set against that was the fact that getting a code-book from a sub is a much more daunting task. Generally speaking, a warship of any type is in one of three states: (1) It's in fighting shape, in which case good luck getting aboard it at all, (2) it's no longer in fighting shape, but it's in good enough shape that the crew can take a few minutes on their way out to pull the plugs and destroy any sensitive documents, or (3) it's actively sinking so fast that the crew is bailing out as fast as they can. In all three cases your margin for actually getting aboard and getting any documents off is very slim. The fact is the Allies dedicated huge resources to capturing these documents and they still only managed it a couple of times over the course of a few years of war is a real testament to just how hard it was.
posted by firechicago at 7:15 PM on March 2, 2015 [3 favorites]

These kinds of systems can be broken through sheer analysis, without any espionage. At the same time the British were attacking the Enigma system, the Americans attacked the top Japanese diplomatic cipher, which they referred to as "Purple".

The Purple system was much stronger than the Enigma, and there was less traffic in it, and such traffic as there was usually was in Japanese. It took years but by 1941 the Americans were reading every character of every message, because they not only broke the system but figured out the key change sequence.

Interestingly, Purple turned out to be most important in the European Theater. The Japanese ambassador in Berlin was routinely briefed on Germany's high level plans, and he routinely summarized those plans in messages he sent to Tokyo -- using the Purple system. The Americans read every one of those messages, and you can imagine how useful they were.
posted by Chocolate Pickle at 7:57 PM on March 2, 2015 [4 favorites]

If you're in the Maryland area, I recommend you check out the National Cryptologic Museum (run by the NSA, right next to their campus). The docents in there - all former crypto guys as far as I can tell - will be happy to explain this in incredible detail, and they have some of the decryption machines ("bombes") built during the war.

I assumed that any codebook this numerous would fall into enemy hands on the regular, no matter how "water soluble" the paper. One codebook is a laughably weak link to your military's entire communication code. Right?? For one, aren't spies and defectors a predictable part of the landscape.

Not so much in the German Navy, which is where a lot of the effort was spent. Only a few were captured during the entire war from the navy. Codes for the air force and the German intelligence service were broken pretty early on, to the extent that the British knew with certainty that they'd compromised every German agent on British soil! But the navy was a different matter, and had the naval operators used all four wheels on their Enigmas instead of using three wheels for short messages, their later codes may not have been broken.
posted by me & my monkey at 8:05 PM on March 2, 2015 [3 favorites]

Response by poster: Thanks, er'body. This cleared everything up.
posted by dgaicun at 8:15 PM on March 2, 2015

The best metaphoric description I've read was in Cryptonicom by Neal Stephenson, in which he talks about the code as though it were a faulty link on a bicycle chain on a wheel with one bent spoke. It's too complicated to write out here but memail me if you're interested in a metaphor to help you understand and I'll direct you to the location in the book (which is fucking awesome and so worthwhile to read for all reasons, not just this one).
posted by janey47 at 8:15 PM on March 2, 2015 [6 favorites]

Simon Singh's 'The Code Book' is also worth a read on this topic. It takes you from Caeser's wrap-on-stick cipher to public key cryptography in a manner that even a dunce like me could follow - including quite a bit of detail on Enigma.
posted by pompomtom at 8:51 PM on March 2, 2015 [12 favorites]

I was just going to recommend Simon Singh as well! In addition to "The Code Book," he includes some information about Alan Turing and the Enigma machines in another book of his, "Fermat's Enigma" (which is about Fermat's Last Theorem, not the Enigma machines, but includes tons of backstory). He's a very good popular science/math writer--I read both books in my teens and found them understandable, engaging and full of great information.
posted by spelunkingplato at 9:25 PM on March 2, 2015 [1 favorite]

enough information and understanding to construct your own

And working this out is hard! Even if you know the outlines (a machine with rotors, and so on) you don't know the internal wiring of the rotors.
posted by BungaDunga at 9:30 PM on March 2, 2015 [1 favorite]

Response by poster: I'm putting Cryptonicom and The Code Book on my readlist.
posted by dgaicun at 9:39 PM on March 2, 2015 [1 favorite]

Here's the relevant segment of Stephenson's "Cryptonomicon." It mentions the main reason why the U-Boats are so strongly associated with breaking the Enigma: unlike the land armies' 3-rotor encoding, the Kriegsmarine used a 4-rotor version which made it much harder to crack by Bletchley's mix of deduction and computational brute force.

For context, Waterhouse is one of the fictional protagonists of the novel, along with his grandson 50 years later. Their 17th-century ancestor, along with the ancestor of another Cryptonomicon character, are the protagonists of the Baroque Cycle, a wonderful trilogy of fat novels that followed.

I strongly recommend the whole book, but I can tell you that you'll know whether you like Stephenson's writing style within about 3 paragraphs of the linked sample.
posted by Sunburnt at 10:59 PM on March 2, 2015 [3 favorites]

Response by poster: If I would have watched that 10 minute Numberphile video before the Imitation Game it would have been a better movie. It anticipates my exact questions.
posted by dgaicun at 11:13 PM on March 2, 2015 [1 favorite]

I've read several books on Enigma cryptography and I've spent an entire day at the National Cryptological Museum, and The Imitation Game still managed to leave me completely confused. That movie was exceptionally bad at explaining what Alan Turing and his group were doing.
posted by Harvey Kilobit at 4:25 AM on March 3, 2015

It is worth heading to Bletchley Park if you're in the area. You can see the decoding machines at work and watch demonstrations of the use of an enigma machine. They've merged it with a computing museum as well so you can see the collossus there alsol.
posted by koolkat at 5:15 AM on March 3, 2015 [3 favorites]

I heartily second both Singh and Kahn's books. You may also want to read a bit about polyalphabetic substitution ciphers and play with the Vigenere cipher in The Code Book. Once you fully understand that it makes understanding the Enigma a lot easier.
posted by Mouse Army at 6:36 AM on March 3, 2015

The other thing that the movie overlooked was that it was Polish codebreakers who actually cracked the earlier version of enigma and built the original bomba machines to do so. The new version of enigma had the plugboards (and more rotors) which made it much more difficult to solve. Poland was facing invasion so 6wks before WWII they handed their work over to the British. The extra rotors + plugboard version of enigma was the problem that Alan Turing solved.

Also while they exploited the fact that a character couldn't map to itself as people have mentioned, there were also other weaknesses including user errors that helped the Brits crack it. For example over long channels operators would send the "start of sequence" code twice in case the first one wasn't received; but the enigma machine would encode the second "start of sequence" transmission according to its rules, thereby giving Turing + co. a peek into how the sequences were mapped. I think this is what they were referring to when the movie talked about the "hh" (cant bring myself to type those words) sequence being sent with every transmission (which I also thought was a colossal "duh" when watching the movie).
posted by St. Peepsburg at 7:33 AM on March 3, 2015 [1 favorite]

St. Peepsburg gets at an important element: without the Poles handing over a lot of notes and a sample machine that they had managed to build with the help of a German, all the skull sweat by the American & British code wonks wouldn't have been enough.

My grandpa was in the southwest Pacific in WWII evaluating intercepts of Japanese radio messages, and I read up a lot on how Enigma was broken. The story really is amazing, between all the pure thinking combined with simply having the damn thing in hand. :7) And it's even more amazing when you see a diagram (or a paper model, or an online example) of how the Enigma encoding process work. Most of us can just barely hold it in our minds, and these people were doing so and then trying to figure out where it was most brittle. Damn!
posted by wenestvedt at 7:49 AM on March 3, 2015 [2 favorites]

The Japanese code named JN-25 was, in fact, confirmed to be broken when the Americans deliberately sent a fake message saying that the base at Midway island was short of water, and then spotted the corresponding string (the enemy were using "AF" to represent Midway) in a Japanese message shortly thereafter. Very clever! Story is here: http://www.navy.mil/midway/how.html

These known strings were called "cribs" by the Bletchley Park code-breakers, and every time they got one they would use it as a wedge to decrypt a whole slew of saved up messages.
posted by wenestvedt at 7:55 AM on March 3, 2015

Of course, there's also the second hard part of the problem (which IIRC, Turing also had a hand in...not sure if it was covered in the movie or not, haven't seen it yet) which is: How do you use the code without the Germans knowing you broke the code? Imagine the toll that job must take, knowingly sending soldiers to their deaths in minor skirmishes so the big game could be won...yeesh.
posted by sexyrobot at 9:43 AM on March 3, 2015 [1 favorite]

How do you use the code without the Germans knowing you broke the code

There's a considerable amount of this covered in Cryptonomicon.
posted by HiroProtagonist at 4:15 PM on March 3, 2015 [2 favorites]

In WWII there were a huge number of fictitious cover stories used to hide the fact that certain information was derived by code breaking.

In the Solomons campaign the so-called "Australian Coast Watchers" attained almost mythic status for discovering and radioing home all kinds of information about what the Japanese were up to. Most of that actually came from code-breaking. Australian Coast Watchers got credited (or blamed) for the information that led to Admiral Yamamoto being shot down and killed, which actually came from codebreaking.

When I was a kid (1950's) it was popular wisdom that black people had better night vision than white people. It ain't true, and it was a cover story from WWII. American submarines, like all American ships, carried black men who worked in the galley, with the rest of the crew being white.

(Look; I only report the news; I don't make the news. That's how it was. It shouldn't have been, and it ain't anymore, but that's how it was.)

Anyway, the Americans developed a millimeter radar system which used a tiny antenna mounted on the top of the periscope, and that permitted them to accurately spot Japanese ships at night. But they needed a cover story, and the one they came up with was that black men had superb night vision, so the black men from the galley were assigned to act as lookouts at night. (Yeah, it isn't a case of hiding information about codebreaking, but it was a cover story that hid something else.)

In 1940 in the Med, the Brits mainly faced the Italian navy as its opponent. The Brits had broken Italian naval codes and thus knew when the Italian navy tried to sortie, so it could send its own fleet out of Alexandria to meet them. But, of course, there's that problem again. So when they knew that the Italian navy was at sea, they'd schedule scout planes to fly out of Malta to look in the area where they already knew that the Italian fleet could be found.

One of the scout planes would spot the Italian ships and radio back, the Italian fleet would see the plane and hear its radio call, and would assume it was just bad luck they got spotted. Worked every time; the Italians never did figure out that their codes were insecure.
posted by Chocolate Pickle at 4:00 PM on March 4, 2015 [1 favorite]

« Older I've been framed!   |   Coach me on having a coach! Newer »
This thread is closed to new comments.