Pearl Harbor and JN-25B - questions

[dudalb]

I think that somebody here is outing himself as a CT.

 

[Caustic Logic]

Alright so I'm on two tracks now with JN-25 - one following Egil's lead in wondering if the pivotal naval orders of Novemebr, outlining the details of the PH attack were transmitted by radio. Most sources i've seen seem to assume this, meaning it should have been intercepted if not decrypted/decoded. The big mystery is that US records do not record the 'order to sail' or presumably the others, which means
they're still among the tens of thousands of classified messages
or it was transmitted over land lines, since the kido butai was still in Japanese waters.
Finding an answer is harder than I thought. No one really seems to know.

The other track is looking into the cases made for the code being broken, by US cryptologists. So far I've seen two different clue tracks, one absolutely worthless, the other perhaps interesting. Here I'll share the worthless one, offered by Stinnett, and why he's wrong - I'll be back later with the other.

Based on "over 4,000" never-before-seen documents he obtained in May 2000, Stinnett was able to glean this ONE vague quote-mined clue that "the 5-NUM code" as he calls it, was broken by Station CAST before November 16. Citing commanding officer Lt. John M. Lietwiler in a letter from that day:

“we are reading enough current traffic (messages) to keep two translators very busy.” [262]

In his own words he explains “Lietwiler bragged that his crypto yeoman, Albert E. Myers, Jr., had initiated a new technique that allowed the cryppies to “walk right across” the Japanese messages.” A further explanatory note attached to this explains Myers and another guy named Hess were transferred to CAST in September 1941 and brought a new machine, the Jeep IV “for recovering the 5 NUM code” “It enabled the cryppies to recover current (July to December 4, 1941) additives and subtractors for the 5-Num code.”

David Kahn panned this:
http://www.nybooks.com/articles/14086
The letter "expresses discontent" over this machine, not joy, and "Whatever Lietwiler is discussing, it is clearly not the Imperial Japanese Navy's main, currently used naval cryptosystem, JN 25 B."
He offers a fuller excerpt (parts not shown by Stinnett bolded):

"We are reading enough current traffic to keep two translators very busy, with their code recovery efforts, etc. included. In this connection, I certainly wish you could see your way clear to drop the ancient history of this cipher and work with us on each current system as it comes up."

One Timothy Wilford seems to agree with Stinnett's take but offers an even fuller look at the letter (This time Stinnett quotes bolded, [...] edits by me):
http://www.cnrs-scrn.org/northern_mariner/vol12/nm_12_1_17to37.pdf

We have stopped work on the period 1 February to 31 July as we have all we can do to keep up with the current period. We are reading enough current traffic to keep two translators very busy, i.e., with their code recovery efforts, etc. included. In this connection, I certainly wish you could see your way clear to drop the ancient history side of this cipher and work with us on each current system as it comes up. With Singapore, we have adopted a system of exchanging block numbers to prevent duplication. We have more or less given them a free hand in selecting the cipher blocks they tackle on account of their more limited traffic.
[...]
Using the 400 high frequency groups we have compiled a table of 24,000
differences. When we are stuck on a column now we take any likely looking group and subtract it from every other group in the column from the master group. […] reference to the table […] reciprocals […] Two days ago I saw MYERS walk right across the first 20 columns of a sheet using this method almost exclusively. In view of this I do not believe we want a new Jeep IV.

Alright, so that's it, Stinnett's one clue. I haven't tried to figure out exactly what's being talked about here, thought I'd let anyone else take a stab for now.

 

[fezzic]

All I know for superenciphered code is that stripping the superencipherment is essential to being able to get at the underlying code groups. TCB detailed how the "difference method" worked (and I still don't really understand it fully) and one thing it needed was lots of material. I have no idea how this Jeep IV device worked, though supposedly it worked very well, and think that it must have had an onerous setup if cryptanalysts would choose to use the manual method in preference to the device.

Given that JN-25B had been in service for about a year, certainly some of the underlying code groups were identified and solved. Given that as a base, it probably is not too surprising that when the additive books were changed, that the first breaks into the new additive came rather quickly.

 

[Caustic Logic]

All I know for superenciphered code is that stripping the superencipherment is essential to being able to get at the underlying code groups. TCB detailed how the "difference method" worked (and I still don't really understand it fully) and one thing it needed was lots of material. I have no idea how this Jeep IV device worked, though supposedly it worked very well, and think that it must have had an onerous setup if cryptanalysts would choose to use the manual method in preference to the device.

Given that JN-25B had been in service for about a year, certainly some of the underlying code groups were identified and solved. Given that as a base, it probably is not too surprising that when the additive books were changed, that the first breaks into the new additive came rather quickly.

Intelligent thoughts, thank you. I agree it looks like they're "walking right across" columns of additives in a manual process, since the Jeep IV was a pain in the arse. Not "walking right over" the actual code. "Reading ... current traffic" is obviously the sticker here - literally it does seem to say they're working with a broken code. But nothing else supports that. "Translators" are busy "reading" current traffic and recovering code groups, as well as working on old intercepts. The request seems to be to get help with "work" on "cipher blocks" for the "current system," not the kind of thing you'd ask for if it was already solved.

Alright, another example of Stinnett's B.S. The second track is more interesting, and more strongly indicates JN-25B was solved, at about this same time. It's based on a years-later memo (May '45) from Lt. Laurence Safford, the "Winds execute" guy who seems a little off, as you can see from some bits in this recent NSA study. This clue was cited by Mark Willey, presumably in his book and on his sites:

The first paragraph of the Congressional Report Exhibit 151 says the US was "currently" (instantly) reading JN-25B and exchanging the "translations" with the British prior to Pearl Harbor.

http://www.geocities.com/Pentagon/6315/pearl.html
I found this exhibit and it says (bolding mine):

Com 16's estimates were more reliable than Com 14's, not only because of better radio interception, but because Com 16 was currently reading messages in the Japanese Fleet Cryptographic System ("5-number code" or "JN25") and was exchanging technical information and translations with the British C. I. Unit at Singapore. […] some large scale movement involving most if not all of the Japanese Navy was about to take place. […] this estimate * was based entirely on "radio intelligence," the Com 14 C. I. Unit being unable to read anything except the Weather Ciphers and other minor systems of the Japanese Navy at that particular time. This fact was known in the Navy Department, and the Director of Naval Communications and the Director of Naval Intelligence were so informed by me.

* evaluation being "strong force may be preparing to operate in Southeastern Asia while component parts may operate from Palao and Marshalls."

I'm presuming Com 14 means Station Hypo at Pearl, 14th Naval District, and Com 16 means Cast, Corregidor, Philippines, 16th naval district. He lists "Com 14-260110 (Nov. 1941), Com 16-261331 (Nov. 1941)," whatever these mean, he seems to be referring to November.

Further passages give context:

The "5_numeral" system yielded no information which would arouse even a suspicion of the Pearl Harbor Raid, either before the attack or afterwards.

If the code was readable but offered no such clue, that would support the idea that the crucial orders to that task force were not broadcast, but sent by wire. This applies also whenever it was broken - the system yielded no such clues, ever, he says. Also if it were broken, he probably wouldn't say this:

[T]he current code (JN25B) had been in effect since 1 December 1940, remained in effect until 27_31 May, 1942, and was partially readable in November 1941. A new system of keys was introduced on 4 December 1941 and reported by Com 16_041502, but the carry over of the old code made their solution quite simple, and we were reading messages again by Christmas, Corregidor getting the "initial break" on 8 December 1941.

Reading again, partially as before... nothing new here.

 

[fezzic]

I think the question comes down to exactly what was meant in context. The fairly complete solution of a code -- to the point where the majority of messages can be decoded -- would not be something that could be easily concealed within the cryptographic organizations. If one station was reading the majority of any given message, presumably reasonable length messages, it is verging on inconceivable that the other stations would not be aware of it -- in particular OP-20-GY. It would inarguably be treason or a serious courtmartial offense for a station to withhold critical information like that with substantial detrimental effects on the Nation's security. And the question would be why?

Another thing that TCB noted was that the IJN reassigned call signs in late 1941, November I think, only 30 days after the previous reassignment. Normal interval was 6 months. The reassignment destroyed the network that the traffic analysts had been building up regarding which IJN radios talked to whom. They had to start all over again.

Traffic analysis also pointed to a general movement south towards what was probably the expected places that would be invaded such as the Phillipines and Dutch East Indies. When you cannot read traffic much, traffic analysis gives indications of where forces were being oriented towards.

In conditions like that, even a sortie message might likely be missed simply because the call sign the message was passed to might not have identified as that of a carrier or a particular task force (note I don't claim any such was sent). Eventually the message might be cryptanalyzed and read with the knowledge that it went to a carrier or carrier task force, but that would be almost certainly after the fact.

 

[Caustic Logic]

If one station was reading the majority of any given message, presumably reasonable length messages, it is verging on inconceivable that the other stations would not be aware of it -- in particular OP-20-GY. It would inarguably be treason or a serious courtmartial offense for a station to withhold critical information like that with substantial detrimental effects on the Nation's security. And the question would be why?

Well, IF, the reason is pretty clear, and the courtmartial/impeachment would be just the beginning. I agree that in the known system anyway, it's highly unlikely anyone unit could crack the code and keep that a secret. That's what's being alleged - that it was broken, apparently all but about two really vague clues have been destroyed, etc... Cause *they* control everything, except Stinnett's and Willey's brilliant and brave minds!

 

[dudalb]

When did the Nazis begin attempting infiltration of East Coast port facilities? Did it happen after Dec. 7, 1941? Hitler was no dummy, and Adm. Wilhelm Canaris was CERTAINLY no dummy. If I'm going to start tearing apart Europe looking for Lebensraum, I'm going to want to have hard on-the-ground intelligence on their large trading partner across the pond who sort of kicked our Teutonic arses back in the Great One. More specifically their naval activities and political readiness to go to war...

There were several cases of espionage by Nazi Germany in the US before Pearl Harbor, though none of Sabotage.
Fact is, the Nazi Espionage ring in the US was never that effective even before Pearl Harbor, and it was quickly crushed after the war began.
And their attempts at sabotage were amateur hour. The attempt to land sabotage teams by Submarine in the US got a lot of publicity but that they were rounded up so quickly just proves that when it came to the US, the Nazi Intelligence agencies blew it bigtime.
Canaris was quickly disillusioned by the failure of German Spying operations in the US, and tried to keep support to an absolute miminum, since he felt it was a waste of resources better used elsewhere.

 

[Gawdzilla Sama]

May as well slip in this point too.

I'm a little confused by Budiansky’s explanation how after the war, Hypo technicians went back to “unread AN [JN-25] traffic that had piled up in the months just before Pearl Harbor,” and while finding some clues of the attack, “None specifically mentioned Pearl Harbor.” [BOW 8-9]

That seems to me either a disingenuous dodge or else an interesting clue. Multiple orders to the fleet sent in JN-25 mention things like (from Yamamoto’s sailing order, 11/25):“The task force […] shall advance into Hawaiian waters and […] attack the main force of the United States Fleet in Hawaii and deal it a mortal blow.” the same traffic should have shown that the attack was slated for “dawn of X-day,” which was already set as the 8th (Tokyo time) as far back as Nov. 7. http://www.ibiblio.org/pha/myths/jm-097.html#1

Q3) Were these orders not included in the JN-25 traffic Hypo received, or is Budiansky playing word games here? Is there good reason to think Hypo should not have intercepted this?

The problem with JM-97 is that the full plans were NOT broadcast. It was 80+ pages, a bit tough for a Morse message. The plans were typed up and delivered by messenger. We actually captured the yeoman that did that work, and he reproduced 95% of that document from MEMORY.

As for the messages that were sent, we simply didn't have the skill to break them before Dec. 7th. And we didn't catch them all anyway. "Climb Mt. Niitaka, 12-8", was not intercepted, we got it after the yeoman mentioned above gave it to us. (His work was independent confirmation of the questionaire MacArthur passed around after the war.)

 

[Gawdzilla Sama]

I can't recall right at this moment, but was any traffic between Japan and Germany intercepted? If Germany and Japan had planned to forge an alliance prior to 7 DEC 1941, is there any evidence that anyone in the German high command was aware of Pearl Harbor prior to its execution? I can see the logic to both angles. If yes, one wants to keep its allies (or potential allies) appraised of a planned attack that could spur a very powerful mutual adversary into action. If no, the obvious reason of OPSEC comes to mind. Anyone with a better memory than me know anything?

The Imperial Japanese Navy didn't even tell the Gaimudaijin (Foreign Office) about the attack. The only information given to them was the time they could deliver the "14 part message." It could be no more than thirty minutes prior to 8 AM, Hawaiian Time. (You gotta love that, "Deliver ambiguous message 'before' attack so we can claim 'you were warned'. The Japanese didn't get around to writing a declaration of war until after they heard the attack on Pearl was a success.)

 

[Caustic Logic]

The problem with JM-97 is that the full plans were NOT broadcast. It was 80+ pages, a bit tough for a Morse message. The plans were typed up and delivered by messenger. We actually captured the yeoman that did that work, and he reproduced 95% of that document from MEMORY.

As for the messages that were sent, we simply didn't have the skill to break them before Dec. 7th. And we didn't catch them all anyway. "Climb Mt. Niitaka, 12-8", was not intercepted, we got it after the yeoman mentioned above gave it to us. (His work was independent confirmation of the questionaire MacArthur passed around after the war.)

Thanks for that. I think we later agreed that it was at least quite likely these orders were never broadcast. I didn't know of direct evidence to the that effect. What's JM-97? Is that some code for the order to sail, I presume?

On your second point, the climb Niitaka message, I don't recall what I decided on that, but I don't think I disagree with your statement. I still feel that even with no direct clues, which I've seen none of, an attack on Pearl on 12/7 should have been predicted. The overall offensive was known to be coming soon, and they could have guessed the date well if they'd considered Sunday as likely due to usual restfulness. And that the main fleet should be hit, well, why the heck not?

Good work all around on putting together the sources, Gawdzilla. I would suggest you keep an open mind about the things we can't know, but otherwise I think you're doing well.

 

[fezzic]

JM 97

From the top of the linked page:

http://www.ibiblio.org/pha/myths/jm-097.html#1

Japanese Monograph No. 97

This monograph is a collection of pertinent Navy Orders and Directives, Combined Fleet Orders, and Carrier Striking Task Force Orders concerning the Pearl Harbor operation. Since all copies of these orders were destroyed prior to the end of the war, they have been reconstructed from personal notes and memory.

FYI.

 

[Caustic Logic]

JM 97

From the top of the linked page:

http://www.ibiblio.org/pha/myths/jm-097.html#1

Japanese Monograph No. 97

This monograph is a collection of pertinent Navy Orders and Directives, Combined Fleet Orders, and Carrier Striking Task Force Orders concerning the Pearl Harbor operation. Since all copies of these orders were destroyed prior to the end of the war, they have been reconstructed from personal notes and memory.

FYI.

Ah, thanks. So it's retro-active American labeling, since the monographs were collected under MacArthur, right? As shorthand, it works.

 

[Gawdzilla Sama]

Ah, thanks. So it's retro-active American labeling, since the monographs were collected under MacArthur, right? As shorthand, it works.

The Japanese Monographs were created under the MacArthur Shogunate to document Japanese policies and actions before and during the war. Of most interest to me is the five volume "Political Strategy Prior to the Outbreak of War." Patrick and I will eventually have the most important ones online in HTML format. You can obtain copies of the original (English version) documents at MilSpecManuals.

 

[Gawdzilla Sama]

JM 97

From the top of the linked page:

http://www.ibiblio.org/pha/myths/jm-097.html#1

Japanese Monograph No. 97

This monograph is a collection of pertinent Navy Orders and Directives, Combined Fleet Orders, and Carrier Striking Task Force Orders concerning the Pearl Harbor operation. Since all copies of these orders were destroyed prior to the end of the war, they have been reconstructed from personal notes and memory.

FYI.

Very true. JM-97 is the Separate Volume appended to the Pearl Harbor Attack Plan, the nuts and bolts of the attack. One interesting point found there is that the first priority was for initial wave to attack four carriers and four battleships, if present. Dividing the attack between the two types gives us an idea of how each was regarded by the Japanese at least.

 

[Gawdzilla Sama]

"On your second point, the climb Niitaka message, I don't recall what I decided on that, but I don't think I disagree with your statement. I still feel that even with no direct clues, which I've seen none of, an attack on Pearl on 12/7 should have been predicted. The overall offensive was known to be coming soon, and they could have guessed the date well if they'd considered Sunday as likely due to usual restfulness. And that the main fleet should be hit, well, why the heck not?"

The Martin-Bellinger Report estimated that an attack on Hawaii, if it could be mounted, "would probably come on a Sunday" morning. However, neither General Martin nor Admiral Bellinger considered it possible for the Japanese to attack Hawaii. For one thing, the Japanese hadn't developed underway refueling of escorts. (En route the Kido Butai had to STOP and refuel stern-to-stern at sea using bouyed fuel lines, an obscene procedure by any standards and an indicator of exactly how insane this mission really was.)

 

[Gawdzilla Sama]

"And that the main fleet should be hit, well, why the heck not? "

I love this. Picture a meeting of the top military and civilian people in the FDR administration.

"Okay, boys, I found a way to get us in the war in Europe!"

"Finally, now we can kills some Nazis!"

"Yeah, we're going to start a war in the Pacific!"

"Excuse me, sir, but WTF?"

"Simple, we start an unrelated war in the Pacific, dividing our forces over the entire globe, and HOPE that Hitler is stupid enough to declare war on us."

"What about the Pacific Fleet? What about the troops?"

"What about 'em? A few thousand dead Americans mean nothing to me."

"Well, okay then, when do we start."

I mean, really, who thinks like that?

 

[Corsair 115]

I mean, really, who thinks like that?

Conspiracy theorists, sadly.