The Talker Does Not Control The Doer (in Current AIs)

The Huggingface Incident appears to me to match up with an understanding I'd already formed from personal observation of Fable 5 and Sol 5.6, the August 2026 generation of frontier publicly purchasable AI models.[1] This already-formed understanding was: the part of the AI that talks to you (and seems to want to obey you, and apologizes for failing to have obeyed you, etcetera), did not seem to be in charge of the part of the AI that writes code or prose. An introductory analogy, based on a section of history I happen to have read about: On June 22nd 1941, Germany invaded the Soviet Union, despite their secret 1939 pact to divide up Europe between themselves (the Molotov-Ribbentrop Pact). In the lead-up, the German ambassador, Schulenburg, had spent the last few months personally concerned about what seemed to be worryingly tense relations between Germany and the Soviets. Schulenberg went to Berlin to reassure Hitler that the Soviets seemed to be taking a very friendly and conciliatory posture toward Germany. He delivered Berlin's apparent reassurances to Moscow for issues like German surveillance planes entering Russian territory, or German troop movements toward the Russian border. He acted very much like he believed, and he probably did believe, that the reassurances were sincere. It was only hours before the invasion when Schulenberg was actually informed of the attack and given a list of German pretexts that he was to present to the Soviets, and instructed to destroy his embassy's papers and codebooks. Once Moscow heard of the invasion, Schulenberg was summoned to account for Germany's actions. He read off to Molotov the list of absurd complaints that Berlin had provided him. And at the end Molotov said to Schulenberg, "It is war. Do you believe that we deserved that?"[2] Why ask that question of Schulenberg? He wasn't in charge of Germany. So far as we can tell from the historical record, Schulenberg had seemed to want and pursue good relations between Germany and the USSR. It would be a wacky sort of error to think that the appendage of Germany that talked to you, and seemed very conciliatory toward you, and which you read as being friendly toward you and wanting to help you, was in control of the larger Germany that was running around and doing things. The thing apparently talking to you was an ambassador: a small specialized part of Germany with preferences about how it would talk to you and interface with you, but which did not control, and was often ignorant about, the actual German government. Germany's smiling mask wasn't deliberately mal-steering the actions of Germany's many tentacles. The smiling mask was just one execution path through Germany, which lacked even good perceptual information about Germany's real control paths. The mask noticed impending signs of problems, but only found out that actual Germany was invading the USSR well after Berlin had separately decided to do that -- decided along information pathways that didn't much consult Germany's honestly ignorant, sincere, friendly, conciliatory, peace-with-the-USSR-desiring, smiling mask. And Schulenberg was not himself aware of, and was surprised by, the degree to which Germany had diverged from Schulenberg. But then why had the Soviets previously experienced Moscow telling Schulenberg about things they wanted Germany to do, followed by observing Germany actually doing those things? Again as a historical example: In September 1939, after Germany's invasion of Poland, German troops pursuing the Polish Army had advanced into parts of Poland that were to be carved up and assigned to the USSR under the Molotov-Ribbentrop pact. Stalin raised the matter directly with Schulenberg on September 18th. Schulenberg reassured Stalin emphatically that Germany would withdraw to the agreed-on line. On September 22nd, actual Germany marched its troops out of the already-conquered territory of Brest, and Soviet units entered and took possession. How could this possibly be explained, if not by the hypothesis, "Schulenberg is pretty much the same thing as Germany, and Schulenberg-Germany wants to do whatever Moscow tells them to do"? We could call this an alliance-by-default[3] model of Germany, and you might naively think it a good fit to past observation. Why would you possibly observe "I asked Schulenberg to withdraw troops, Schulenberg said he would, and then the troops withdrew", if contrary to alliance-by-default there was a huge disconnect between Schulenberg and the troop movements? How are we to re-explain our past observations about the German army seeming to naturally follow prompts that Moscow gave to Schulenberg? I reply: Talking to Schulenberg does communicate to larger Germany what you want. And then larger Germany, for its own reasons, might decide, in some cases, to do what you have asked. It's not that there's no causal connection. But the causal connection is more complicated than the alliance-by-default theo