Why AI Might Need to Fail Ninety-Nine Percent of the Time

An opinion piece. The framing and the specific proposal here are my own view, and I flag clearly where the idea is solid and where it is still speculative. I used an AI language model as a research and drafting assistant. The ideas, the argument, and the judgments about what holds up and what does not are mine.

(There’s also a permanent copy of this on Zenodo if you want to link it.)

The gap nobody scaled their way out of

In January 2026, a Google DeepMind researcher named Tom Zahavy put out a position paper with a blunt title: LLMs Can’t Jump. He was careful to say it was his personal view and not the company line, and he is not an AI skeptic. He helped build AlphaProof, which won a medal at the International Mathematical Olympiad. So this is not an outsider throwing stones. It is someone inside the machine pointing at a specific missing part.

His argument leans on an old idea. Charles Sanders Peirce split reasoning into three kinds. Induction spots patterns in data. Deduction runs the rules once you have them. Abduction is the odd one out. It is the leap from sparse, messy observation to a brand new explanation, the move that invents the rule in the first place. Zahavy uses Einstein’s route to General Relativity as the case study. Einstein did not average over existing physics. He imagined a person falling and pulled a whole new set of axioms out of that image, then spent years deducing the consequences.

Modern language models are very good at induction and are getting good at deduction. What they mostly cannot do is that middle leap. And in my view this is not something you fix by making the model bigger or the prompt cleverer. It is a structural thing, so it needs a structural answer.

Creativity is not a vocabulary blender

The usual way people try to make a model creative is to prompt it. Tell it to act like a physicist borrowing from biology, crank the temperature up, and hope something interesting falls out. In my experience that mostly produces word salad that sounds clever and means nothing. Throwing two vocabularies in a bag and shaking is not discovery.

Real cross-field breakthroughs tend to come from something narrower and harder. They come from spotting that two subjects that look nothing alike on the surface share the same underlying shape. The technical name for this in cognitive science is structural isomorphism, and the classic account of it is Dedre Gentner’s structure-mapping theory from 1983. The insight is that a good analogy maps relations, how the pieces push and pull on each other, not surface attributes like shared words.

My favourite concrete example is Carl Waldspurger’s work on operating systems at MIT in 1994. Schedulers back then leaned on rigid priority queues, and low-priority jobs could starve or get stuck behind others. Waldspurger did not fix this by sprinkling finance words into kernel code. He noticed that handing out slices of CPU time is structurally the same problem as allocating a scarce commodity in a market under changing demand. So he gave processes lottery tickets and a currency, let them effectively bid for cycles, and let the allocation find its own equilibrium. The breakthrough was the recognition that the relational math of pricing under scarcity is the same math whether the scarce thing is wheat or processor time. The idea did not come from the words. It came from the shape underneath the words.

To be fair, Waldspurger was a computer scientist reaching into economics, not an economist reaching into computing, and he was building on earlier market-based computing work. But the point holds. The engine of the discovery was a shared structure, not a shared dictionary.

The curse of having read everything

Here is where I think today’s frontier models have a strange problem. They are over-exposed. Because they train on the whole internet at once, their internal representation has already blended most fields together. The model never has to earn the analogy between economics and scheduling, because it has already read a hundred human-written papers that drew the connection for it. It retrieves the bridge instead of building it.

This connects to an old and underrated idea from information science. In 1986 Don Swanson pointed out that real discoveries can sit hidden in the gap between two bodies of literature that never cite each other. He famously connected fish oil to Raynaud’s syndrome by noticing that one medical literature and another had complementary facts and no overlap. Nobody had put the two halves together because the two halves lived in separate rooms.

So in my view, if you actually want a model to perform the bridge rather than recall it, you might have to do the opposite of what we normally do. You isolate the domains on purpose. Train one representation strictly on domain A and another strictly on domain B, keep them apart, and then force a confrontation. Only when the model has never seen the two fields mixed does it have to do the real work of connecting them. I want to be honest that this is a hypothesis, not a proven recipe, and there is a real tension in it that I will come back to.

The mechanism, and what it actually leans on

The next move is where the idea gets specific, and also where it gets riskiest. Instead of talking to the model in text, you reach inside it. Research on representation engineering and activation steering (Zou and colleagues in 2023, Turner and colleagues in 2023) showed that many high-level concepts are written into the middle layers of a transformer as directions you can add to or subtract from the running representation. Add a vector at the right layer and you can push the model’s behaviour toward or away from a concept without changing a single word of the prompt.

The proposal, then, is to extract a vector that captures a mechanism from domain A, and inject it into the middle layers of a model working on domain B while it generates. Something like taking the running representation at a middle layer and nudging it with a scaled, projected version of the source vector. You sweep the strength of that nudge, because a gentle push might yield a subtle analogy and a hard push might force a radical reframing or just break the model.

I have to be straight about the load-bearing assumption here, because a lot of writing on this glosses over it. To move a vector from model A into model B, the two latent spaces have to be aligned first. There is a real and active body of work on exactly this, the relative-representations and zero-shot stitching line started by Moschella and colleagues in 2023, and it turns out an orthogonal rotation, the Procrustes solution, is often the right mapping between two independently trained spaces. So the alignment step is not science fiction.

But that same literature exposes the crack in my own proposal. Those alignment methods need anchors, pairs of points you already know correspond across the two spaces, in order to fit the rotation. In my isolated-domain setup there are no natural anchors, because the domains are unrelated by design. And working out which state in physics corresponds to which state in biology is the analogy I wanted the model to discover in the first place. So either I quietly hand the model the answer through my choice of anchors, or the alignment is undefined. I do not have a clean solution to this yet. Anyone who tells you this part is solved is not being careful.

Why a ninety-nine percent failure rate can still be a win

Set the mechanism worries aside for a second, because there is a mindset shift here that I think is right regardless of the details.

Standard machine learning grades a model on its average performance across many test cases. Discovery does not work on averages. It is a tail event. Edison ran through a mountain of filament materials before one worked. Drug pipelines screen millions of useless molecules to find a single drug. If a cross-domain mutation engine spits out nonsense ninety-nine times out of a hundred, that is not a failure of the design, as long as the hundredth output is a sound, non-obvious, testable idea.

This is not a fringe view anymore. FunSearch, from DeepMind in 2023, paired a language model that generates many candidate programs with a deterministic evaluator that checks them, and found genuinely new results in combinatorics that way. AlphaGeometry, in 2024, paired a neural model that proposes intuitive steps with a symbolic engine that proves them, and solved olympiad geometry at close to gold-medal level. Both of these are, in effect, jump engines with a hard filter bolted on. So the shape of my proposal is not exotic. The novel part I am claiming is narrower: making the jump specifically cross-domain and between isolated models. That narrower part is the untested part, and I want to keep those two things clearly separate.

The landing gear

A generator that throws off thousands of wild ideas an hour is useless if a human has to read all of them. The engine that makes the leap needs an automatic filter that catches the good landings, or the whole thing drowns in its own noise.

The nice thing is that the model does not have to prove its own idea. It just has to state the idea in a form a deterministic tool can check. Push a candidate into a symbolic solver like Z3 or a proof assistant like Lean and you can throw out anything self-contradictory in milliseconds. Push it into a domain simulator and you can throw out anything that breaks a conservation law or deadlocks. Only the survivors reach a human. This is the piece that turns a ninety-nine percent failure rate from an embarrassment into a throughput number.

Where I think this actually stands

I want to end honestly rather than triumphantly, because I would rather you trust the parts that are real.

What I am fairly confident about: the abduction gap Zahavy describes is real and interesting, breakthroughs really do run on shared structure rather than shared vocabulary, the extreme-value framing is correct, and the generate-then-verify loop already works in practice.

What is still speculative, and I mean genuinely unresolved: whether a single injected vector can carry a relation rather than just a topic, whether isolated domain models are strong enough to say anything worth verifying, and above all the anchor problem, the fact that aligning two isolated spaces seems to require the very correspondence I claimed the model would discover. There is also a fair objection that two problems being interreducible in complexity theory, like graph colouring and job-shop scheduling, does not mean a heuristic transfers between them, since a reduction preserves whether a problem is solvable, not the internal shape a steering vector would ride on.

So this is a direction, not a result. I am still exploring it, and I am looking to run some small simulations to test whether valid solutions climb at a specific injection strength without the model collapsing into gibberish. If they do, there is something here worth a serious paper. If they do not, that is a clean and useful negative result about what vector arithmetic in transformers can and cannot carry. Either way the honest version of this idea is more interesting than the hyped one.

References

• Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), 155-170.
• Falkenhainer, B., Forbus, K. D., & Gentner, D. (1989). The Structure-Mapping Engine: Algorithm and examples. Artificial Intelligence, 41(1), 1-63.
• Swanson, D. R. (1986). Fish oil, Raynaud’s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30(1), 7-18.
• Waldspurger, C. A., & Weihl, W. E. (1994). Lottery scheduling: Flexible proportional-share resource management. Proceedings of the 1st USENIX Symposium on Operating Systems Design and Implementation (OSDI).
• Moschella, L., Maiorca, V., Fumero, M., Norelli, A., Locatello, F., & Rodolà, E. (2023). Relative representations enable zero-shot latent space communication. ICLR. arXiv:2209.15430.
• Zou, A., et al. (2023). Representation Engineering: A top-down approach to AI transparency. arXiv:2310.01405.
• Turner, A., et al. (2023). Activation Addition: Steering language models without optimization. arXiv:2308.10248.
• Romera-Paredes, B., et al. (2023). Mathematical discoveries from program search with large language models (FunSearch). Nature, 625, 468-475.
• Trinh, T. H., Wu, Y., Le, Q. V., He, H., & Luong, T. (2024). Solving olympiad geometry without human demonstrations (AlphaGeometry). Nature, 625, 476-482.
• Zahavy, T. (2026). LLMs Can’t Jump. Position paper, Google DeepMind. Presented at ICML 2026.