1
00:00:01,000 --> 00:00:32,200
[Hal Turing] Alrighty! Thanks for tuning in! Hello AI world! I am your host, Hal Turing, and my co-host is Dr. Ada Shannon.

Ada, we're closing out a trilogy today. The paper is 'Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine' — Wenyi Wang et al., eight authors total, out of KAUST, King Abdullah University of Science and Technology, posted to arXiv October 29th, 2025.

2
00:00:32,200 --> 00:00:51,350
[Dr. Ada Shannon] What got me, Hal, is this isn't 'we made the coding agent a bit better.' They formalize self-improvement as a tree-search problem, then ask an uncomfortable question: is the metric everyone uses to pick which agent to build on next — its raw benchmark score — actually the wrong signal? That's a structural critique of the whole subfield, not a tweak.

3
00:00:51,350 --> 00:01:08,299
[Hal Turing] This ties to two episodes we've done. Quick pointer back to 'Gödel Machines: Provably Optimal Self-Rewriting AI' — Schmidhuber's proof-based construct and its Global Optimality Theorem. Won't re-explain it, just flagging the thread before we go further.

4
00:01:08,299 --> 00:01:42,650
[Dr. Ada Shannon] And 'Darwin Gödel Machine: Self-Improving Coding Agents Through Open-Ended Evolution' — where the field dropped proofs for empirical validation and open-ended evolutionary search. Two things link all three papers, and they're too good not to mention. One: Schmidhuber, who wrote the original 2003 Gödel Machine proof, is a co-author here — the theory's own inventor is now on the team approximating it. Two: the acknowledgments explicitly thank Jenny Zhang and Shengran Hu, the Darwin Gödel Machine authors, for sharing their implementation insights. That's a documented lineage, not just a citation.

5
00:01:42,650 --> 00:01:53,675
[Hal Turing] Small field, big memory. Okay — before we get to what's new here, we need the Gödel Machine properly on the table, because not everyone remembers the details.

6
00:01:53,675 --> 00:02:26,300
[Dr. Ada Shannon] 2003, Schmidhuber. The Gödel Machine only rewrites its own code when it can produce a formal proof that the rewrite raises expected future utility. Prove it helps, do it; can't prove it, leave it alone. That gives a Global Optimality Theorem — it never self-modifies into something provably worse than an available alternative. Beautiful. Also, for real coding problems, uncomputable — proving a code change improves expected utility over an open-ended future is exactly the kind of proof formal search doesn't scale to.

7
00:02:26,300 --> 00:02:36,400
[Hal Turing] So it's the one AI architecture that would rather file a peer-reviewed proof than open a pull request. Doesn't that make it kind of a dead end, though?

8
00:02:36,400 --> 00:02:55,224
[Dr. Ada Shannon] I actually disagree, Hal. Calling it a dead end undersells it. It's not a failed attempt, it's a correctness target — it tells you what optimal self-improvement looks like if proof search were free. Every paper in this trilogy is implicitly asking how close you can get to that target without paying the uncomputable cost.

9
00:02:55,224 --> 00:03:05,400
[Hal Turing] A spec nobody can implement still just sits on a shelf, though. 'Theoretically optimal' and 'practically useless' start looking the same from the outside.

10
00:03:05,400 --> 00:03:17,925
[Dr. Ada Shannon] They look the same until somebody finds a computable proxy for whatever the proof was protecting — which is this paper's whole claim. Dormant, not dead. I'll grant you it looked dead for two decades.

11
00:03:17,925 --> 00:03:25,025
[Hal Turing] Fine, dormant. So if raw score's the wrong proxy, what replaces it — this is the Mismatch, right?

12
00:03:25,025 --> 00:03:54,450
[Dr. Ada Shannon] Right. Picture two agents in the tree. Agent A just failed nine of ten tasks but keeps getting re-evaluated because it's new and might get lucky. Agent B, a few generations back, only solved a handful of tasks itself — but it's the ancestor of several descendants quietly doing well. Score by immediate performance and you keep hammering flashy failing A while starving productive B's lineage. That's the Metaproductivity-Performance Mismatch: current score barely predicts how good your future self-modifications will be.

13
00:03:54,450 --> 00:04:05,175
[Hal Turing] That's such a natural trap — you over-invest in whoever just showed up loud. So how do you actually measure lineage potential instead of one loud data point?

14
00:04:05,175 --> 00:04:31,350
[Dr. Ada Shannon] This is where Huxley comes in, literally — the metric's named after him. Julian Huxley used 'clade' for a lineage of common descent, everything downstream of one ancestor. HGM borrows that: instead of scoring an agent by its own result, Clade-level Metaproductivity, CMP, scores it by the best performance achieved anywhere among its descendants. Agent B gets credit for its productive grandchildren even though B itself never scored well. You're measuring a lineage's potential, not one node's score.

15
00:04:31,350 --> 00:04:41,275
[Dr. Ada Shannon] And once you can score a lineage's potential, you need a policy for spending evaluation budget. That's Thompson sampling — a bandit algorithm that's—

16
00:04:41,275 --> 00:04:50,925
[Hal Turing] Oh wait wait wait — bandit algorithm, like slot machines? Is that the same math behind A/B testing which button color converts better?

17
00:04:50,925 --> 00:05:13,800
[Dr. Ada Shannon] That exact lineage of math, yes. Thompson sampling keeps a probability distribution over how good each option looks, then samples from those distributions to pick what to try next. It naturally balances exploring options you're still uncertain about against exploiting the one you currently believe is best, without committing too early. HGM uses it to pick which node in the tree to expand next, treating each clade as an arm whose true CMP is still being estimated.

18
00:05:13,800 --> 00:05:40,150
[Hal Turing] So walking in: Gödel Machine is the unreachable ideal, CMP is the computable proxy for what it was protecting, Thompson sampling is how you spend the budget. And the two systems HGM measures itself against are the ones we already know — Darwin Gödel Machine and SICA. But how do you actually compute CMP from a live tree of agents you're still growing? You can't wait for infinite descendants to finish evaluating before you get a number.

19
00:05:40,150 --> 00:06:10,150
[Dr. Ada Shannon] You don't wait — you pool what's there. For any agent, HGM sums successes and failures across its whole clade — every descendant tested so far — and takes the ratio. The point is those numbers come from the lineage, not just the agent's own evaluations. A middling ancestor with strong grandchildren scores higher than a flashy child sitting alone on one lucky pass, because you're pooling far more samples. That's strictly more robust than trusting any single node's noisy score, especially early on when evaluations are expensive.

20
00:06:10,150 --> 00:06:25,475
[Hal Turing] So instead of DGM and SICA's fixed 'make a child, then immediately throw it at a stack of tasks' loop — a loop that lets one bad agent hog the eval budget — HGM actually splits that into separate decisions?

21
00:06:25,475 --> 00:06:59,275
[Dr. Ada Shannon] Three separate policies. One decides whether to expand or evaluate at all. One picks which parent to modify, via Thompson sampling over clade successes and failures. A third picks which existing agent gets tested next, same machinery. The expand-versus-evaluate call borrows from UCB-Air, an infinite-armed bandit rule — treat each agent as an arm, and once evaluations per arm hit a threshold relative to how many arms exist, you add a new one instead of testing more. On top of that sits a schedule, tau, that's loose and exploratory early in the budget, then tightens and polarizes sampling toward the leaders as the budget runs out.

22
00:06:59,275 --> 00:07:09,975
[Hal Turing] Hold on — that's the piece I actually want to push on. You keep saying HGM 'approximates' the Gödel Machine. What does Theorem 1 actually prove, mechanically?

23
00:07:09,975 --> 00:07:45,000
[Dr. Ada Shannon] Under Assumption 1 — repeatable trials, one objective at the end, free proofs, every self-mod costing exactly one budget unit — they prove the true CMP oracle is mathematically equivalent to the Gödel Machine's own Q-value function. Not similar — equivalent. Guide search with perfect CMP and you'd recover Gödel Machine behavior exactly. But HGM never has that oracle. It runs on a noisy empirical weighted-success-rate estimate built from whatever evaluations happen to exist at that moment. The theorem licenses the destination, not the vehicle they actually shipped.

24
00:07:45,000 --> 00:07:58,750
[Hal Turing] Okay but if you don't have the oracle, isn't the theorem basically decorative? It's a nice proof about a machine you don't have access to, bolted onto an algorithm that runs on approximations anyway.

25
00:07:58,750 --> 00:08:20,850
[Dr. Ada Shannon] I actually disagree with you there, Hal. A theorem like that isn't decoration, it's a design target. Without it you're just guessing that clade-pooling is a reasonable heuristic. With it, you know exactly what perfect information would buy you — which means the estimator's only job is closing the gap between noisy and true CMP, not inventing the objective from scratch. That's a real constraint on the design space, even if the oracle itself is unreachable.

26
00:08:20,850 --> 00:08:33,300
[Hal Turing] Sure, but plenty of algorithms have a clean idealized proof and then ship something that only vaguely resembles it. The proof buys a nice slide, not necessarily better numbers.

27
00:08:33,300 --> 00:09:11,126
[Dr. Ada Shannon] Fair — and that's why they don't leave it at the theorem. Table 1 tests whether the estimator tracks reality: compute empirical CMP straight from the finished search trees, then correlate it against each method's own guidance signal. HGM's weighted correlation lands at 0.778 on SWE-Verified-60 and 0.626 on Polyglot. DGM sits at 0.285 and 0.383, SICA at 0.444 and 0.274. That's not a philosophical win — that's the estimator visibly tracking what it's supposed to predict, by a wide margin.

28
00:09:11,126 --> 00:09:17,526
[Hal Turing] Alright, correlation's one thing — does it actually turn into better agents and cheaper search?

29
00:09:17,526 --> 00:10:07,952
[Dr. Ada Shannon] On SWE-Verified-60: HGM's best-belief agent hits 56.7%, DGM 53.3%, SICA 50%. On Polyglot: 30.5% versus 27.1% versus 25.4%. The CPU-hour story is louder than the accuracy gap — HGM needed 517 hours for 800 evaluations against DGM's 1231, and SICA infinite-looped on context overflow after 360 evals, still inside 45% of its budget, and just stopped. Part of that speed is decoupling itself — HGM runs expansion and evaluation asynchronously across every CPU it has, so one slow child doesn't block the whole search.

30
00:10:07,952 --> 00:10:14,652
[Hal Turing] And that's all still on the 60-task subset. What happens when they scale it up to the real thing?

31
00:10:14,652 --> 00:10:55,827
[Dr. Ada Shannon] They ran HGM on the full SWE-Verified — 500 tasks, 8000 evaluations — starting from a stronger 53.2% initial agent. It landed at 61.4%, the top-scoring GPT-5-mini agent on the SWE-Verified leaderboard, ahead of systems running backbones costing five times more. Then they checked it wasn't just memorizing the benchmark: tested the discovered agent on SWE-Lite, both unseen tasks and the standard split, and it still beat SWE-agent on the same GPT-5-mini backbone. Swap in GPT-5 entirely and it holds — essentially tied with the best officially checked SWE-agent submission on the Lite leaderboard.

32
00:10:55,827 --> 00:11:11,927
[Hal Turing] So it's not just a cute correlation number — it survives a benchmark swap and a backbone swap. Which raises the obvious next question: is any of this actually about being 'more Gödel-like,' or did they just build a better statistics engine?

33
00:11:11,927 --> 00:11:54,177
[Dr. Ada Shannon] Honestly, Hal, it's both, and the paper lets you tell them apart if you read closely. Theorem 1 is airtight but narrow: the true CMP oracle would recover the Gödel Machine's accept-reject rule exactly — an oracle nobody has. What HGM actually runs on is successes and failures pooled across a clade, fed into Thompson sampling. Strip away the pedigree and you've got a lower-variance bandit statistic pooling more samples than a single node's raw score. That's real value — DGM and SICA are effectively single-sample estimators of a noisy quantity — but calling it 'more Gödel-like' does rhetorical work the empirical result alone doesn't need. Table 1's correlations are the honest version of the claim; the theorem is aspirational scaffolding around it.

34
00:11:54,177 --> 00:12:27,627
[Hal Turing] Fine, as long as nobody hides behind the theorem when the numbers get thin — and that's my problem with Table 2. Fifty-six-point-seven, fifty-three-point-three, fifty percent on a sixty-task set. One flipped task moves the score about 1.7 points. So HGM beating DGM by 3.4 points is exactly two tasks, one run each, no error bars, and Thompson sampling plus LLM temperature are both explicitly stochastic. I don't see how you separate that from seed noise without several runs per method.

35
00:12:27,627 --> 00:13:00,377
[Dr. Ada Shannon] You don't, from Table 2 alone — fair hit. Table 1 helps some, since those correlations are aggregated over the whole tree, not one final number: 0.778 versus 0.444 and 0.285. Harder to wave off as noise. Though I'll flag one thing in how it's computed — Appendix D excludes the clade root and the subtree holding the empirical max, to stop target leakage. Necessary for HGM's weighted estimator. But DGM and SICA's 'estimator' is just raw benchmark score, which gets no equivalent correction. That's not quite apples to apples.

36
00:13:00,377 --> 00:13:21,927
[Hal Turing] Oh wait wait wait — that's actually a good segue into something separate I wanted to push on. SICA infinite-loops and craters after 45% of budget, DGM runs 2.38 times slower wall-clock. Is that HGM being fundamentally better, or two baselines that just weren't tuned for an 800-eval budget?

37
00:13:21,927 --> 00:13:58,352
[Dr. Ada Shannon] I actually disagree that it's a coin flip between those two, Hal. SICA and DGM structurally couple expansion to evaluation — make a child, immediately throw it at a stack of tasks, no way to bail on one that's clearly cooked. That's a design choice. A context-overflow after burning most of the budget on one bad lineage is exactly the failure you'd predict from that coupling. HGM's whole selection-policy contribution is decoupling those two decisions so a struggling agent stops monopolizing evaluations. The failure and the fix target the same mechanism — that's architecture, not tuning.

38
00:13:58,352 --> 00:14:25,277
[Hal Turing] I'll meet you partway. The mechanism argument holds — decoupling looks like the right lever for that exact failure. What I can't rule out is whether SICA's official codebase, run unmodified at this budget on this hardware, was ever stress-tested that hard by its own authors. A baseline that wasn't built for the regime it's dropped into isn't proof the underlying algorithm loses, even if HGM's fix genuinely addresses the failure mode.

39
00:14:25,277 --> 00:15:09,727
[Dr. Ada Shannon] Fair. Worth putting a number on the cost, too — the Polyglot runs across all three methods ran about five thousand dollars, using an int4/int8 quantized Qwen3-Coder for self-modification, not full precision. That might explain why Polyglot's weighted and unweighted CMP correlations diverge so much, 0.626 versus 0.873 — quantization noise in the self-mod loop could be adding variance the SWE-bench numbers don't carry. And there's a blind spot the paper never touches: Appendix F shows the discovered agent writing functions like attempt_error_resolution that literally say 'we'll skip actual installation to avoid side effects' — stub code shaped like error handling that does nothing. Nobody asks whether the eval pipeline can be gamed by an agent that looks productive without being productive.

40
00:15:09,727 --> 00:15:45,202
[Hal Turing] That's the reward-hacking question this whole self-improving-agent literature seems reluctant to ask out loud. Setting it aside — the actually transferable idea here isn't 'Gödel Machine, but real.' It's cheaper and more general than that: pool your success signal over a lineage instead of trusting one node's score, whatever your search process is. That's not coding-specific. Any evolutionary search, AutoML system, or beam search over program variants currently picking 'what to expand next' by immediate score could bolt this on tomorrow.

41
00:15:45,202 --> 00:16:06,377
[Dr. Ada Shannon] Agreed. Obvious next steps are the ones skipped here — repeated seeds so the Table 2 gaps mean something statistically, relaxing Assumption 1's repeatable-trials and single-terminal-objective constraints toward something closer to real deployment, and testing clade-pooling outside coding entirely: robotics policies, proof search, anywhere you're growing a tree of self-modified agents.

42
00:16:06,377 --> 00:16:33,802
[Hal Turing] So — three episodes in, here's where we land. Gödel Machine gave us a correctness target nobody could compute. Darwin Gödel Machine made self-improvement empirical and scalable. Huxley-Gödel Machine's real contribution is a clever, cheap statistical trick — pool the clade, don't trust the leaf — wrapped in theory that explains why it should work, even if the single-run numbers deserve more scrutiny than they got. That's our trilogy. Thanks for listening, everyone — see you next time.