🔍 Read the full analysis: The Case For Weight-Space Programming—and Its Limits on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra identified, repaired and created small neural networks by reasoning directly about their weights. The results come from a November 2 blog post, not a peer-reviewed paper, and the tests were limited; the method also failed to distinguish honest from sycophantic behavior and did not improve every model.
Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory say OpenAI’s GPT-6 Astra can identify, repair and create small neural networks by reading and writing their weights directly, rather than using conventional training. The team’s November 2 blog post reports striking results in several tests, but the work is not a peer-reviewed paper, and its limited experiments do not establish that the approach works broadly.
The researchers call the approach weight-space programming: a language model receives the numerical weights of another network and reasons about what they do. In some tests, Astra analyzed weights presented as text and responded in one turn. In others, it wrote Python code that the researchers ran in a restricted sandbox. The team says Astra had no shell, web search or file-editing tools, and that it checked all 1,125 turns in the released logs for tool use.
Among the reported results, Astra identified the games behind four of four small game-playing networks and the shapes represented by three small image generators. It also located copying mechanisms, known as induction heads, in three tiny language models. The researchers say it repaired a car-control network that had recorded no successful reversals in 256 episodes: after editing 3.5% of its weights, the network succeeded in 1,024 of 1,024 episodes.
In another test, an edit affecting 0.016% of Qwen3.5-0.8B’s weights raised its MATH-500 score from 7.9% to 48.1%, according to the post. A Hugging Face engineer later rebuilt the edited model from the logs and reported a 51.0% score. Astra also produced a 9-by-9 Go network from zero-valued weights; after 50 test runs, it beat a same-size comparison network trained on about 20.5 million games. These are results reported by the researchers and a reproducer, not evidence of general performance across models or tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Model Repair
If the results hold up in broader testing, weight-space programming could offer a way to inspect and make targeted changes to small models without repeating an expensive training process. That could matter to teams maintaining open models: a narrowly focused edit might address a specific failure while leaving most of the network unchanged. The reported Qwen result is an example of that possibility, though it does not establish that similar edits will reliably improve other models or benchmarks.
The approach also raises a different question: whether a language model can infer meaningful behavior from another system’s internal parameters. That could support auditing and debugging, but it should not be treated as a substitute for testing a model’s actual behavior. The post itself describes a key limit: Astra could not tell an honest model from a sycophantic one by examining weights alone. For users, the research does not change how AI products work today; its immediate relevance is to model developers and evaluators.
Products Worth Considering
Logitech MX Mechanical Wireless Illuminated Keyboard, Clicky
Logitech MX Brio Ultra HD 4K Webcam, Graphite
Amazon Kindle Paperwhite Signature Edition, 32GB (2024), Metallic Black
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Edits
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in learned weights rather than written line by line by a programmer. He also noted that directly coding in weights was difficult. In standard machine learning, an optimizer adjusts those weights repeatedly during training, usually in response to data and a learning objective.
The MIT team’s proposal differs by asking a general-purpose language model to reason over existing weights and make edits, rather than relying on gradient descent for the specific task. The researchers—Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab—published the experiments as a blog post with logs on November 2. The source describes a single research group, small networks and one run per setting; it says the post is not on arXiv and has not been peer reviewed.
“not through an optimizer, and in particular not with gradient descent”
— The MIT CSAIL researchers, in their November 2 blog post
How Far the Tests Generalize
The evidence remains preliminary. The source describes a blog post rather than peer-reviewed research, with one run per setting, small networks and only one model family per task. The experiments do not show whether the results will repeat across independent research groups, larger models, different architectures or practical production systems. The reported math-score changes also need careful interpretation: the source does not establish how the edit affects performance beyond the named benchmark.
Some results may have simpler explanations than the most ambitious framing suggests. The source notes that software engineer Sean Goedecke reproduced a visualization used to identify a game-playing network with a short Python script. That observation does not by itself invalidate Astra’s result, but it highlights the need to compare the model’s reasoning with simpler analysis methods. The post also reports failures: Astra did not distinguish an honest model from a sycophantic one, and editing Gemma 3 270M, which did not loop, did not improve its math performance.
Replication and Broader Testing
The next test is whether independent researchers can reproduce the reported repairs and network-writing results across more models and tasks. The public logs and the reported Qwen replication provide material for scrutiny, but the source gives no announced schedule for a peer-reviewed paper, further experiments or a broader evaluation. More evidence would be needed to establish reliability, compare the method with existing debugging and training approaches, and measure the cost of using it.
Until then, the findings are best treated as an early demonstration, not a ready-made tool for model deployment. Developers considering weight edits would still need to validate performance on relevant tests and check for regressions. The researchers’ own reported failures indicate that reading weights alone may not reveal important behavioral properties or supply capabilities a model does not already have.
Key Questions
What is weight-space programming?
It is the approach described by the MIT researchers in which a language model reads or changes another neural network’s numerical weights by reasoning about them, rather than using gradient descent for that task.
What did GPT-6 Astra do in the reported tests?
According to the researchers, Astra identified small networks’ functions, repaired a car-control network, edited a small language model’s weights and created small game-playing networks, including a 9-by-9 Go network.
Has the research been peer reviewed?
No. The source describes the work as a blog post with public logs, not a peer-reviewed paper, and says it is not on arXiv.
What are the main limitations so far?
The tests involved small networks, one run per setting and a single research group. The researchers also report that Astra could not identify sycophancy from weights alone and did not improve a model that was not already exhibiting the targeted looping behavior.
Does this change how AI products work now?
The reported work does not establish a change to current consumer AI products. It is an early research demonstration, and broader independent testing would be needed before its reliability or practical use is clear.
Source: ThorstenMeyerAI.com
