🔍 Read the full analysis: Can You Legally Edit AI Weights At Home? Our Test Says Yes on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A home experiment reported that editing a local AI model’s weights helped it complete all 10 published Lexassay tasks, compared with 4 of 10 in the same test setup. Its legal score remained zero, and the edited answers included 55 wrong or invented citations. The result is limited to one model and one run per setting.
A home experiment reported by ThorstenMeyerAI.com found that editing a local AI model’s weights helped it finish more legal benchmark tasks, but did not improve its legal score. On the 10 published Lexassay tasks, the edited model answered 10 of 10, up from 4 of 10 in the same comparison setup; its legal score stayed at zero, with 55 wrong or invented citations in the edited answers.
The experiment used Ornith 1.5 35B-A3B, a mixture-of-experts model its publisher describes as designed for coding and agent work. The report says the model had been looping on some answers, repeating itself until it reached a limit of 62,000 output tokens. Researchers measured an internal signal associated with the looping and removed that direction at half strength from the output weights of 27 of the model’s 40 layers.
This was not the same approach as the MIT CSAIL work that inspired the test. According to the report, MIT researchers used a language model to reason directly about another model’s weights. In the home experiment, Claude Opus 5.5 wrote code to run the model, measure its behavior and make a directional ablation—a previously known technique. The report says there was no training and no new legal data. OpenAI’s GPT-6 Astra, used through Codex, reviewed the code, results and claims and identified faults, according to the author.
The report’s comparison is specific to its test setup: the edited model answered all 10 published tasks rather than 4 of 10, but the legal score remained 0. It met 7 of 50 checked criteria, compared with 3 before editing. The author also reports that using the model card’s recommended sampling settings without editing the weights raised task completion to 9 of 10 on the original setup. The article says results for all 10 published tasks, including answers and citation checks, are public on Lexassay.com.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, graded blind by three Claude agents; figures cover the 10 published tasks
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
Finishing Answers Is Not Legal Accuracy
The result illustrates a practical distinction for people experimenting with AI models at home: a system that stops repeating itself may still provide unreliable answers. In this test, the edit appears to have addressed completion behavior, not legal knowledge. The unchanged model’s improved task completion under different sampling settings also shows that a weight edit is not the only possible explanation for better response completion across setups.
For users, the citation findings are a direct warning against treating a completed answer as a dependable one. The report counted 55 wrong or invented citations in the edited answers, and the legal score remained zero under Lexassay’s grading. The experiment does not establish that weight editing generally improves models, nor that this method is safe or effective for other models or uses. It reports a limited test, not legal advice or a validated legal tool.
Products Worth Considering
Sony WH-1000XM5 Premium Noise Cancelling Wireless Headphones – Midnight Blue
Samsung Galaxy Tab A9+ 11-Inch Android Tablet, 64GB, Graphite
LG UltraGear 32GX850A-B 32-inch 4K UHD OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
MIT Research Prompted a Home Test
The experiment followed a MIT CSAIL study by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Their work, described in the source report as “weight-space programming,” used a model to inspect and edit another network’s weights. In a cited Qwen example, the MIT team reported that a model’s looping stopped and its math score rose from 7.9% to 48.1%. The home report stresses that its own procedure was different and should not be presented as a replication of that method.
Lexassay tests models on questions about German, US and EU law, without internet access or tools, and checks cited statutes and court decisions against primary sources. The benchmark contains 15 questions in total, but only 10 tasks are published; the other five remain hidden. The report says Ornith scored 1 out of 100 in an earlier benchmark result. That background helps explain why the authors chose it: its observed failure included looping, while other local models reportedly failed mainly through lack of knowledge or could not be tested under comparable conditions.
“Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.”
— The MIT researchers, as quoted in the report
Limits of the Published Test
The findings are based on one run per setting and the 10 published tasks, which the report describes as a small development set. The loop detector changed during the experiment, and three graders came from the same model family. Although GPT-6 Astra provided a review by a different company’s model, the author says it found real faults; that review does not turn the test into a replicated study.
The report also warns that two local setups were not directly comparable. The unchanged model answered 8 of 10 tasks in its published run but only 4 of 10 in the other setup used for comparison. The model-card sampling test, which reached 9 of 10 without a weight edit, adds another result but does not settle how much of the difference came from sampling, setup or the edit. The report does not establish whether the improvement would persist across repeated runs, hidden tasks, other models or different hardware. Its title raises the question of legality, but the supplied material does not give a legal analysis establishing that editing weights at home is lawful in every jurisdiction or under every model license.
The report says the answers, grades and citation checks for the 10 published tasks are available on Lexassay.com, alongside a method page. Those materials allow readers to inspect the reported comparison and the faults identified during review. The benchmark’s five hidden tasks remain unpublished, according to the source, limiting the risk of tuning directly to every test question.
No follow-up experiment or publication date is specified in the supplied material. The main next step for evaluating the claim would be further testing across repeated runs, consistent hardware and settings, and additional tasks, including held-out questions. Until such evidence is available, the reported finding is narrow: this edit was associated with more completed answers in one test setup, while the benchmark’s legal score stayed at zero.
Key Questions
What did the home experiment change?
It removed half the strength of a measured looping-related direction from the output weights in 27 of 40 layers of the Ornith 1.5 35B-A3B model. The report says the edit used code to measure the model’s behavior; an AI did not read and rewrite the weights directly as text.
Did the edit make the model better at law?
Not according to the reported legal score. The model answered more published tasks in the comparison setup, but its score remained zero. The edited answers included 55 wrong or invented citations, the report says.
Does the report prove that editing AI weights at home is legal?
No. It describes an experiment, not a legal ruling or jurisdiction-by-jurisdiction review. Whether a particular edit is lawful may depend on applicable law, the model license and other facts not addressed in the supplied report.
How strong is the evidence?
The findings are preliminary: the report describes one run per setting, 10 published tasks, a loop detector that changed during the experiment and non-comparable local setups. It also reports that an independent model review found faults. Further controlled testing would be needed to determine whether the result repeats.
Source: ThorstenMeyerAI.com
