🔍 Read the full analysis: A Closer Look At Inkling 2 Mini And Local AI Performance on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release includes support for several inference runtimes and a research version with adjustable repeated-layer passes. Benchmark results and practical hardware requirements have not been independently established in the supplied material.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal model aimed at local AI use that the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release adds support for several inference runtimes and a research checkpoint with adjustable layer passes, but its benchmark claims and practical hardware needs have not been independently verified in the supplied material.
According to the release, Inkling 2 Mini accepts text and image input and has a claimed 1 million-token context window. Its architecture has 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers are reused in three passes, producing 48 logical layer visits per token. The company says weight reuse raises capacity without requiring separate weights for every logical layer.
The model uses a mixture-of-experts design: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says the experts use a SwiGLU design. The middle layers’ repeated passes share information through learned messages, but the technical description says the model still executes all 48 visits and retains state for each visit. The release does not provide a complete model file size or minimum hardware configuration.
The fixed-pass version is listed for Transformers, llama.cpp and MLX, with SGLang support for datacenter inference. A separate research artifact, TML-d24-varloop-research, allows users to adjust the number of passes; the company says fewer passes may improve speed at a possible cost to quality. The release also describes NVFP4 and MXFP8 expert weights and a DSpark drafter intended to speed generation with a small memory cost.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
What Local AI Users Can Expect
Inkling 2 Mini gives people evaluating local models another option that combines image input, a claimed long context window and support for several established inference tools. Those features may make it relevant to developers and users who want to run AI on their own systems rather than rely entirely on hosted services. However, the release alone does not establish which consumer devices can run it comfortably or how responsive it will be.
The architecture’s weight reuse and selective expert routing are design choices, not proof of a particular memory or speed advantage in everyday use. The model still performs 48 logical layer visits per token, according to the technical description, and each visit retains state. Readers choosing hardware or comparing models will need measured memory use, generation speed and output quality on specific setups—not only parameter counts and company benchmark summaries.
The adjustable-pass research checkpoint also raises a practical trade-off for evaluation: fewer passes could change both latency and quality. The company’s claim that this configuration can speed inference should be treated as a reported expectation until tests show how much it changes performance across tasks and hardware.
Products Worth Considering

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, White Smoke

Samsung Galaxy Tab A11+ Kids Edition, 128GB, Royal Blue

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)
As an affiliate, we earn on qualifying purchases.
How Inkling Reuses Its Layers
The release distinguishes physical layers—the model’s distinct sets of layer weights—from logical layer visits during computation. Inkling 2 Mini has 24 physical decoder layers, but its 12 middle layers run three times. The company says learned messages carry information between passes, with a learned gate controlling how much information is passed forward.
Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as executing 48 layer visits per token, but Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material says Mini is substantially smaller but gives no complete file size. It also notes that other work, including Nanbeige architectures, has explored looping; the release does not provide a detailed comparison with those systems.
These details help explain the model’s design, but they do not settle how it compares in practice. Parameter totals, active parameters per token and layer visits measure different aspects of a model, and none on its own supplies a full account of runtime speed or system memory requirements.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Benchmark and Hardware Questions
Thinking Machines reports competitive results, including a lead in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0. These are company-reported comparisons; the supplied material does not include a full score table or independent replication.
It remains unclear how much memory the full model requires, what generation speeds users can expect on consumer hardware, and how well image and long-context performance hold up under independent testing. The source material also does not fully assess the research checkpoint’s quality at one or two passes. The full model card may include more detail, but it was not provided for this report.
Independent Tests and Model Details
The next evidence to watch for is the full model card, reproducible benchmark results and hands-on reports that identify the hardware and runtime used. Those details would help distinguish performance across tasks from results on any single benchmark and clarify whether the model is practical on local machines.
Users testing the available runtimes can also report memory use, generation speed and output quality, including how results change with image inputs, long prompts and fewer layer passes. Until those measurements are available, claims about performance and the benefits of the adjustable research version should remain attributed to the company rather than treated as independently confirmed findings.
Key Questions
What is Inkling 2 Mini?
It is a multimodal model released by Thinking Machines Lab. The company describes it as accepting text and image input, with about 26 billion total parameters and about 3 billion activated per token.
Can Inkling 2 Mini run locally?
The release lists support for Transformers, llama.cpp and MLX, which are used for inference setups, and SGLang support for datacenter deployments. The supplied information does not state a minimum consumer hardware configuration or complete memory requirements.
How does its repeated-layer architecture work?
Inkling 2 Mini has 24 physical decoder layers. Its 12 middle layers run three times, giving 48 logical layer visits per token, according to the release. The company says learned messages carry information between passes.
Are the reported benchmark results independently verified?
Not in the material supplied for this report. Thinking Machines reports results across several benchmarks, but the full score table and independent replication were not included.
What is the adjustable-pass research checkpoint?
TML-d24-varloop-research is a separate artifact that allows the number of repeated middle-layer passes to be changed. The company says fewer passes may improve speed with a possible quality trade-off; independent measurements of that trade-off were not provided.
Primary source: Hugging Face · via ThorstenMeyerAI.com
