A Closer Look At Inkling 2 Mini And Local AI Performance
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: A Closer Look At Inkling 2 Mini And Local AI Performance on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release includes support for several inference runtimes and a research version with adjustable repeated-layer passes. Benchmark results and practical hardware requirements have not been independently established in the supplied material.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model aimed at local AI use that the company describes as having about 26 billion total parameters and activating about 3 billion per token. The release adds support for several inference runtimes and a research checkpoint with adjustable layer passes, but its benchmark claims and practical hardware needs have not been independently verified in the supplied material.

According to the release, Inkling 2 Mini accepts text and image input and has a claimed 1 million-token context window. Its architecture has 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers are reused in three passes, producing 48 logical layer visits per token. The company says weight reuse raises capacity without requiring separate weights for every logical layer.

The model uses a mixture-of-experts design: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says the experts use a SwiGLU design. The middle layers’ repeated passes share information through learned messages, but the technical description says the model still executes all 48 visits and retains state for each visit. The release does not provide a complete model file size or minimum hardware configuration.

The fixed-pass version is listed for Transformers, llama.cpp and MLX, with SGLang support for datacenter inference. A separate research artifact, TML-d24-varloop-research, allows users to adjust the number of passes; the company says fewer passes may improve speed at a possible cost to quality. The release also describes NVFP4 and MXFP8 expert weights and a DSpark drafter intended to speed generation with a small memory cost.

At a glance
announcementWhen: Announced November 2026; independent te…
The developmentThinking Machines Lab released Inkling 2 Mini, along with runtime support and a research checkpoint that allows its repeated-layer passes to be adjusted.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

What Local AI Users Can Expect

Inkling 2 Mini gives people evaluating local models another option that combines image input, a claimed long context window and support for several established inference tools. Those features may make it relevant to developers and users who want to run AI on their own systems rather than rely entirely on hosted services. However, the release alone does not establish which consumer devices can run it comfortably or how responsive it will be.

The architecture’s weight reuse and selective expert routing are design choices, not proof of a particular memory or speed advantage in everyday use. The model still performs 48 logical layer visits per token, according to the technical description, and each visit retains state. Readers choosing hardware or comparing models will need measured memory use, generation speed and output quality on specific setups—not only parameter counts and company benchmark summaries.

The adjustable-pass research checkpoint also raises a practical trade-off for evaluation: fewer passes could change both latency and quality. The company’s claim that this configuration can speed inference should be treated as a reported expectation until tests show how much it changes performance across tasks and hardware.

Products Worth Considering

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, White Smoke

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, White Smoke

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

As an affiliate, we earn on qualifying purchases.

How Inkling Reuses Its Layers

The release distinguishes physical layers—the model’s distinct sets of layer weights—from logical layer visits during computation. Inkling 2 Mini has 24 physical decoder layers, but its 12 middle layers run three times. The company says learned messages carry information between passes, with a learned gate controlling how much information is passed forward.

Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as executing 48 layer visits per token, but Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material says Mini is substantially smaller but gives no complete file size. It also notes that other work, including Nanbeige architectures, has explored looping; the release does not provide a detailed comparison with those systems.

These details help explain the model’s design, but they do not settle how it compares in practice. Parameter totals, active parameters per token and layer visits measure different aspects of a model, and none on its own supplies a full account of runtime speed or system memory requirements.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Benchmark and Hardware Questions

Thinking Machines reports competitive results, including a lead in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0. These are company-reported comparisons; the supplied material does not include a full score table or independent replication.

It remains unclear how much memory the full model requires, what generation speeds users can expect on consumer hardware, and how well image and long-context performance hold up under independent testing. The source material also does not fully assess the research checkpoint’s quality at one or two passes. The full model card may include more detail, but it was not provided for this report.

Independent Tests and Model Details

The next evidence to watch for is the full model card, reproducible benchmark results and hands-on reports that identify the hardware and runtime used. Those details would help distinguish performance across tasks from results on any single benchmark and clarify whether the model is practical on local machines.

Users testing the available runtimes can also report memory use, generation speed and output quality, including how results change with image inputs, long prompts and fewer layer passes. Until those measurements are available, claims about performance and the benefits of the adjustable research version should remain attributed to the company rather than treated as independently confirmed findings.

Key Questions

What is Inkling 2 Mini?

It is a multimodal model released by Thinking Machines Lab. The company describes it as accepting text and image input, with about 26 billion total parameters and about 3 billion activated per token.

Can Inkling 2 Mini run locally?

The release lists support for Transformers, llama.cpp and MLX, which are used for inference setups, and SGLang support for datacenter deployments. The supplied information does not state a minimum consumer hardware configuration or complete memory requirements.

How does its repeated-layer architecture work?

Inkling 2 Mini has 24 physical decoder layers. Its 12 middle layers run three times, giving 48 logical layer visits per token, according to the release. The company says learned messages carry information between passes.

Are the reported benchmark results independently verified?

Not in the material supplied for this report. Thinking Machines reports results across several benchmarks, but the full score table and independent replication were not included.

What is the adjustable-pass research checkpoint?

TML-d24-varloop-research is a separate artifact that allows the number of repeated middle-layer passes to be changed. The company says fewer passes may improve speed with a possible quality trade-off; independent measurements of that trade-off were not provided.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

Need A Graphics Card For A 2027 PC? Here Are 10 To Compare

A buyer-focused roundup compares 10 Radeon, GeForce and Intel graphics cards by GPU tier, memory, cooling and fit—without claiming unprovided benchmark results.

Do Games Load Faster On A PCIe 5.0 SSD? My Upgrade Says No

An XDA writer reports no noticeable gaming load-time improvement after moving from a PCIe 4.0 to a PCIe 5.0 SSD.

The 2026 AI Story So Far: Ten Monthly Chapters

A month-by-month recap through October 2026 traces AI agents, market disruption, security concerns, falling prices and unresolved claims.

Apple Watch Series 12 Vs. Ultra 4 Buyer’s Guide

Comparison guide between Apple Watch Series 12 and Ultra 4, highlighting key features, differences, and what buyers should consider.