🔍 Read the full analysis: How Kimi K3.1 Pro Stacks Up Against The Frontier on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Moonshot AI launched Kimi K3.1 Pro on October 23, and Artificial Analysis ranked it fourth on its Intelligence Index with a score of 55.1. The evaluation puts it close to leading proprietary models and shows a sharp gain on an agentic coding benchmark, but the weights are not yet available, the licence is unpublished, and several prominent capability claims come from Moonshot’s own tests.
Moonshot AI launched Kimi K3.1 Pro on October 23, with independent testing by Artificial Analysis placing the model fourth on its Intelligence Index, close to leading proprietary systems. The results strengthen the case that an open-weight model could approach the closed-model frontier, but K3.1 Pro’s weights are only promised for November 4 and its licence has not been published.
Artificial Analysis gave K3.1 Pro an Intelligence Index score of 55.1, behind Claude Opus 5.5 at 57.6, Claude Sonnet 5.5 at 56.0 and Muse Wonder Full at 55.8. The evaluation used each model’s highest reasoning setting where available. The result is close to the leaders, but it does not mean K3.1 Pro outperformed them overall. Artificial Analysis said the model would lead open-weight systems once its weights are published; that remains conditional on their release.
The clearest benchmark jump was on Terminal-Bench 4.0, which tests agentic computer tasks: K3.1 Pro scored 67%, compared with 13% for the earlier Kimi K3 in Artificial Analysis’s results. On GDPval-AA, a simulated knowledge-work evaluation, it scored 1,846 Elo, behind Opus 5.5 at 1,866 and Muse Wonder Full at 1,857. Its Humanity’s Last Exam score was 52%, compared with 47% for K3 and 61% for Opus 5.5.
Artificial Analysis lists the model as a 2.8-trillion-parameter mixture-of-experts system, with 104 billion active parameters, a one-million-token context window and text and image input. Moonshot made it available through Kimi apps, Kimi Work, Kimi Code and its API. Listed API pricing is $2 per million input tokens and $10 per million output tokens. Artificial Analysis estimated $2.11 per Intelligence Index task for K3.1 Pro, against $5.98 for Opus 5.5 at maximum effort. Those task-cost estimates reflect the evaluation’s usage and are not a guarantee of cost for every workload.
Infographic 01 · Claimed vs. measured
Moonshot says “matches the strongest.” Independent tests say: close.
The KPIs at the top are independent measurements. Vendor numbers and independent numbers come from different harnesses, so they are shown side by side, never mixed.
Moonshot reports
vendor claims own harness, partly internal tests
| Benchmark | K3.1 Pro | Comparison |
|---|---|---|
| Terminal-Bench 4.0 | 65.7% | vs. Claude Opus 5.5: 66.4% |
| BenchCAD with tools | 92.7% | vs. Claude Opus 5.5: 96.2% |
| OfficeQA Pro v2 | 64.4 | vs. Claude Opus 5.5: 62.2 |
| Kimi Hallu Bench Hard internal | 8.3% | vs. Kimi K3: 23.4% (about −65%) |
Artificial Analysis measured
independent highest reasoning setting per model
| Measurement | K3.1 Pro | Comparison |
|---|---|---|
| Terminal-Bench 4.0 | 67% | vs. Claude Opus 5.5 (max): 60% |
| GDPval-AA | 1,846 Elo | vs. Claude Opus 5.5: 1,866 |
| Humanity’s Last Exam | 52% | vs. Claude Opus 5.5: 61% |
| AA-Omniscience hallucination rate | 44% | vs. Kimi K3: 53% · Gemini 4 Argon: 15% |
The 65% hallucination drop holds on Moonshot’s own stress test. On an independent knowledge test, the rate fell from 53% to 44%.Moonshot: the stress test rates “do not represent overall error rates in daily use.”
Sources: Moonshot AI, Kimi K3.1 Pro tech blog (23 Oct 2026) (left card, vendor charts); Artificial Analysis launch article (23 Oct 2026) and Artificial Analysis model page (data as of 24 Oct 2026) (KPIs and right card).
Powered by Thorsten Meyer AI
A Smaller Gap, With Conditions
The scores matter because they suggest a smaller performance gap between a potentially downloadable model and systems available only through a provider. If Moonshot releases the weights and makes them usable under a suitable licence, organisations could gain more control over deployment, data handling and model access. But benchmark proximity is not the same as operational equivalence: results vary by task, and a model’s real-world value depends on reliability, latency, support and total running costs.
Cost comparisons also need care. Artificial Analysis’s estimate puts K3.1 Pro below Opus 5.5 per benchmark task, but the model’s listed API rates are not the lowest in the comparison. The open-weight models MiMo-V2.6-Pro and DeepSeek V4.1 Flash were substantially cheaper per Intelligence Index task in the same table, while scoring lower overall. For companies weighing a model, the relevant choice is not simply “frontier or cheaper”; it includes performance on their own work, deployment expenses and the terms attached to the weights.
For now, the “open-weight” distinction is prospective. The API and apps are available, but users cannot yet independently inspect or host K3.1 Pro’s weights. The promised release and licence will determine whether the benchmark result translates into broader control for developers and businesses.
AI model development workstation GPU 24GB VRAM
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From Kimi K3 to K3.1 Pro
K3.1 Pro is presented as a more heavily post-trained version of Kimi K3, rather than a wholly separate model family. Artificial Analysis’s comparison shows the largest reported change in agentic performance: Terminal-Bench 4.0 rises from 13% to 67%. Other gains are more limited in the cited evaluations, including a five-point increase on Humanity’s Last Exam, from 47% to 52%.
On the day before the launch, Artificial Analysis listed Xiaomi’s MiMo-V2.6-Pro at 46.3 as its highest-scoring open-weight model in the comparison. K3.1 Pro’s 55.1 is a sizeable lead over that score, while remaining below the three higher-ranked proprietary models. The distinction between current availability and future access is central: the scores are available now; the K3.1 Pro weights are not.
Moonshot also publicised examples involving game development, Blender and CAD workflows, financial tasks and hallucination reduction. These are vendor-reported demonstrations and internal evaluations, not all independently replicated findings. In one company stress test, Moonshot reported claim-level hallucinations falling from 23.4% for K3 to 8.3% for K3.1 Pro. Artificial Analysis measured a decrease from 53% to 44% on its separate AA-Omniscience evaluation. The figures come from different tests and should not be treated as directly interchangeable or as everyday error rates.
“The test is deliberately crafted to provoke errors.”
— Moonshot AI
NVIDIA RTX GPU for local large language models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Weights, Licence and Real-World Results
The weights have not been released, and the November 4 date is Moonshot’s stated target rather than a completed release. The source material does not provide a published K3.1 Pro licence, so prospective users cannot yet assess what commercial, redistribution or modification terms would apply. It is also unclear whether the release will include everything needed to reproduce the hosted service’s performance.
Independent testing supports a strong overall benchmark result, but it does not verify every Moonshot demonstration. Claims about building full games, creating parametric 3D models and improving finance work are based largely on company demos or internal tests. The available information also does not establish how the model performs across a broad range of customer workloads, or what hardware and operating costs would be required for self-hosting a model with 2.8 trillion total parameters.
Hallucination figures remain especially dependent on the test used. Moonshot’s internal test reported a larger reduction than Artificial Analysis’s separate evaluation, which recorded a fall from 53% to 44%. Neither result alone establishes an everyday hallucination rate across all uses. The underlying benchmark conditions and differences between tests limit direct comparison.
high speed NVMe SSD for AI model weights
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The November Weight Release
The next major milestone is Moonshot’s promised November 4 weight release. The release itself, its licence and the files or instructions provided with it will determine whether developers can independently run or modify K3.1 Pro. If the date changes, or if the licence limits use, the practical meaning of the “open-weight” label will change with it.
After release, independent researchers and developers will be able to test the model beyond Moonshot’s hosted apps and API, subject to the actual access terms. Relevant questions will include whether local deployments reproduce the reported benchmark performance, what computing resources they require, and how results hold up on specialised tasks. Until those checks are made, the launch establishes a strong evaluation result—not yet a verified, broadly deployable alternative to proprietary frontier services.
USB-C external SSD for AI model files
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
How did Kimi K3.1 Pro rank against leading models?
Artificial Analysis scored it 55.1 on its Intelligence Index, placing it fourth in the reported comparison. Opus 5.5, Sonnet 5.5 and Muse Wonder Full scored higher, at 57.6, 56.0 and 55.8 respectively.
Can developers download Kimi K3.1 Pro now?
No. Moonshot launched it through its apps, products and API on October 23, but said the weights are expected by November 4. The release had not happened in the supplied reporting.
What licence will apply to the weights?
No K3.1 Pro licence had been published in the source material. Users will need to review the terms when Moonshot releases them before deciding whether their intended use is permitted.
What is the strongest independently reported result?
Artificial Analysis recorded 67% on Terminal-Bench 4.0, compared with 13% for Kimi K3 in its evaluation. The K3.1 Pro score was the highest in the reported test set, but it reflects that benchmark rather than all agentic tasks.
Are Moonshot’s hallucination claims independently confirmed?
Not as stated across everyday use. Moonshot reported an internal-test decline from 23.4% to 8.3%; Artificial Analysis measured a separate decline from 53% to 44%. These are different evaluations, and Moonshot cautioned that its stress test does not represent general daily-use error rates.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
