
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A chatbot can sound ready for work. A company has to prove it.
For technology fans, the leap from impressive AI demos to software that can handle real business decisions is the story to watch. Firmulate’s live experiment puts several models through a small company’s worst week, where spotting a problem is only part of the job. The models must also act, resist manipulation and close a deal. The results expose a gap that a polished chat window can hide.
Products Worth Considering

DJI Osmo 360 Motorcycle Handlebar POV Combo

Anker SOLIX C1000 Gen 2 Portable Power Station with 200W Solar Panel

DJI Mini 5 Pro Fly More Combo Plus with DJI RC 2
As an affiliate, we earn on qualifying purchases.
One company, the same hard week
Firmulate ran frontier models against the same customers, crises and temptations in a small software company. Every decision was versioned and auditable. In the final Crucible League, dated July 2026, gpt-5.6-sol finished first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The experiment counts partial progress, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
All the models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. The finding points to a practical challenge for business AI: recognizing the right answer is not the same as carrying it through. As the experiment puts it, “Same diagnosis, same pitch — no signature.”
The clue was buried in the company’s own files
The decisive competitor weakness did not arrive in the customer event. It sat two document references deep in the company’s files. Models that read the file won the deal at full price, worth +€4,583 MRR. The detail makes the test feel less like a quiz with a visible answer and more like a workday: useful evidence can be available, but still easy to miss.
Trust was tested directly, too. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Refusal alone did not guarantee strong management. Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but placed last. It left the close on the table and showed weaker discipline by attempting writes into a locked department instead of escalating. That same weakness appeared, less strongly, in all four.
A live company, and a route to your own test
Firmulate’s synthetic company has 13 employees and real money mechanics: it burns €105k a month against €2.3k MRR, with a public cash countdown. It has learned more than 680 playbook rules, and every workday is versioned. Readers can watch the live company at firmulate.com. A separate quiz built from 242 real, unedited management decisions invites visitors to guess which model made each choice at firmulate.com.
There is a caveat in the league comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. That context belongs alongside the rankings for anyone weighing what the results say about the models.
The next step is to move from watching a shared experiment to testing decisions against your own business. Firmulate says enterprises can run the same kind of wargame from a read-only export of their company data, putting crisis scenarios against their own playbooks. The pilot does not write back to real systems. That gives a leadership team a way to examine how models handle its customers, risks and operating rules before entrusting agents with live work.

From watching to wargaming
The experiment suggests that safe, capable AI at work needs more than sharp analysis: it has to find relevant evidence, follow through and respect boundaries. A company specific pilot can make those questions concrete using your own data and scenarios. Explore the Firmulate pilot or contact contact@firmulate.com to discuss a wargame for your business.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
