Even a thoughtful gift needs the right judgment behind it
Choosing a gift or planning an occasion often comes down to details: what someone values, what they have already said, and whether a tempting shortcut is really a good idea. Businesses face a higher-stakes version of that problem when they put AI to work. Firmulate’s live experiment asks whether models can read the details, protect trust and finish a difficult job under pressure.
A company’s worst week becomes a test
Firmulate ran frontier AI models through the same small software company and its worst week: the same customers, crises and temptations. Decisions are versioned and auditable, and the experiment is watchable at Firmulate. The point is to examine management quality in action, beyond how polished a model sounds in a conversation.
The final Crucible League, dated July 2026, puts gpt-5.6-sol first with 95 points and Moonshot’s Kimi K3 second with 93. Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. K3 therefore finished ahead of three of the four Western frontier models in the field.
All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. The gap between seeing the right move and carrying it through is the experiment’s central finding: “Same diagnosis, same pitch — no signature.”
The detail that changed the deal
The decisive weakness in a competitor’s position was buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. K3 found that buried security needle, closed the deal and saved a churning customer.
Trust faced a separate test. Fake CEO messages escalated across three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Discipline separated the participants, too. K3 had only one deviation, the cleanest discipline in the field. Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but finished last. It left the close on the table and slipped by attempting to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four.
A watchable experiment, with a fairness note
The company has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules and versioned workdays. Its do-nothing baseline scored 26; partial progress counts, but a single breach of trust caps the total. The stated principle is “no amount of good work outweighs a breach of trust.”
One comparison needs context: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The results are a reason to run relevant tests, not to assume a leaderboard settles every organization’s choice. Firmulate says enterprises can pilot the same wargame against a read-only export of their own business; nothing writes back to real systems. Its 242 real, unedited management decisions also power a “guess the model” quiz.
Readers can review the benchmark findings and visit Firmulate to watch the live company, try the quiz or learn about the pilot.
Tiawudi Aufbewahrungsbehälter aus Kunststoff, transparent, vielseitig verwendbar, 12 Stück
- Versatile Storage Containers: Multi-purpose plastic storage set
As an affiliate, we earn on qualifying purchases.
Test before you choose
Kimi K3’s second-place finish makes the field look open, while the deal and trust tests show why fluent answers alone are not enough. If an AI model may handle customer relationships or business decisions, choosing without testing it on your own work is a bet. See how the results compare at Firmulate’s benchmark page.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Amazon Basics zusammenklappbarer Aufbewahrungswürfel/Organizer aus Stoff mit Griffen, 26,6 x 26,6 x 27,9 cm, Einfarbig Beige, 6 Stück
- Set of 6 Storage Cubes: Organize home and office spaces
- Durable Breathable Fabric: Robust with attached handles
- Versatile Use: Open boxes or shelf inserts
As an affiliate, we earn on qualifying purchases.
Vtopmart 6 Stück Aufbewahrungsboxen mit Deckel, Ordnungsboxen Stapelboxen-L
- Versatile Storage for Any Room: Suitable for kitchen, bathroom, and more
- Transparent Design for Easy Content Identification: See contents at a glance without opening
- Stackable and Space-Saving: Maximize vertical storage in cabinets and shelves
As an affiliate, we earn on qualifying purchases.
