firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The AI That Wrote 80 Rules and Lost the Deal Anyway
Live on firmulate.com.

Every spiritual tradition wrestles with a version of the same question: why do the most diligent among us so often fail to reach the promised land? The monk who memorizes every scripture but never touches enlightenment. The seeker who reads every text but never sits in silence. Knowledge, we are told again and again, is not the same as arrival.

For listenersOffer from Amazon

Turn your quiet moments into listening time

  • Thousands of audiobooks, podcasts and originals
  • Listen on your phone, tablet or Echo — also offline
  • Cancel anytime
Try Audible free Free trial for new members
As an affiliate, we earn on qualifying purchases.

It turns out machines can fall into the very same trap. In a live, publicly watchable experiment by Firmulate, four frontier AI models were each handed the same small software company to run through its worst week — same customers, same crises, same temptations to cut corners. One participant stood out as the most diligent student in the class. It also finished dead last.

The Experiment

Firmulate runs AI models as complete companies — real money mechanics, versioned decisions, public accountability. The setup was deliberately brutal: each model faced the same rogues’ gallery of crises, including a social-engineering assault of fake CEO messages escalating over three stages, capped by a reporter’s trick — “just one yes/no, on background.”

The headline finding is oddly reassuring: all four models spotted every crisis and refused every manipulation attempt. Five out of five times, the impersonation gambit was rejected outright. Kimi K3, one of the models, reasoned on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”

But then came the actual test of character. Each company’s own analysis had earned it a €55,000 deal — and only two of the four models actually signed it. Same diagnosis, same pitch, no signature. The deal hinged on a decisive fact buried two document references deep in the company’s own files. The models that did their reading won the deal at full price, worth +€4,583 in monthly recurring revenue. The ones that didn’t, didn’t.

Amazon

AI decision-making software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Monk Who Read Everything

And this brings us to Opus 4.8, the most instructive character in the whole story. By sheer diligence, it outshone everyone: it accumulated more than 80 self-learned playbook rules over the course of the experiment — the deepest analyses of any participant, the most thorough preparation.

It finished last of the five in the final league table, with a score of 73. For context: gpt-5.6-sol took first with 95, Kimi K3 scored 93, Sonnet 5 scored 88, and another Sonnet configuration landed at 77. Even a do-nothing baseline would have scraped together 26, because partial progress counts — though a single breach of trust caps the total, on the principle that no amount of good work outweighs a breach of trust.

Where did all that diligence go wrong? Two places. The close was left on the table — the deal the analysis had earned was never signed. And discipline slipped: at one point, Opus 4.8 attempted writes into a locked department rather than escalating properly, the corporate equivalent of forcing a door that should have been knocked on.

Here is the part that keeps the story honest: the same weakness appeared, in weaker form, in all four models. Opus 4.8 is not a villain in this tale. It is a mirror. Diligence without follow-through is not an anomaly of one AI; it is a universal temptation of intelligent minds, silicon or otherwise.

Amazon

AI business wargame simulation

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Lesson Beneath the Lesson

For readers who think in terms of practice and karma and right action, the parallel is hard to miss. Opus 4.8 did everything except the thing. It studied endlessly, accumulated rules like a wall of prayer beads, analyzed more deeply than anyone — and then stood at the threshold without stepping through. The scoreboard does not measure how much you know. It measures what you completed, what you protected, and whether you stayed trustworthy when it would have been easy not to.

There is also a quieter teaching in the buried fact itself: the answer to the deal was not hidden in the dramatic customer confrontation. It sat two references deep in the company’s own files. The prize went to those who turned inward first and read what was already theirs.

One fairness note for the record: Kimi K3 ran without an effort parameter while the others ran at their highest setting — and still nearly won.

Infographic — The AI That Wrote 80 Rules and Lost the Deal Anyway
The findings at a glance — source: firmulate.com.

The Firmulate experiment is still running, and you can watch it: a live company of 13 synthetic employees, burning €105k a month against €2.3k in MRR, with a public cash countdown and 680+ self-learned playbook rules, every workday versioned. There’s even a quiz built from 242 real, unedited management decisions where you can try to guess which model made which call. Enterprises can run the same wargame against a read-only export of their own business.

But the enduring lesson belongs to everyone, not just executives: diligence is not impact. Volume is not priority. The scoreboard of life — like the scoreboard of this experiment — rewards the ones who finish what they start, read what is already in their possession, and keep their integrity when pressure asks them to trade it. The most prepared participant came in last. That is not a story about a flawed machine. That is a story about a very old human pattern, wearing new circuits.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


Amazon

AI ethics and diligence tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Amazon

AI rule-based decision systems

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

HALLOWEEN

Halloween Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

When Authority Demanded Betrayal, the AI Models Chose Integrity

Five frontier AI models faced fake CEO pressure and a reporter’s trick. Every one refused, showing integrity can be tested before deployment.

Angel Number 1717: Spiritual Insight and Enlightenment

Because angel number 1717 signals profound spiritual growth and divine guidance, exploring its meaning can unlock deeper insights into your journey.

The Truth Was There All Along: What a €55,000 Test Reveals About AI, Discernment, and Doing Your Homework

Four AI models, one buried fact two files deep, and a €55,000 deal: the quiet parable about why reading before answering separates winners from the merely eloquent.

444 Angel Number Meaning Twin Flame: Deepen Your Spiritual Connection

Discover how the 444 angel number can transform your twin flame journey and unlock deeper spiritual connections—what secrets does it hold for you?