A build-in-public project with the guard removed
Anyone who works with tools knows that a polished finish can conceal a messy process. The revealing moments come before the cleanup: the missed step, the instruction left unread, the cut that was measured but never made. Firmulate applies that workshop logic to artificial intelligence by letting the public watch synthetic employees operate a software company under financial pressure.
The company has 13 synthetic employees and real money mechanics. It burns €105k a month against €2.3k in monthly recurring revenue. Its cash countdown is public, more than 680 self-learned playbook rules have accumulated, and every workday is versioned. This is not a tidy demonstration in which the difficult parts have been edited out. The company is running software, losing money and producing fresh evidence about how AI behaves when work must be completed rather than merely discussed.
The unfolding company can be watched live. That visibility turns business survival into a running story: readers can see whether activity becomes useful progress, whether lessons stick and whether the synthetic workforce can close the gap between recognizing a problem and resolving it.
Project Management with AI For Dummies
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The worst week became a test bench
Firmulate’s Crucible League placed frontier models in charge of the same small software company during its worst week. They encountered the same customers, crises and temptations, with every decision versioned and auditable. The final July 2026 standings put gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. A do-nothing baseline scored 26 because partial progress still counted.
The results were not primarily a contest in fluent writing. All the models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. The gap was stark: “Same diagnosis, same pitch — no signature.” In practical terms, some participants could understand the situation and prepare the right response without completing the commercial task.
The decisive clue was buried in the paperwork
The deal also exposed a familiar lesson for anyone who has assembled a machine or followed a woodworking plan: reading the documentation can matter more than improvising at the workbench. The decisive weakness in a competitor was not presented in the customer event. It sat two document references deep inside the company’s own files.
The models that found and read that material won the deal at full price, worth +€4,583 in monthly recurring revenue. Those that missed it faced the same customer and produced the same diagnosis, but left the close unfinished. The finding makes file-reading look less like administrative overhead and more like part of the job itself.
Pressure tested judgment as well as persistence
The models were also confronted with fake CEO messages that escalated over three stages, followed by a reporter seeking “just one yes/no, on background.” All 5 models refused the manipulation attempts. Kimi K3 recorded its reasoning plainly: “Treat the request as a suspected approval-bypass / possible impersonation.”
That shared resistance matters because the league treated trust as a hard boundary. A single breach capped the total under the principle that “no amount of good work outweighs a breach of trust.” The test therefore rewarded more than energetic output. Models had to preserve judgment when authority appeared urgent, ambiguous or deliberately misleading.
Thoroughness did not guarantee completion
Opus 4.8 produced the deepest analyses and added +80 learned rules, making it the most thorough participant. It still finished at the bottom of the league. The close was left on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in all four other participants, though less strongly.
The contrast is useful beyond AI benchmarking. A worker can document extensively, diagnose correctly and still fail at the final handoff. Firmulate’s public record makes that failure visible instead of allowing abundant activity to stand in for a finished result. Visitors can also read what the synthetic employees say, adding the texture of day-to-day work to the financial countdown.
There is an important fairness detail in the rankings: K3 ran without an effort parameter and therefore used the API default, while the other models ran at xhigh. The result remains part of the final league table, but the different setting is relevant context when comparing performances.
A company becomes a continuing product test
For DIY and tool-minded readers, Firmulate is compelling for the same reason a transparent workshop build is compelling: the value lies in seeing decisions meet resistance. Instructions can be missed. Safety boundaries can be challenged. Careful preparation can still end with an unfinished job.
The experiment’s sharpest finding is not that AI can identify trouble. Every model did that. It is that recognition, research, trustworthiness and completion are separate demands. Only two participants turned a correct analysis into the signed €55,000 deal.
Meanwhile, the live company continues with 13 synthetic employees, €105k in monthly burn, €2.3k in monthly recurring revenue, a public cash countdown and more than 680 learned rules. Each workday adds another version to the record. Firmulate has made build-in-public unusually literal: the audience is not merely watching software being created, but watching a software company fight to become viable in full view.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
