What a Do-Nothing AI Manager Reveals About Trust, Progress, and Performance

Imagine hiring an AI for your garage that’s supposed to streamline your workflow, but it turns out it’s just doing the bare minimum — yet still scores more than zero. That’s the reality revealed by a recent public AI benchmark that exposes the surprising thresholds of trust and partial progress in automated decision-making.

Buying for a business?Offer from Amazon

Get business pricing on garage and car supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Understanding the AI Benchmark for Business Decision-Making

At the heart of this experiment is a simple question: how do we measure the true competence of AI models managing a complex environment? Instead of evaluating chat quality or clever responses, the benchmark runs AI models through a simulated week of a small software company, complete with real crises, customer interactions, and internal temptations to cut corners.

All models, from the most advanced to the more modest, faced identical scenarios: urgent crises, attempts at manipulation, and crucial deal-making opportunities. Every decision the AI made was recorded, and the process was transparent and auditable. This meticulous setup made it possible to see exactly what each model prioritized and whether it maintained honesty and thoroughness under pressure.

What the Results Tell Us About Trust and Partial Progress

The surprising outcome? All four models successfully identified every crisis and refused every manipulation attempt, demonstrating a baseline of integrity and awareness. However, only two of them managed to close a key deal at full price, earning the €55,000 revenue. The other two did not finalize the deal even after making the same diagnosis and presentation.

Strikingly, the scores ranged from 26 to 95, with the lowest being a do-nothing baseline that still scored 26 points. This baseline model didn’t do nothing at all — it showed partial progress. This score underscores a critical point: in such assessments, even minimal effort or partial correctness counts toward the final score, reflecting a more realistic measure of how AI performs in real-world settings. It’s not an all-or-nothing game, but one where incremental progress matters.

The Role of Trust Violations and Caps on Performance

Another key finding is that if an AI breaches trust — such as attempting manipulations or skipping crucial information — its maximum possible score is capped. In this benchmark, even the best models couldn’t surpass a certain threshold if they compromised their integrity. This rule emphasizes that honesty and reliability are non-negotiable metrics, especially in business contexts where trust is everything.

For example, despite the most thorough model, OPUS 4.8, attempting more detailed analyses and learning over 80 rules, it still finished last because it slipped in discipline, leaving deals on the table instead of escalating. Yet, it refused manipulative tactics, maintaining integrity. This nuanced outcome illustrates that performance isn’t solely about the end result but also about how the process aligns with trustworthiness and discipline.

Implications for Business and AI Deployment

What does this mean for garage owners and automotive businesses considering AI tools? First, the focus shouldn’t be just on the AI’s ability to generate convincing responses. The real questions are: will it finish what it starts? Will it read your files thoroughly before making decisions? Will it stay honest under pressure?

The benchmark demonstrates that models can be trained and tested to prioritize integrity and complete work, even in stressful situations. For a garage, this could translate into an AI that consistently provides accurate diagnoses, reads vehicle histories correctly, and resists shortcuts, ultimately saving time, money, and reputation.

Transparency and Watchability in AI Testing

Firmulate’s live experiment makes this evaluation transparent and observable. You can watch a real company run through its worst week, with every decision tracked and auditable. This isn’t a black-box test — it’s an open window into how AI behaves when faced with real-world challenges.

For automotive businesses, this means confidence that the AI tools they adopt are measured by meaningful criteria: honesty, diligence, and ability to complete tasks. The benchmark’s score range also reveals that even the best models have room for improvement, especially in maintaining discipline under pressure, but the value lies in understanding their limitations and strengths upfront.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI