Why I don't believe the promise of 'AI increases productivity by 300%'

The promise of "AI increases productivity by 300%" is a marketing claim, not a verified research result. You've seen headlines like "AI helps increase productivity by 300%" or "Automation saves 80% of time". I do AI Automation every day, and I don't believe those numbers. Not because AI isn't useful, but because numbers like that usually don't clarify what is being measured, who it's measured for, or compared to what. Research with names and years, such as NBER in 2023 or METR in 2025, shows much more modest results and varies by group of people; the section below presents the specifics.

Are the numbers "productivity increase of 300%" trustworthy?

A productivity number is a comparison, and it only makes sense when it clarifies what is being measured, for whom, and compared to what baseline. "Increase by 300%" means increase from what? For example, a hypothetical case: someone spends 3 hours a day composing emails, now spends only 1 hour thanks to AI. Email composition is faster by a factor of three, that's true. But that doesn't mean the business produces three times as much, because productivity really depends on what the 2 recovered hours are used for. That's why on this website, I don't put out numbers like that. When working with clients, I measure with concrete things specific to them: which tasks used to be done by hand now run automatically, how much time is recovered each week, where errors have decreased.

What results did NBER research on customer support staff show?

The research "Generative AI at Work" by Erik Brynjolfsson, Danielle Li and Lindsey Raymond (NBER Working Paper 31161, 2023, read on 25/10/2026) is a controlled study of productivity when using AI, and the results are not 300%. The authors tracked 5,179 customer support staff. First, the AI assistant increased the number of issues resolved per hour by an average of about 14%. Next, the newly hired or low-skill group increased by 34%. Finally, the experienced group changed very little. This result came from one specific job at one specific company, so it doesn't automatically apply to other professions.

What results did METR's experiment with programmers show?

METR's experiment, published on 10/7/2025 (read on 25/10/2026), showed opposite results: programmers took longer when using AI. The research team tracked 16 experienced open source programmers on 246 real tasks, each task taking about 2 hours and randomly selected to be done with or without AI. When using AI, they took 19% longer, even though they previously expected to be 24% faster and afterward still believed they were 20% faster. METR noted the small sample, tools from early 2025, and did not draw conclusions for all programmers or all industries; by 2026 they had published newer results. The practical lesson: the feeling of being faster is not enough, you have to measure actual time.

What does AI do well and not well?

AI does well with tasks that have clear patterns and are easy to check, but struggles with tasks requiring judgment. From what I use daily with large language models:

• Does well: draft writing, summarizing, classifying, transferring data between systems, answering repeated questions based on existing documents, writing and editing code under supervision.
• Does poorly: tasks requiring judgment in new situations, tasks where one mistake costs money, answers that need accurate data that AI has no source to verify.

AI can be wrong in a very confident way. That's why every system I build has a place for people to check before results are sent out, for example a review step for drafts before emails reach customers.

How do I evaluate an AI tool before paying for it?

The way to evaluate an AI tool is to ask 5 questions about your actual work before paying, in the exact order below. For example, a customer care tool with a free trial but no way to export customer lists to a file will lock you in to that vendor. To compare models with independently measured numbers instead of advertising claims, see What is Artificial Analysis.

1. Does it solve something on my to-do list? If it doesn't answer yes, no need to buy yet.
2. Is there a trial with my real data? A demo is always better than reality.
3. When it gets it wrong, how do I know? Good tools let you review and correct.
4. Where does my data go? With customer information, this matters more than any feature.
5. If I stop using it, can I get my data out?

What is the only promise OPCAIOS makes?

The only promise OPCAIOS makes is the method, not the numbers. The two studies above show that results change from person to person and task to task, so I don't promise a fixed percentage. I start from your real work, pick the right things to automate, build until it runs, and guide you to operate it yourself. For any work that AI doesn't do well yet, I tell you straight, even when it means advising you not to automate that task yet. If you want to know whether your work has anything worth handing over to AI, leave a few lines on the contact page. I read and reply to every message myself.

Source: Brynjolfsson, Li, Raymond, "Generative AI at Work", NBER Working Paper 31161, 2023 (nber.org/papers/w31161, accessed 25/10/2026); METR, research on open source programmers, 10/7/2025 (metr.org, accessed 25/10/2026; small sample, tools from early 2025). The rest is personal experience, hypothetical examples.