What Your KPIs Can't See, and What an AI Readiness Evaluation Should Hand You

A lot of my job comes down to how things land with the person on the other end. So when an owner finishes an AI readiness evaluation and tells us, "You told me things I already knew," I take it as a note on the experience. Somewhere between the work we did and the report they read, the value got lost.

To be fair, they usually did know some of it. Owners tend to know their business. That's sort of the job. What's harder to see from the owner's chair is how the work gets done hour by hour, and that's where most of the AI opportunity sits.

What's hard to see from the owner's chair

Most owners can tell you revenue by quarter, margin by customer, and which clients always pay late. Fewer can tell you how a project coordinator spends a Tuesday, how many times the same customer address gets typed into different systems, or which steps exist mainly because nobody has gotten around to questioning them.

That stuff adds up. Asana surveyed more than 10,000 workers and found people spend 58 percent of their day on what it calls "work about work," meaning status updates, hunting for files, and toggling between apps until the browser has 40 tabs open. About a third of the day goes to the skilled work people were hired to do. I'd love to see that line item on a P&L.

My favorite stat on this comes from MIT's Project NANDA. Only 40 percent of companies had bought an official AI subscription, but workers at more than 90 percent of companies were using personal AI tools for their jobs anyway. In other words, your team has probably already picked a favorite chatbot. Whether they're using it like a slightly smarter Google or having it do hours of real work is a different question, and you won't find the answer in a monthly report.

An evaluation fills in that picture, down to the task and the role.

If you already knew, why is it still a problem?

When a finding feels familiar, there's a slightly uncomfortable question to ask: if we already knew about this, why hasn't it been fixed?

There's usually a perfectly good reason. The team is stretched, the fix crosses three departments, nobody clearly owns it, or a tool was rolled out to solve it and quietly faded away. That last one is the version we hear most. Licenses get bought, an announcement goes out, maybe there's a lunch-and-learn with decent sandwiches, and six months later everyone is working the way they did before.

The research says this is about as common as it gets. MIT's researchers found that about 95 percent of enterprise generative AI pilots were producing no measurable impact on profit and loss, despite $30 to 40 billion in spending, and they traced much of it to a "learning gap": tools that didn't fit how the work was done, and organizations that didn't adjust around them. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027. And in a survey of 1,000 executives, Boston Consulting Group found that about 70 percent of the challenges companies face with AI come from people and process, 20 percent from technology, and only 10 percent from the algorithms. In plain terms, most of what goes wrong with AI happens around the software.

So the value in a familiar finding usually lives in the detail underneath it: where exactly the work breaks down, who it affects, roughly what it costs each week, and why the last fix didn't stick.

Why evaluations can feel like an entry fee

Part of the complaint was on us. From our side, the evaluation was step one in a process: it told us what to build. From the client's side, it looked a lot like an entry fee, the thing you pay for so someone can send you a bigger proposal. Nobody loves paying for a sales pitch, even a well-researched one.

So we changed the experience. Every evaluation now ends with a couple of hours walking through the findings together, one at a time, with plenty of room to ask, "Wait, how'd you get that number?" It should be worth the money whether you ever call us again or not.

What a good evaluation should hand you

Whether you work with us or someone else, here's what I'd want to walk away with.

• A picture of how the work really happens: The findings should come from sitting with your people and watching the work, role by role. Surveys and leadership interviews help, though they mostly capture how everyone thinks the day goes.

• Evidence behind every finding: Hours spent, steps repeated, handoffs where information falls through the cracks, places where people are already using AI on their own. The evidence is what turns a hunch into something you can act on and explain to your team.

• A ranked roadmap: Every opportunity scored by impact and effort and put in order, so you know what to tackle first and what can wait. If nothing is worth automating, the evaluation should say so.

• Tool options, with trade-offs: Most problems can be solved by several tools at different prices and levels of difficulty. You should see them side by side and choose with your budget in mind.

• A do-it-yourself path: Enough detail to run with it on your own: the plan, the steps, and who on your team would own each one. A good evaluation will also tell you where that path tends to stall. In our experience, that's usually training, follow-through, and the months after launch when the new-tool excitement wears off.

• A real conversation: A PDF is easy to skim and even easier to lose in your downloads folder. Talking through the findings, and pushing back on the ones that seem off, is where most of the value comes through.

It also helps to tell your team early what the evaluation is for. Nobody is being graded. When people know that, they describe their day a lot more accurately.

Why we call it an evaluation

We spent a surprising amount of time deciding what to call this thing. "Survey" sounds like the little card they hand you with the check at a restaurant. "Audit" sounds like a letter from the IRS. We kept coming back to the idea of a diagnostic, the kind you'd run on a car that's still driving fine. Nobody panics about a diagnostic. You just want to know what's going on before it gets expensive.

Your business is running, and you know your numbers. An evaluation fills in the part the numbers can't show, then hands you what you need to decide what's next, with us or without us.

If you'd like to see what that looks like for your team, you can book an AI Readiness Evaluation here.

Sources

• CNBC, "People spend more than half their day doing busy work, according to survey of 10,000-plus workers" (April 6, 2022), reporting on Asana's Anatomy of Work Index: https://www.cnbc.com/2022/04/06/people-spend-more-than-half-of-the-day-on-busy-work-says-asana-survey.html

• Virtualization Review, "MIT Report Finds Most AI Business Investments Fail, Reveals 'GenAI Divide'" (August 19, 2025), reporting on MIT Project NANDA's "The GenAI Divide": https://virtualizationreview.com/articles/2025/08/19/mit-report-finds-most-ai-business-investments-fail-reveals-genai-divide.aspx

• Fortune, "MIT report: 95% of generative AI pilots at companies are failing" (August 18, 2025): https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo

• Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027" (June 25, 2025): https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027

• Boston Consulting Group, "AI Adoption in 2024: 74% of Companies Struggle to Achieve and Scale Value" (October 24, 2024): https://www.bcg.com/press/24october2024-ai-adoption-in-2024-74-of-companies-struggle-to-achieve-and-scale-value