AI Made Engineers Faster. Leadership Fell Behind
Every conversation about AI and software starts in the same place: we can build faster now. Fine. Nobody's arguing. The question I actually care about is the one nobody wants to sit with — did anything downstream of engineering get faster too?
For Episode 17, Oscar and I brought in Mike Lyons and Greg Pfister from KaiRise. They've spent years inside government teams, consultancies, and engineering orgs watching how products actually get built, which means they've watched a lot of them get built wrong. This is a guest deep-dive: no Signal or Noise, no rotating segment. One question, four people, an hour.
The $4.3 million system that did everything right
Mike opens with a story I haven't stopped thinking about. A $4.3 million voter registration system. Shipped on time. Shipped on budget. Shipped on scope. Every number a program manager gets graded on came back green.
Users rejected it.
The team built exactly what was requested. What was requested was not what anyone needed. That gap — between specified and needed — is the entire episode, and it's the thing AI makes more dangerous, not less. Every one of those green metrics measured whether the team did what it was told. None of them measured whether it was worth doing.
Here's why that matters now: if you had a process that could ship the wrong thing in eighteen months, and you bolt AI onto the engineering half of it, congratulations — you now ship the wrong thing in six.
Engineering was 20% of the problem
The number Mike and Greg put on it: engineering is maybe twenty percent of the full delivery process. I'd argue the exact figure varies wildly by org, but the shape is right, and it's the shape that matters.
If writing the code was one-fifth of the work, then making that fifth dramatically faster does not make the company faster. It makes the other eighty percent the bottleneck, and it does it suddenly. That's the part leaders keep missing. They rolled out AI tooling, watched throughput jump, and then got confused when time-to-value didn't move. It didn't move because the slow part was never in the IDE. It was in the decision to build, the approval to fund, the argument about scope, the three weeks waiting on a stakeholder who's in a different quarter than you are.
AI didn't fix your organization. It removed the excuse you were using to avoid looking at it.
Planning cycles built for a slower company
This is where it gets uncomfortable for leadership specifically.
Five-year plans, annual budget cycles, multi-week approval chains — none of that is stupid. It was rational infrastructure for a world where delivery took quarters. You planned far out because you had to; the cost of changing course mid-build was enormous.
Compress delivery and that math inverts. Now the plan is stale before the budget clears. You've got teams that can turn around a working version in days sitting inside a governance structure that assumes they can't. The friction isn't a people problem or a tooling problem. It's a tempo mismatch, and the slow half is the half that signs the checks.
Nobody wants to hear that the constraint moved into the executive suite. But if engineering got 5x faster and the company got 1.1x faster, that's where it went.
Stop specifying. Start showing.
The most immediately useful thing in this episode is the smallest.
Requirements sessions exist because building was expensive. When it cost a fortune to be wrong, you spent a month writing down precisely what you wanted so you'd only pay once. That trade made sense.
It doesn't anymore. When you can stand up a live prototype in an afternoon, a month of requirements documents is a month of people arguing about a thing none of them can see. Put a working version in front of them and the argument resolves in one session, because now everyone's reacting to the same artifact instead of their own private mental model of it.
Building got cheap enough that showing should be the default. Most teams haven't updated the habit.
What is a story point when an agent writes the code
Oscar and I have kicked this around before and I still don't have a clean answer. Story points were a proxy for human effort — relative complexity, sized by the people doing the work. When a meaningful chunk of the work is done by an agent in ten minutes, what exactly is the two-pointer measuring?
The trap I see teams walking into is swapping one output metric for another. Points out, lines of code in. Or worse, token spend, which measures nothing except how much you paid. Every one of these tells you the machine was busy. None of them tell you a customer was better off.
Mike and Greg's framing here is the one I'd steal: measure outcomes, not activity. It sounds like a poster in an HR hallway until you notice that almost nobody actually does it, because activity is easy to count and outcomes require you to talk to a human being.
The skill I think is actually scarce now
Somewhere around the forty-minute mark this landed for me, and it's the line I'd put on the whiteboard: the developer skill that matters most now is knowing when to stop.
Producing more is solved. It's free, it's instant, it's not a differentiator. The scarce judgment is recognizing the point where additional code stops improving the outcome — where the next feature is you building because building feels productive, not because anyone needs it. That's the same failure mode as the $4.3 million system, just distributed across a thousand small decisions instead of one big one.
And it connects to the thing AI genuinely cannot touch. Unclear ownership. Misaligned incentives. A roadmap full of features nobody asked for. Those are organizational problems, and there is no model release coming that resolves them. If your company shipped the wrong things before, it will ship more wrong things now, faster, with better test coverage.
Where I landed
Cheaper building doesn't make product skill less valuable. It makes it the whole game. When execution was the constraint, being right about what to build was nice. Now that execution is close to free, being right about what to build is the only thing left that's hard.
The question Mike, Greg, Oscar and I ended on is the one I'd put to you:
What became the slowest part of your company after AI made building faster?
If the honest answer is "the part where we decide," you're not behind. You're just early to noticing.
Watch or listen to the full conversation below.
Human in the Loop — Watch on YouTube · Listen on Spotify · Listen on Apple Podcasts
Every episode — full show notes, sources, and the archive — lives at podcast.vallyseed.com. The show is produced by VallySeed, where Oscar and I help organizations design, build, and deploy AI systems that create measurable competitive advantage.
