Amazon held a mandatory engineering meeting yesterday to address a pattern of outages linked to AI-assisted code changes. Their SVP acknowledged that site availability “has not been good recently.” The new policy: senior engineers must now sign off on all AI-assisted code produced by junior and mid-level developers.
Amazon set an internal target of 80% weekly usage of AI coding tools. They pushed engineers toward their in-house tool even as many preferred alternatives (see comments). And now they’re adding human review gates because the deployment pipeline wasn’t built for the speed and volume at which AI tools produce changes.
They are not alone in this. Every engineering leader is navigating the same tension: real productivity gains on one side, and a growing list of problems on the other.
Surprise bills as AI coding tools shift from flat-rate to consumption-based pricing.
Massive efficiency gaps between tool combinations that most teams aren’t even measuring.
Traditional delivery metrics like story points and velocity no longer describe what’s actually happening in their teams.
At Jeavio, we’ve been living this since 2023, when we enabled GitHub Copilot for every developer. We rolled out Cursor in early 2025. Now we’re using Claude Code alongside Cursor. Each wave has moved faster than the last, and alongside productivity gains, we also found significant problems.
Our approach has been to pair bottom-up experimentation with structured governance. Engineers try new tools and share what they learn. A council of senior engineering, security, and operations leaders develops the guardrails. And we make tool decisions based on measured outcomes, not enthusiasm.
We recently ran a controlled evaluation: a senior developer built the same feature using multiple AI-assisted development approaches spanning tools, models, and plugins. The findings challenged several assumptions we held about which tools and configurations deliver the best results. The biggest cost driver, for instance, wasn’t what most teams would guess. Our team will be publishing the comprehensive research on LinkedIn soon.
Despite all the hype, AI is still a frontier technology. The gap between “this saved me two hours” and “this took down a production environment” is narrower than most people realize.
Engineering leadership right now requires holding both of those realities at once, and building organizations that can move fast without skipping the work of understanding what they’re deploying.
Many thanks to Ankit, Kamal, Monika, Krunal, Tushar, Manan and many others at Jeavio on laying the ground for continued experimentation and AI adoption at Jeavio.
DM me if you want to learn more.