An update from Amazon engineering leadership on the role of the developer in the age of AI
Amazon rewrote its SDE role guideline around what AI will not change, and admits its AI adoption metrics are flawed but still worth watching.
Learnings
- Define the role by what doesn’t change. The guideline stays silent on specific AI practices because any definition would be “antiquated by this time next year”. It anchors on impact for customers, the business and the team, citing Bezos’s “what’s not going to change” question.
- Input metrics are admitted to be flawed, and used anyway. Token usage, tool adoption and code-change throughput are “deeply flawed” as targets. The memo says “tokenmaxxing is at odds with frugality” and lines-of-code counts are at odds with “invent and simplify”. But they correlate with real output for now, so use them with judgement.
- Watch the tails, not the average. Look at the 10th percentile of AI adoption to learn why teams aren’t benefiting, and the 90th to check whether heavy use is matched by more delivery. Amazon calls the leading teams “frontier teams”.
- “If your work does not benefit from current AI tooling, that is a finding, not a failing.” Engineers are asked to document why and share it.
- Leaders must use the tools on production work. The bar: name a task where AI helped and one where it didn’t, and explain that judgement to the team.
- Proximity to the customer is the multiplier. Engineers closer to their product and business get the most from agentic development. Amazon wants to reverse the drift of ICs and managers away from customers.
What struck me
I wanted to understand how a big tech company is thinking about and adopting AI, and how that thinking could be applied or could guide one.
Will I apply it
Once I understand it, I will apply it.
My take
Amazon admits its AI metrics are flawed and watches them anyway. That’s the right posture: measure, but judge by the tails, not the average.
Applied at Freshly
In progress.