Today I’d like to introduce a post that appeared on the Uber Engineering Blog.
Since adopting AI, many people have said, “I’m not sure about productivity, but the bill scares me.” This article is helpful for those folks.
The title is “Running a Software Factory Efficiently at Uber Scale.”
The numbers are provocative, right? ^^
Uber was once cited as a prime example of wasteful spending due to reckless AI adoption, having burned through its AI budget early on. However, from February to August this year, agent users grew 7‑fold and request volume grew 9.4‑fold, yet total AI spend has been essentially flat since April. Moreover, after fixing the model and re‑measuring, the cost per 1,000 requests dropped 34% and the cost per session fell 52%.
It’s not that they switched to a cheaper model or blocked tool usage. Today Uber’s agents generate more than 70% of all PRs (a PR is a unit where a developer submits code changes for peer review; think of it as a single work item).
The engine that makes this possible is engineering. In the end, AI costs can also be solved through engineering. This is why a capable CTO or skilled IT leader who understands business operations is essential for an enterprise.
To break down Uber's engineering approach,
Uber doesn't look at the bill as a whole; it decomposes it into multiple multiplicative components.
Number of users × number of sessions × number of exchanges × characters per exchange × cost per character
The first two factors represent adoption and utilization rates, which must be continuously grown. Reducing them isn't cost saving; it's business regression. The optimization target is the latter part, where AI adds overhead on top of the user's request.
Accordingly, here are the five sources of waste Uber eliminated.
1. Notes that disappear every 5 minutes
The AI re-reads the entire conversation each time a turn occurs, temporarily caching the transcript with a 5‑minute TTL. Developers often stepped away longer than that, causing the cache to expire and forcing a full reread. We extended the TTL to one hour.
2. Carrying unused documentation
When external tools are integrated with the AI, their usage guides are bundled automatically. Adding 100 tools drags along 99 guides that may never be used that day. We switched to a just‑in‑time loading model, pulling documentation only when needed.
3. Assigning a task and then watching from the sidelines
For each data query, the AI was exchanging messages like “Sent”, “Is it still pending?”, “Is it done?” one by one. All that back‑and‑forth is just cost. When we let the program handle the repetition and the AI only received the final result, the time for a task that produces a single line of output dropped by more than half.
4. Improving Export Without a Map
We compared using the same question and the same AI. The version linked to the internal knowledge map answered correctly in 38 seconds, whereas the unlinked version wandered for 20 minutes and got it wrong. We therefore built a map.
5. Looking Only at the Total Amount
“How much did we spend this week?” does not drive any action. Uber identified 16 waste categories per task, attaching loss amounts and recommendations. Instead of capping spend, they added real‑time cost displays and alerts.
These five items may not be universally applicable. However, if they delivered results in organizations with thousands of engineers and billions of lines of code, they are worth a look. Each company should tailor the methodology to its own circumstances.
Moreover, a large portion of current AI spend is likely non‑value‑adding. In such cases, the savings come from redesigning the solution rather than merely cutting the budget.