Uber's AI infra, from the inside

The central story of this Pragmatic Engineer interview is not that Uber suddenly built a coding harness. Udai describes years of investment in service infrastructure, standardized languages, ownership records, testing, deployment and observability before agentic tools arrived.

Those investments were responses to a business spanning many products and local conditions. They later gave agents something structured to use. The case therefore argues for the value of foundations, rather than proving that every company should recreate Uber’s internal stack or can obtain the same result from a new chat interface alone.

Udai connects language-specific monorepos and standard build systems to Dev Pods: cloud environments with the repositories, tools and caches already prepared. A larger agent-oriented environment can work across repositories without repeatedly rediscovering setup steps.

The important leverage is repeatability. A human and an agent can use known build and test commands, while top-level instructions explain the shared conventions. Fast environment creation rests on images and warm infrastructure prepared beforehand, not on an agent magically avoiding dependency or build work.

The interview describes a gateway exposing compatible model APIs behind project identity and caller credentials. Different model providers sit behind that interface, alongside policy checks, observability and accounting.

AI Guard is presented as a shared request-and-response control layer, while spend attribution and audit traces help teams understand use. These are descriptions of intended safeguards, not proof that all sensitive content or every attack is perfectly blocked. The design benefit is a consistent place to apply and improve controls as models change.

A wrapper around different coding harnesses carries shared plugins, skills and user configuration across local and cloud environments. The goal is to make switching tools less dependent on each engineer reconstructing a personal setup.

Cost remains visible. Udai describes spending tiers, feedback in the tool and a lightweight manager discussion when more capacity is needed. That combines access with an ROI conversation, rather than treating unrestricted token consumption as the measure of progress. Interactive and managed background agents can have distinct budgets.

Uber’s MCP gateway builds on an existing registry of service contracts. The interview describes generating tool schemas from Thrift or Protocol Buffers definitions, refining descriptions and putting custom or vendor tools behind the same gateway.

A CLI projection lets a harness use another interaction form without each team rebuilding the underlying integration. Generation is not the same as authorization: configuration and security policy still matter before exposing a tool. The case’s advantage comes partly from having structured contracts and gateway expertise already in place.

The shared skills repository combines central platform work with team-specific processes, using common installation and quality gates. Portable access to tools helps the same workflow survive a change of harness.

Udai distinguishes making a first version from making a skill reliable across use cases. Trace-based improvement and evaluation are continuing work, not a guarantee that usage volume proves quality. Moving suitable tool interactions into CLI form is described as a way to reduce unnecessary context load, rather than a universal claim that one protocol is always superior.

The graph links people, assets, documents, changes and operational records. An assistant can use those relationships to identify the relevant domain or owners before following the actual documents, queries or service information needed for an answer.

The displayed schema’s many node and edge types describe categories, not the total number of records. Udai reports quality, cost and latency benefits, but the interview is not an independent benchmark. He also explicitly says existing user permissions carry through: a connected graph is not unrestricted access to every person’s data.

Design documents and approval records give people and agents a source of truth about the intended feature. Minion is described as a background coding agent that can use a well-scoped specification and produce changes across repositories, with steering available through chat.

After generation, checks move earlier into the inner loop, including build rules, linting and visual validation. A Code Inbox brings evidence together for reviewers. The demo includes a mock interface and work in progress; importantly, Udai says every production pull request still requires an engineer’s review at recording time. A possible low-risk green lane is future work, not the current rule.

Slate lets engineers deploy a branch into a test environment and route selected test accounts to the services under test while the remaining calls use the surrounding staging system. Tracing baggage and service-mesh routing help carry that decision across service boundaries.

Agents can then use the same setup to compare main with a branch and attach screenshots, videos or API evidence. That is a concrete feedback loop, not proof that every possible regression is absent. The system took years of instrumentation and compatibility work; its apparent ease in an agent demo depends on that prior investment.

63:00View full image

The capture-and-replay system records requests, responses and downstream interactions, then tags relevant dimensions so teams can compare behavior before and after a change. A particular bug can be replayed against a branch, while broader observed combinations help expose unintended deltas.

Observed combinations are not every logically possible state. Nor does a public architecture demonstration establish how every payload is redacted, retained or authorized. The useful principle is reproducibility with controlled comparison; it is not a reason to use private production user data as an ad hoc test fixture.

Shepherd organizes large changes into campaigns, using mechanical transformation policies where appropriate and agentic work where more judgment is needed. Fox extends that idea into recurring maintenance proposals for enrolled services; the described rollout is still opt-in for relevant skills.

Wisdom supplies another path: employee-reported bugs include telemetry and reproduction context, an assistant proposes a root cause and a person can hand the checked diagnosis to a coding agent. These loops generate reviewable work and evidence. They do not establish that every proposed cleanup or diagnosis is correct, already merged or safe without validation.

Udai describes moving the productivity discussion beyond diff counts toward feature cycle time, categories of work, developer experience and quality signals. Linking plans, tasks and changes is part of the effort to understand where the actual delay sits.

The interview does not present a controlled causal estimate of productivity. Its closing shift is practical: engineers increasingly design the context, constraints, tools and verification path that let an agent do work reliably. The goal remains useful business outcomes, with human checks and ongoing measurement, rather than artifacts produced for their own sake.