OpenAI: How Habitat Serves More Than 70 Million Requests per Second

OpenAI has published a technical account of Habitat, the online storage platform that powers ChatGPT, Codex, and its APIs.

The figures immediately convey its scale: according to OpenAI, Habitat now handles more than 70 million requests per second, manages over 500 petabytes of data, and operates across nearly 40 geographic regions. The platform supports products used by more than one billion people every week.

These requests correspond to the data accesses required for the products to operate: a single action in an application can trigger many reads.

From a Python library to a distributed platform

Habitat was not originally designed as such a critical infrastructure.

The project began as a client-side Python library connected to a database. As OpenAI’s products and needs multiplied, this architecture became difficult to scale.

OpenAI gradually transformed Habitat into an intermediary service capable of handling routing, caching, authorization, encryption, data residency, isolation between applications, and traffic limiting.

This evolution illustrates a common principle in software architecture: a simple solution can be perfectly suited to the early stages of a product without having to remain permanent.

Python held up to 20 million requests per second

The technical account is also interesting in terms of technology choices.

OpenAI says its Python version of Habitat managed to exceed 20 million requests per second. The company therefore did not immediately replace Python simply because another language could theoretically be faster.

It chose to keep this architecture as long as other problems remained higher priorities.

It was only in 2026, when CPU and memory costs became significant enough, that a rewrite in Rust became relevant.

Rust significantly reduces the resources required

The new version of Habitat was rewritten in Rust in the second quarter of 2026.

According to OpenAI, it now handles approximately 95% of production traffic. Internal measurements show a sixfold increase in CPU efficiency and a fifteenfold increase in memory efficiency compared with the Python version, along with improved average and tail latencies.

These results concern the Habitat service and the workload measured by OpenAI. They do not constitute a universal comparison between the two languages.

The story of Habitat instead shows that technical choices depend on the product phase. Python made it possible to develop and evolve the platform quickly over several years. Rust becomes relevant when infrastructure gains become significant enough to justify the cost of a rewrite.

What businesses can take away

Few applications will ever need to handle several tens of millions of requests per second.

But the reasoning remains valid at a much smaller scale.

An architecture does not need to be designed from day one to support one billion users. Above all, it needs to be clear enough to measure its limits and progressively replace the components that become genuinely problematic.

Caching, traffic limiting, isolation, observability, and access control often become more important as a product grows.

The example of Habitat therefore reminds us that an infrastructure capable of scaling is not necessarily one that uses the most complex technologies from the outset: it is one that can evolve when real constraints emerge.

Official source OpenAI Engineering — September 11, 2026

Figures and technical results attributed to OpenAI, verified on September 12, 2026. The CPU and memory gains come from the internal measurements described in this publication.

• OpenAI Engineering — Rapidly scaling online storage to serve over 1 billion ChatGPT users, by Jon Lee, Chaomin Yu, and Ben Ries, published on September 11, 2026.