Who Feeds the GPUs? Inside AI's Hidden $30B Layer | Renen Hallak, VAST Data

Buying GPUs does not by itself create a working AI service. In this interview, VAST Data founder Renen Hallak describes a stack with power, hardware, infrastructure, models and applications. His company occupies the infrastructure layer, where storage, networking, databases and compute coordination determine whether the other layers can work together.

The framing matters because an “AI factory” need not mean that every organization owns a physical data center. Hallak also includes enterprise-controlled environments and cloud arrangements. The common ambition is control over how an organization uses its data and intelligence, rather than a single prescribed ownership model.

This is a supplier’s view of an expanding market. It helps identify the work hidden beneath a model API, while the interview’s growth forecasts and claims about competing architectures remain the founder’s arguments rather than independent benchmarks.

Hallak traces VAST’s starting point to storage trade-offs: performance, price, resilience, scale and ease of operation were difficult to obtain together. His architectural answer separates compute from a shared data layer so different machines can reach the underlying data and metadata.

He contrasts that design with shared-nothing systems, where data is divided among nodes and distributed work can require coordination between them. In his account, larger AI workloads make those coordination costs increasingly important. The useful question is which data each part of a workload needs, and what movement or coordination is required to obtain it.

The interview does not establish that every shared-nothing database fails at a particular universal node count. Nor does disaggregation eliminate networking. Hallak is advocating a design choice for the workloads his company targets; customers still need to evaluate its behavior against their own access patterns and operating constraints.

At 30:21, the interviewer asks whether adopting the platform means moving all enterprise data to VAST. Hallak says the starting point is usually a new AI application, with access expanding if the organization finds that initial deployment useful.

He describes support for familiar interfaces: S3, file systems and block devices for storage, SQL for database access and Kafka for streaming. His proposed bridge lets older applications operate alongside newer agent workloads rather than demanding an immediate wholesale replacement.

The interviewer also raises data gravity and dependence on one vendor. Hallak answers that standard interfaces make moving away possible. That is an important distinction from proving migration has no cost: interfaces alone do not describe the time, data volume, application dependencies or operating work involved in an exit.

Hallak gives training a deliberately simple description: many GPUs consume large amounts of data and periodically save checkpoints. The emphasis is fast access at scale and the ability to recover without restarting everything after a failure.

Inference adds people waiting for responses. At 33:55, he shifts to low latency, geographic distribution, resilience and security. His contrast is operational: an interrupted training run delays work, while interrupted inference can mean that the user-facing service is unavailable.

Adding multiple models and agents creates further decisions. A programming task may deserve one model; a low-value task may call for a cheaper one. Sensitive information may be permitted in one environment and prohibited in another. Availability and routing therefore become part of the data system’s responsibility, not merely matters of reading files quickly.

At 35:43, Hallak describes an optimization beyond choosing a model: route the request to a GPU that already has that model loaded and holds relevant context in cache. The same logical request can have different operational costs depending on where its state lives.

He connects recent conversation context with longer-lived memory and retrieval. The system must manage raw inputs, generated outputs, model weights and context, while keeping older information accessible when a task needs it. That broadens the meaning of “feeding the GPUs” from bulk data delivery to coordinating several kinds of state.

Permissions complicate the picture. If information was used in fine-tuning, the organization also needs to consider what later model interactions could reveal. The interview identifies that governance problem; it does not demonstrate that tracking training inputs alone prevents every possible disclosure.

Hallak starts with existing authentication and authorization mechanisms, including directory identities and access-control lists. He says those policies can extend to agents through the people or other agents deploying them, with role-based and attribute-based access adding finer distinctions.

The scope goes beyond which record an agent can read. At 39:10, the discussion includes permitted tool calls and communication partners. Persistent streaming provides a record of conversations that the platform can query, connecting observability to policy enforcement across locations.

Physical AI raises the stakes further: an action can harm the world rather than merely expose a document. Hallak wants infrastructure in the data path to intervene, not only report afterwards. That is a proposed responsibility and product direction, not evidence that general robot or vehicle safety has been solved.

The Data Enclave discussion begins with conflicting concerns. Model builders want to protect proprietary weights; enterprises may be unable to send sensitive data into a model provider’s environment. Hallak proposes running inference in enterprise premises or a controlled virtual private cloud while keeping model weights encrypted.

He describes hardware support for confidential execution and protected movement through network, memory and GPU stages. The infrastructure layer must also know which model belongs to whom, who may access it and where policy should block access. In his account, the advance is ecosystem integration rather than a single newly invented cryptographic primitive.

These are the intended trust boundaries of the design. The interview’s absolute assurances about nobody seeing enterprise information are vendor claims, not a substitute for examining a particular deployment’s threat model, attestation, configuration and contractual guarantees. The useful insight is the two-sided problem being addressed, not an unconditional security promise.

Asked whether demand can support the infrastructure boom, Hallak describes customers repeatedly increasing storage plans. One AI-cloud customer, he says, first discussed roughly 500 petabytes over three years and later requested another two exabytes. He expects further increases, but that expectation is a forecast.

His explanation of the bottleneck is physical: land, power, chips and the time required to build capacity. He also describes financing gaps when infrastructure spending must precede returns, arguing that companies inside the ecosystem may be willing to supply liquidity because they understand the demand.

These anecdotes show what this supplier reports seeing. They do not independently establish utilization across the industry or eliminate financing risk. Hallak himself allows for future hiccups; his suggestion that the pace could continue for five to ten years depends on AI delivering the capabilities he anticipates.

For NeoCloud operators, Hallak lists access to power, hardware and money, then stresses the ability to build. Customers who need compute immediately may choose an operator with capacity ready rather than wait for a joint build next year.

He argues that repeated deployment creates practical expertise and that cheap financing alone does not confer it. That is his response to the idea that larger hyperscalers must inevitably win because their cost of capital is lower. It remains a competitive interpretation, not a settled prediction of which companies survive.

The sovereign-AI discussion makes another distinction: jurisdictional control is not the same as producing every layer domestically. A country may value infrastructure within its borders without being able to manufacture the chips or develop all the software itself. Hallak frames that as a motivation he sees, not a complete policy prescription.

Hallak looks to earlier computing eras to argue that infrastructure software can capture substantial value. He is less certain where the model layer will settle, while the interviewer notes that model builders and application companies increasingly move into one another’s territory.

Nvidia appears as both investor and important technical partner. Hallak describes extensive joint engineering while saying the relationship is not exclusive and VAST also works with AMD and others. Those are statements about the relationship at the time of the interview, not a newly verified partnership announcement.

The tension is useful: participating deeply in a dominant ecosystem can accelerate product development, while preserving alternatives matters to independence. Neither the interview’s historical analogy nor its partnership account resolves where future profits will ultimately accumulate.

The most concrete management advice arrives near the end. Hallak describes speed as repeatedly finding a limiting factor, removing it and then locating the next one. As CEO, he says, he does not automatically know where every constraint sits; the people doing the work often do.

That leads to his preference for direct access and an organization where problems can be raised without fear. Bad news needs to travel clearly and often, rather than being softened by a chain of command or buried under self-congratulation.

The interview closes with much broader speculation about AI and society. Those possibilities are not necessary to use the operational lesson: identify what is actually slowing the system, hear from the people closest to it and check what becomes limiting after a change. That remains practical even when the industry’s long-term trajectory is uncertain.