One of the more difficult questions facing organisations at the beginning of their AI journey is also one of the most basic: how much infrastructure are we actually going to need? The honest answer, in many cases, is that nobody knows yet.
AI adoption tends to happen in stages rather than all at once. Organisations try out different use cases, models and datasets, and some of that work pays off and moves into production, while other projects change direction or fall away entirely. What looks like the right requirement today can look very different six months later, and that creates an interesting infrastructure problem.
The cloud solves one part of the problem very well
When requirements are uncertain, the flexibility offered by hyperscale cloud makes complete sense. Rather than attempting to predict your eventual requirement, you can consume GPU resources as you need them, experiment, learn and increase capacity as successful use cases emerge. For organisations at the beginning of an AI journey, that flexibility can be extremely valuable.
Increasingly, though, it isn’t the only consideration. Questions around data sovereignty, jurisdiction, security, long-term economics and control over critical infrastructure are causing more organisations to think about what role privately owned AI infrastructure should play alongside public cloud.
Sovereignty deserves closer scrutiny than a “sovereign cloud” label alone can offer. It raises real questions:
- Where is the data?
- Who operates the infrastructure?
- Which jurisdiction applies?
- Who controls encryption keys, software, firmware and updates?
- What external dependencies remain?
Sovereignty isn’t a product. It’s a set of design decisions.
Craig Wilson, MD and founder, Thaldare
For some organisations, owning and managing more of the underlying infrastructure will form part of the answer. That brings us straight back to the original problem: how do you plan for infrastructure you can’t accurately predict?
Traditional server architecture forces you to buy in large chunks. Needing two more GPUs can mean approving an entire heavily configured 8-GPU server, what I think of as an investment cliff.
We’ve all seen this pattern in enterprise IT: buying far more capacity than needed because predicting three to five years ahead is nearly impossible. With GPUs such a large share of the cost of AI infrastructure, that stranded capacity gets expensive fast.
A different approach: make the infrastructure composable
This was the part I found most interesting at a recent lunch at Aqua Shard, hosted by Hammer AI Works, Liqid and AMD. Liqid takes a different approach: rather than permanently tying GPUs and memory to individual servers, its Matrix software and high-speed PCIe/CXL fabric technology let those resources be pooled and assigned to physical servers according to workload.
The commercial implication matters more than the technical detail. Infrastructure can grow in smaller increments, instead of the next stage of an AI project requiring another substantial server purchase, aligning investment much more closely with how an organisation’s AI project actually progresses.
That looks like:
- 1Experiment.
- 2Prove the use case.
- 3Create value.
- 4Add capacity.
- 5Then repeat.
It is much closer to the consumption logic organisations have become accustomed to in cloud, but applied to infrastructure they own and control.
Smaller growth increments could change the AI business case
AI infrastructure planning is happening against huge uncertainty. Organisations may know they want internal AI capability without knowing which models will be used, how fast demand will grow, or how much capacity they’ll eventually need. Requiring a large investment every time capacity increases only makes that harder to manage. Smaller increments let investment follow real, realised returns instead.
It doesn’t remove the need for proper planning, or make private infrastructure automatically cheaper than cloud. But it gives organisations a middle option: consuming everything from hyperscalers indefinitely, or betting on a large infrastructure purchase upfront.
GPU diversity is another interesting consideration
There’s another aspect of composable infrastructure worth flagging: silicon diversity. Liqid supports GPUs from multiple vendors: AMD, NVIDIA, and increasingly CPU-based options few are considering. As the accelerator market evolves and pricing, availability and architectures shift, that flexibility means organisations aren’t locked into whichever vendor they chose for their first deployment.
Perhaps the objective shouldn’t be to predict the final destination
There is a natural tendency in enterprise infrastructure planning to want to design the end state. With AI, that may be the wrong starting point. For organisations still developing their use cases, perhaps the better question isn’t:
What will our AI infrastructure need to look like in five years?
It is:
How do we build an architecture that allows us to make sensible investments as we learn?
Public cloud will remain right for many workloads, private infrastructure for others. Most organisations will land somewhere between the two. What’s interesting about Liqid’s approach is that composable private infrastructure means that choice no longer has to be so stark.
And when nobody can say with certainty exactly how much AI infrastructure they will need a year from now, let alone five, the ability to change the answer may be more valuable than trying to predict it perfectly in the first place.
