From GPUs to token factories: The next phase of enterprise AI is about infrastructure
As AI moves from experiment to everyday operations, the question for business leaders is shifting from which model to use to how reliably – and at what cost – AI work can be delivered.
Whether a business is experimenting with ChatGPT or already running AI across its operations, the leadership question is moving beyond what a chatbot can answer. It is becoming a question of how much useful work AI can take on, what that work costs and how reliably it can be delivered.
For business leaders, the challenge is to connect AI investment with operating results: faster document processing, shorter service queues or more efficient software delivery. Decisions about where to obtain compute, and who should operate it, are now part of that productivity agenda.
At Nebula Data, we have seen this shift in customer conversations – from server specifications to securing GPUs and, increasingly, to accessing usable compute and model services.
I find it helpful to describe it through four overlapping phases. This is a framework for understanding customer priorities, not a universal technology timeline.
From tools to control of compute
AI 1.0 was the discovery phase. Tools such as ChatGPT gave office workers an accessible way to draft, summarise and explore information, often before organisations had a wider deployment strategy.
You might like
AI 2.0 brought attention to model development and customisation. For some businesses, training or adapting a model offered a real advantage. For others, the visibility of model builders made ownership seem more important than it needed to be.
In AI 3.0, access to compute became strategic. A capable model was only part of the requirement; businesses also needed capacity to run it. Dedicated GPUs, deployment locations, technical support and delivery dates became central to planning.
The deployment gap I call AI 3.5
The transition is complicated by physical constraints. The International Energy Agency’s analysis of energy and AI describes bottlenecks across energy infrastructure, advanced chips and other data centre equipment. Power connections, cooling readiness and hardware delivery can all affect when a deployment becomes usable.
I think of AI 3.5 as the gap between wanting to deploy AI and having the infrastructure ready to support it. A business may have the application and the budget, yet face a deployment schedule that does not match its operating plans. Buying available capacity can bring that work forward while a dedicated deployment is prepared.
The right choice will differ by workload. One organisation may need dedicated GPUs for predictable demand and specific controls. Another may want managed inference, while a team testing an application may prefer a token-based API. These options can coexist – and access to a wider pool of infrastructure does not remove data residency, security or latency requirements.
AI 4.0 and the token factory
The fourth phase is organising AI around sustained production. Token factories already exist; what is developing is their role in how enterprises obtain and measure AI capability.
For language-model inference, a token factory combines GPU infrastructure, networking, model-serving software and operations to process and generate tokens at scale. Its value lies in delivering that work efficiently under an application’s quality, response-time and reliability requirements.
For a customer buying a managed service, it can turn complex infrastructure into metered usage – much like a utility bill, paying for consumption without owning every asset behind the service. A token factory can equally serve a private enterprise deployment; it does not require public-cloud delivery or a single billing model.
This reflects a changing division of work across the AI economy: infrastructure operators provide capacity and operational expertise, while enterprises decide how to apply that capability to their own business.
How that shift is shaping Nebula Data
Our initial focus was building a local compute offering in Australia. Constraints on locally available capacity led us to broaden our sourcing and infrastructure partnerships into Europe and the US. Through NEB Compute, we are bringing large-scale GPU clusters online across Australia, Europe and the US.
As that network develops, our platform is evolving towards a token factory model, bringing dedicated compute capacity, model deployment and metered inference services together.
Measurement remains essential. Tokens are useful units of usage, but they are not interchangeable units of business value. Model quality, prompt length, repeated attempts and human review all change the total cost of completing a task. Enterprises should assess cost per successful task alongside response times, reliability and governance.
The decision for enterprise leaders
The practical next step is to identify a valuable workload, estimate its demand and agree how success will be measured. Then compare owned infrastructure, dedicated capacity and managed services against the same requirements.
The urgency is not in buying compute for its own sake. It is in making a grounded deployment decision, with a realistic route from investment to useful work.
Darren Shamsharip is the founder, managing director & CEO of Nebula Data Pty Ltd, an Adelaide-based AI infrastructure and platform services company. Its NEB Compute platform provides enterprises with access to dedicated GPU compute capacity and token-based AI model services.
Want to see more stories from InDaily SA in your Google search results?
- Click here to set InDaily SA as a preferred source.
- Tick the box next to "InDaily SA". That's it.
