If you use large language models (LLMs) at scale, you may have noticed a pattern. Token prices are falling (even more so for older models), while the price of electricity at the data centres powering them keeps climbing. So where does the money really go? And should you worry about oil prices staying high? The answer is probably not quite what you might expect.
Where the money goes
A query to a model such as GPT-4o (already superseded, but it illustrates the point) uses about the same electricity as running your fridge for a few minutes. That sounds like a lot until you look at what you are actually paying. At around $2.50 per million input tokens, the provider is effectively covering the energy at something like 4.5 cents per kilowatt-hour. In most developed countries, commercial power costs three times that.
The margin on inference (the day-to-day use of AI) is therefore arguably thin, which suggests AI companies are racing to capture market share with loss-leading prices. Whether this is sustainable or will end in boom and bust is open to debate, but it looks a lot like the dotcom boom of the late 1990s and early 2000s. We are already seeing AI company IPOs hoping to ride this wave, consolidating market share with enough financial resource to muscle out the competition.
For businesses that simply want to use AI, this is not an abstract concern. Committing to AI adoption has short- and medium-term consequences that need to be carefully aligned with business and technology planning. Avoiding lock-in is important and, as with previous technology waves, it takes expertise to spot when lock-in is happening, whether by design or by accident.
Energy isn't the biggest cost, yet
Energy is not even the largest cost. The GPUs (the graphics chips that are critical to training and running LLMs) still dominate, followed by cooling and operations. Energy accounts for around a quarter of the operating budget today, though some projections suggest it could become the single biggest cost by the end of the decade.
It helps to think of it like airline fuel. Fuel matters to the airline, but the ticket price is shaped more by competition and how full the plane is than by what the airline paid for the fuel.
Training versus inference
This creates a clear divide between the cost of training and the cost of inference. The crypto boom taught us that miners will chase cheap power, to Iceland, for example, where geothermal electricity costs pennies per kilowatt-hour. The same pattern is repeating for AI training. Microsoft and Amazon are building enormous facilities in Quebec and the Nordic countries because the power there is cheap and stable.
Training is a bit like baking bread: you need the oven on for hours, but you are free to choose when. Inference, the part that answers your questions in real time, is more like a restaurant kitchen; it has to be close to the customers. The geographic spread is also striking. Singapore pays roughly 14 to 18 cents per kilowatt-hour because it imports nearly all its gas as liquefied natural gas (LNG). Iceland pays three to five cents. Yet their prices for LLM API use (inference) are almost identical. Iceland’s energy advantage is largely eaten up by other overheads, including shipping, staffing and serving customers who are mostly based elsewhere, and competitive pressure flattens what remains, leaving geography largely invisible to the end user.
Will oil prices push up your AI bill?
Oil prices are elevated and, with disruption in the Middle East widely expected to persist, are likely to stay that way for some time. So will your API bill follow them upward? Historically, no. Token prices have trended downward through every energy price cycle of the past three years, because gains in model efficiency and hardware have outpaced anything electricity markets could throw at them. The race to the bottom has masked any upward pressure from power bills.
The real constraint: grid capacity
There is, however, a structural issue bubbling underneath that may be more pressing than oil: power distribution grids are struggling to keep up. In Northern Virginia, the densest concentration of data centre capacity on the planet, new facilities now face four-year waits just to get connected. Similar bottlenecks are appearing elsewhere, for example in Singapore and Ireland. You can order the GPUs and start construction, but if the substation doesn’t exist yet, you won’t be switching anything on any time soon.
This kind of energy market friction will cause disruption, large or small, which in the medium term is likely to affect the price of tokens, and the bottom line of businesses that have become too reliant on them.
Practical ways to reduce the risk
So what practical mitigations might help? Local models have quietly crossed a threshold for routine work. For example, Google's Gemma 4 running on a consumer GPU could handle roughly 70 to 80 percent of everyday coding and analysis tasks at near-zero marginal cost (though probably not full “agentic” development, the new frontier of software development). The hardware pays for itself in about 18 months if you consume a couple of million tokens a day. Local LLMs do have downsides, such as non-centralised control and a lag in support for the latest models, for example, but they are a viable option for some organisations.
If you want to stay ahead without over-investing, the pragmatic approach is to route simple queries to cheaper models automatically, send only genuinely hard reasoning problems to frontier APIs, and cache aggressively so you don't pay twice for the same answer. Software developers should also consider using locally running tools that reduce round trips to LLMs.
None of this means energy costs are irrelevant. Sustained high energy prices could eventually compress margins enough that providers slow their price cuts or introduce regional pricing. But the immediate risk is not the price of a barrel of oil. It is whether there is enough physical power infrastructure to run the machines we have already bought.
What this means for our customers
So, getting to the heart of the debate, how does this affect the advice we give our customers?
- Look at the supply side. Review the available LLMs, their relative costs today and the competition for the specific applications in question. LLMs can be more or less expensive “token hogs” depending on capability, speed, context window size and other technical features. These should be matched to actual and predicted need, rather than relying on an all-you-can-eat model. It’s the same process as for any service a business buys in. Consider, too, alternative data centre providers that self-host LLMs and are open to negotiation.
- Look at the demand side. Companies that encourage, or even mandate, employees to use AI to the max will spend a lot of unnecessary money learning what works and what doesn’t. They should instead target their usage and build tools that apply AI in cost-effective, efficient ways. Look also at ways to reduce token traffic and keep processing local where possible.
