When cloud AI becomes a risk to your business

I can’t count how many times for the past couple of months I was talking about the power and advantages of local AI models. I believe they are the future of AI and I will lay my reasoning in this post.

Blocked accounts

Only this week I encountered 4 people with frozen business accounts of a major cloud LLM provider. Most of them paid for the full year. These people have businesses that rely heavily on AI. They had to spend a lot of resources to adopt a new AI provider but also “fight” the provider with their decision. And as you start to notice major companies nowadays are deploying AI agents acting as customer support which makes it extremely hard to reach a human to understand you.

So this is just one example where a cloud AI provider can drain your business financially if one does not understand the consequences of putting all your eggs in one basket.

Decommissioning models

You finally find the right model to produce sustainable results for your business processes to work smoothly and then the model got decommissioned. Models are trained on different data, have different reasoning, handle tools differently, and in the case of LLM providers they have different guardrails and protections. There are many things you don’t control that can influence the output of the model.

Transmitting data to cloud

Check whatever tick boxes you find but no matter what your LLM provider says about using your data you are still transmitting data to the cloud. End of story. Especially when you work with sensitive data that should not leave your infrastructure (due to regulations, company policies or just out of privacy concerns).

Hallucinations and out of scope output

LLMs are trained on large amount of data. When it comes to specific business processes that require specific set of knowledge a large language model can be a liability. It can go in a direction that has nothing to do with your business and propose actions (or even do if there is an autonomous agent) that does not correlate with your business targets, values, vision.

Price per token

I cannot say that the AI models are currently cheap but the price that you pay is heavily influenced by huge subsidies and investments. All of the LLM providers are losing money from their AI business. ALL. You can check whatever news you want but the fact will not change. The only thing that keeps these companies running are investments and the promise of future profits (capital expenditures from other big companies that are planning to pay the AI providers). Anyway, I don’t want to talk about the financials of these companies nor the global financial situation, what I want to share is that the price that you pay today per token is not the real number that can make these companies profitable. And unless the needed computing power cut down significantly there is no way this price stay like this. So if you can afford 10000, 50000 or even 1000000 per year for tokens consider if you can afford to pay 10 times or even more.

There are also other reasons to avoid cloud AI but right now I can see the above listed as the main pillars. I am not ruling them (LLM providers) out. They have their place and use:

• They are powerful,
• They are accessible,
• They are protected (in a way),
• They are (relatively) cheap.

Local LLMs have their challenges though:

• They need hardware
• They need infrastructure to run
• They need maintenance
• They need security. And all this cost money!

But you own the entire process with the infrastructure and your data and you have full control of what is happening.

So if you are building a business with AI right now or enabling AI in your organization consider all the pros and cons for cloud and local LLMs. Make AI support your business processes not the other way around. AI should be a companion and you have to make it easily replaceable. Like you have a well made machine and AI is just one part of this machine. Be prepared to have a spare part or at least know that it exists and you can find.

API Gateways

Speaking from a technical perspective a good way to prepare your infrastructure is to work with API gateways.

An API Gateway (for LLM providers) is a single entry point that sits between your application and multiple LLM providers, such as OpenAI, Anthropic, Google Gemini, etc. Instead of integrating separately with every LLM provider, your application communicates with one API Gateway, which handles routing, authentication, monitoring, cost management, and provider selection.

Having such API configured it will make it fairly easy to switch between providers. Not only that but:

• You can set limits on each API key;
• You can monitor spendings
• Make specific routing (i.e. some applications can use cloud LLM other local)
• Monitor suspicious activities or just patterns and behaviour
• And better protect your environment overall.

Summary

If you are using or planning to use cloud AI - that’s fine. Just be aware of the risk especially if your business relies on AI for key business processes.