Your own AI model or the public cloud: what to choose

Two years ago the answer was simple: cloud models were incomparably better, and running your own required a research team. Today open models that run on a single server with a GPU card are enough for most business tasks, and the cloud has acquired processing agreements and EU regions. The decision has stopped being technical and become a business one.

What “your own model” really means

Your own model does not mean the company trains something from scratch. It means a ready, open language model runs on hardware under your control: on a server at the company or in a data centre where you have a dedicated environment. Prompts and documents do not leave that server.

On top of it there is usually a layer that connects the model to your data: search across documents, a knowledge base or a case system. That layer delivers most of the value, not the model itself. And it is the layer that needs an environment with a database, backups, access control and someone to run it.

The public cloud, in turn, is access to a vendor’s models through an API or through an assistant in the office suite. You run nothing, pay per use, and the vendor processes data on the terms of the agreement.

Criterion one: quality of answers

The largest cloud models are still better at hard tasks: multi-step reasoning, writing code, working on very long documents. If that is your use, the cloud wins.

Local models match them on repetitive, well-defined tasks: ticket classification, summaries, answering questions from your own knowledge base, extracting data from invoices and contracts. In a company that is most of the work. The quality difference is worth measuring on your own documents, not on leaderboards, because leaderboards measure tasks a company does not perform.

Criterion two: data

Here the decision is often simple. If the data is covered by professional secrecy, sector regulation or NIS2, it should not go to the cloud, even with an agreement. A law firm, an accounting office, a medical practice, a manufacturer with process documentation: local model.

If the data is internal but not sensitive, the cloud under a company agreement, with an EU region and training switched off, is sufficient. How to check that with a vendor is covered in where to keep your data when adopting AI.

Criterion three: cost

The cloud is cheaper to start: no hardware, you pay for usage. At low volume, a few thousand prompts a month, it stays cheaper for good.

Your own model costs a GPU server up front, or a subscription for an environment in a data centre, and then operations. It starts to pay off when volume is high and steady: thousands of documents a day, an assistant used by the whole company, overnight processing. Then the cloud bill grows linearly with usage while the server cost stays flat.

There is also a cost that does not appear on any price list: people’s time spent on operations. A GPU server standing in the company without backups, monitoring and updates is a problem a year later. That is why your own model makes sense either with your own IT team or in an environment someone runs for you.

Criterion four: response time and availability

A cloud model depends on the internet and on the vendor. For an assistant in the office that is not a problem. For a system on a production line, in a vehicle or in a facility that has to work during a link outage, it is a fundamental problem. A local model answers inside the company network, with no dependency on the outside world.

Then there is predictability. Cloud vendors change models, prices and limits. Your own model works the same next year, until you decide to change it.

The mixed model, or how most companies end up

In practice you rarely pick just one. A typical setup looks like this: tasks on sensitive data and high volumes go to the local model, hard and rare tasks to the cloud under a company agreement. The user does not choose where a prompt goes, because the rule is built into the environment.

Such a setup needs one place where both models are available on the same terms: with company login, logs, backups and cost control. Without it the company ends up with three tools and nobody knows which is for what.

Questions before the decision

Which tasks do you want to hand to AI, and are they repetitive or rather hard and rare. Which data will go into them, and is there any that must not leave the company. How many prompts a month will there really be, today and in a year. Does the application have to work without the internet. Who will run the server if it stands in the company.

Answers to these five questions are enough to choose a route. If you do not know how to answer them, start with a small pilot in the cloud on data that may go there, and measure quality and volume. You will then decide about your own model on numbers, not on a hunch.