AI agent sovereignty: the real issue is not the model, it is dependency

Fable 5: the AI "kill switch" precedent

The Fable 5 affair struck a nerve because it illustrates a new kind of dependency: on the advanced AI models used as development assistants and as tools that augment human work.

When a team relies on an AI agent to write code, analyze a project, suggest refactorings, generate tests or help with debugging, it does not only depend on the quality of the answers. It also depends on the model's availability, its commercial policy, its jurisdiction, its security rules, its guardrails, its terms of use and sometimes even on political decisions taken abroad.

A model can become unavailable not because it is technically obsolete, but because a government, a provider or a regulator decides it should no longer be accessible under certain conditions.

For a development team, that is no small detail. If an AI agent is embedded in daily work - reading code, generating functions, assisting with design, interacting with an IDE - then its disappearance does not necessarily cause a visible outage, but a sharp drop in productivity and a break in working habits.

This is where AI sovereignty becomes a concrete subject, not only for automated workflows, but for the tools that directly augment people.

Mistral in the public sector: sovereignty or displaced dependency?

France is now pushing sovereign solutions built around Mistral AI for public sector staff. On paper, the logic makes sense: public data should be processed within a controlled framework, on compliant infrastructure, with a European provider able to meet security, confidentiality and governance requirements.

It is a pragmatic answer to excessive dependency on the large American providers. For the State, it is hard to accept that administrative documents, internal summaries or sensitive data should travel through infrastructure it does not control.

Real sovereignty is not just "using a national model". It means being able to change model, move your inference, keep your data, control your prompts, version your workflows, measure the quality of the answers and degrade the service intelligently if the main model becomes unavailable.

A sovereign AI agent should therefore not be designed around a single model. It should be designed around a multi-model architecture.

Not every workflow needs a frontier model

A common mistake is to assume that every AI use case requires the most powerful model available. That is false. In many cases, the most powerful model is above all the most expensive, the slowest, the most dependent on an external provider and the hardest to govern.

For structured data extraction, a small, well-framed model can be enough. If the input is clean, if the output schema is strict, if the examples are good and if automatic validation runs behind it, there is no real need for a huge model.

For asynchronous classification, the need is different again. A perfect real-time answer is not always what you are after. You want low cost, sufficient robustness and the ability to handle volume. In that case, a small or mid-sized local model can be more attractive than a very high-performing cloud model. You can even use relatively slow models, producing only a few tokens per second, while still getting good quality results. On asynchronous tasks, that latency is often acceptable. It lets you reuse existing infrastructure, such as older servers with limited memory, and turn those hardware constraints into an economic advantage.

For code generation, complex analysis, software architecture or multi-step reasoning, requirements ramp up quickly. The model has to understand context, reason across several files, avoid hallucinations, propose coherent fixes and sometimes handle complex technical dependencies. There, the more powerful models keep a real edge. In other words, sovereignty does not mean "doing everything locally": it means knowing what has to be local, what can be cloud, and what needs a backup strategy.

What you can realistically run locally

Today it is possible to host useful models on relatively modest machines. Not to systematically replace the best cloud models, but to cover a significant share of workflows.

On a machine with 8 to 16 GB of RAM, you can already run small quantized models for simple classification, field extraction, request routing, rephrasing or basic checking.

On a machine with 32 GB of RAM, or a graphics card with enough VRAM, you can use more comfortable models at 7B, 8B, 14B or sometimes more depending on quantization. That opens the door to internal agents able to process documents, answer from a knowledge base, pre-fill forms or assist business teams.

On a more powerful workstation, with 24 GB of VRAM or more, you can start targeting heavier models for code, advanced analysis, longer reasoning or more ambitious multi-step agents.

But there is no point pretending: a local model on a small machine will not deliver the same quality as a cloud frontier model. It will be more limited, sometimes slower, weaker at complex reasoning, less reliable on ambiguous tasks. Its value lies elsewhere: it stays available, controllable, cheap at scale and independent of an external API.

A realistic strategy: routing tasks by level of criticality

The right approach is to sort AI workflows into several levels.

Level one: simple, repetitive and verifiable tasks. Extracting dates, amounts, references, categories, customer intent, language, document type. These tasks can often be handed to small local models, especially when validation rules follow behind.

Level two: asynchronous tasks. Email classification, ticket pre-analysis, request scoring, draft preparation, non-critical internal summaries. Here latency is not always a problem: you can accept that a job takes a few seconds longer if cost and sovereignty are better.

Level three: sensitive tasks. Confidential data, customer information, public documents, regulated data, trade secrets. These tasks have to be handled in a controlled environment: local, sovereign cloud or certified infrastructure.

Level four: complex tasks. Development, architecture, legal reasoning, contract analysis, business arbitration, critical decisions. Here the model has to be more capable, but it should not be left on its own. Classic RAG approaches are now outdated in many cases: the shift is towards MCP (Model Context Protocol) architectures, far more capable, able to dynamically connect sources, tools and environments.

This logic avoids two traps: using an overly powerful model for simple tasks, or using a small local model for tasks that clearly exceed its capabilities.

Local as a continuity plan

Local hosting should not be seen only as an ideological alternative to the cloud. It should be seen as a continuity plan.

Even if a company keeps using very capable cloud models, it should ask itself: what happens if the API becomes unavailable? If prices change? If certain uses are banned? If the provider imposes a data retention policy incompatible with our internal rules?

A local model, even a less capable one, can keep a degraded service running. For example, if the main agent can no longer generate a complete answer, a local model can still classify requests, extract the important fields, prepare a draft or route the case to a human.

It is not perfect, but it beats a total shutdown.

The real issue: designing replaceable AI agents

An AI agent should never be built as a direct dependency on a single model. The business workflow has to be separated from the inference engine. The model should be an interchangeable building block, not the immovable core of the system.

This implies several good practices:

• standardize inputs and outputs;
• use strict JSON schemas;
• keep prompts and prompt versions;
• measure quality task by task;
• plan for several models depending on the use case;
• keep a local or sovereign fallback capability;
• trace important decisions;
• automatically validate whatever can be validated;
• keep a human in the loop for sensitive decisions.

In that logic, using an LLM proxy such as LiteLLM can play a key role. This kind of tool unifies calls to different providers (OpenAI, Mistral, Anthropic, local models, and so on) behind a single interface. You are no longer exposed to API format variations, parameter changes or provider-specific evolutions. In practice, it lets you:

• switch models without touching the application code;
• route requests dynamically based on cost, latency or expected quality;
• implement fallback strategies easily;
• centralize the management of keys, quotas and logs;
• test several models on the same use case without heavy refactoring.

This kind of abstraction is essential to make agents genuinely replaceable and to keep early technical choices from turning into structural constraints. AI sovereignty therefore does not come down to picking Mistral, OpenAI, Anthropic, Google, Meta or an open source model: it is about not making your company a prisoner of a single choice.

Conclusion: sovereignty is architecture before flags

The Fable 5 affair is a reminder that an AI model can become a strategic breaking point. The rollout of Mistral in the public sector shows that France is trying to regain control over its critical uses. Local models show that a third way exists: accepting that not every workflow needs the same level of power.

The right strategy is neither to put everything in the cloud, nor to run everything locally. The right strategy is to build AI agents able to pick the right model for the right task. For data extraction, a small local model can be enough. For asynchronous classification, a lightweight model is often an excellent compromise. For development, complex analysis or long-horizon agents, the more powerful models remain necessary. But in every case, the architecture has to plan for change.

AI sovereignty is not won with a national champion alone. It is won with systems able to survive an outage, a price change, a regulatory restriction, a change of provider and the next geopolitical crisis.