Open Source and the path to digital sovereignty: A practical example of hybrid AI infrastructures
Why digital sovereignty is not a luxury, but a necessity
The current debate surrounding digital sovereignty is too often conducted purely theoretically. We have decided to tackle it practically. In a world increasingly dominated by a few large multi-cloud providers and simultaneously fractured by geopolitical upheavals, the question, "Where is my data located and who controls it?" is no longer just relevant for intelligence agencies. It has become a crucial strategic consideration for every data-driven company.
True sovereignty doesn't mean demonizing the cloud. It means having the freedom to choose. Hybrid integration scenarios that combine the best of both worlds - the scalability of the cloud with ultimate control over on-premises components - are key. Our recent work on our on-premises data center is a concrete example of how to gradually regain this sovereignty without sacrificing cutting-edge technology.
The foundation: A data center setup for maximum control and security
Our goal was to build an infrastructure that is flexible enough for modern workloads (such as AI) but makes no compromises on security. To achieve this, we consistently rely on a pure open-source stack.
Here are the details of our setup:
- The virtualization hub (Proxmox VE): At its core is a Proxmox cluster. This Debian Linux-based KVM virtualization solution offers us the flexibility to efficiently manage virtual machines (VMs) and containers without license cost lock-in.
- Network segmentation (VLANs): In addition to the physical base network, we have created several virtual local area networks (VLANs). Each VLAN forms an isolated zone in which specific systems are located. All management and routing between these networks is handled centrally via our central switch and firewall. This allows for granular control over which systems are permitted to communicate with each other.
- Network Backbone (LACP & VLANs): To ensure reliability and performance, we have combined the physical network interfaces of the Proxmox host into a logical bond using LACP (Link Aggregation Control Protocol). This technology combines multiple Ethernet connections into a single logical connection, increasing bandwidth and providing redundancy in case a physical connection fails. The logical separation of the data streams from the various VMs is consistently achieved via the previously defined VLANs (Virtual Local Area Networks).
Strict Network Segmentation for AI Workloads The core of our security architecture is the strict separation of network segments, as innovative technologies like artificial intelligence (AI) process massive amounts of data and therefore require a particularly high level of protection. To achieve this, we have consistently divided our network into logical zones. The foundation is the management network (VLAN 20), where the critical infrastructure – from the Proxmox access interface to the management interfaces of the switches and firewalls – is located and completely isolated from the outside world. Separately, the AI network (VLAN 50) exists as a dedicated high-security area, where two VMs currently operate for AI services such as OpenClaw. This "security by design" concept stipulates that the AI network has no direct connection to the outside world or to other internal networks by default. Access to the services is instead controlled via PIN holes. Our central firewall, based on OPNsense, precisely monitors every packet flow between the VLANs. Secure administrative access from the management network to the AI VMs is granted only via specific, password-protected rules. Regular user access to web interfaces is also not direct, but rather via a reverse proxy in the standard network, which forwards requests to the respective AI VM through a precisely defined PIN hole. This setup guarantees that even if a service is compromised, an attacker remains isolated within the secure VLAN 50 and cannot access the management network or other sensitive data.
Outlook: The next steps towards digital sovereignty
We have laid a solid foundation, but the journey is not yet over. To complete our vision of a fully controllable and independent infrastructure, two major projects are on our roadmap:
- Setting up an on-premise LLM (Large Language Model): After successfully segmenting our network, we will host a high-performance, open language model (such as Llama or Mistral) directly within our isolated AI network (VLAN 50). This will allow us to achieve a new level of data protection: All requests and processed data will remain 100% within our own infrastructure, without ever touching an external cloud.
- Setting up a Kubernetes cluster with OpenDesk: To modernize and scale the orchestration of our container workloads, we are planning to set up a Kubernetes cluster (K8s) within our Proxmox environment. Based on this, we will deploy OpenDesk (the open-source business platform). The goal is to create a sovereign alternative to proprietary office, collaboration, and communication tools that integrates seamlessly into our hybrid architecture.
These steps mark the transition from a mere infrastructure provision to a fully-fledged, sovereign application platform.
Conclusion: Open Source dominates the stack
Our work to date clearly demonstrates that open-source technologies dominate the entire stack – from the hypervisor and network to security. It works consistently reliably and efficiently, without the unpredictable risks of proprietary vendors. Digital sovereignty is achievable – you just have to do it!
Do you have questions about our setup or are you planning similar hybrid integration scenarios? Feel free to contact us. We'd be happy to share our experience.
