Field note status: this article records a real workload assessment and migration plan. It does not claim that every listed migration has already been completed.
This k3s migration started with a simple goal: move everything that can realistically run on k3s.
Before changing any workload, I needed a repeatable way to decide what should move, what should remain on a virtual machine and what should simply be retired.
Table of Contents
The expected benefits were also reasonable. Fewer virtual machines should mean less operating-system maintenance, better use of compute resources and a more consistent deployment model. GitOps could describe the desired state of the applications, while Kubernetes would handle scheduling and restarts.
But “can it run in Kubernetes?” turned out to be the wrong first question.
The better question was:
Does moving this service to Kubernetes make the complete system easier to operate and recover?
That changed the k3s migration plan. Some application VMs became obvious k3s candidates. Some infrastructure services were better left alone. One obsolete system did not need a migration at all—it needed a backup and retirement plan.
The environment I was evaluating
The existing Proxmox environment contained a mixture of very different workloads:
- internal web and API applications;
- an asset-management application that had already been migrated to k3s;
- SonarQube;
- a self-managed GitLab instance and several runners;
- a network-management controller;
- an old wiki;
- Active Directory, DNS, DHCP, NPS, AD CS and WSUS;
- Veeam components;
- development and test VMs whose future was not yet clear.
Treating all of them as equivalent “VMs to eliminate” would have produced a bad architecture. They have different support models, state requirements, dependencies and recovery paths.
The k3s migration matrix
After reviewing the inventory, the practical plan looked like this:
| Workload type | Decision | Reason |
|---|---|---|
| Asset-management application | Already on k3s | Application, database and uploads were migrated with a verified rollback path |
| Test advertising application | Migrate next | Good candidate for an inventory-driven application migration |
| Internal finance application | Migrate later | Application workload already expected to move to Kubernetes |
| Development and test web applications | Migrate where practical | Good fit for shared cluster capacity and GitOps |
| SonarQube | Strong candidate | Official Kubernetes deployment path exists, with storage and database planning required |
| Self-managed GitLab | Keep on a VM for now | Operational complexity and disaster-recovery dependency loop |
| Legacy GitLab runners | Keep until projects change | Some jobs still depend on the VM environment |
| Network-management controller | Keep on a VM | Official deployment model is Linux/OVA; container option was not vendor-supported |
| Old wiki | Back up and retire | No active use; migration would preserve unnecessary complexity |
| AD, DNS, DHCP, NPS, AD CS and WSUS | Keep on VMs | Foundation services that the cluster may depend on |
| Veeam infrastructure | Keep on VMs | Part of the recovery system, not an application migration target |
| Unclear development or utility VMs | Investigate first | No migration decision without process, port, data and ownership inventory |
This matrix reduced the project from “remove as many VMs as possible” to a sequence of explicit, defensible decisions.
k3s migration candidates: application workloads
The clearest candidates are conventional application stacks:
web or API
+
database
+
optional Redis
+
persistent files
+
scheduled tasks
These workloads benefit from a standard deployment structure:
- manifests stored in Git;
- Argo CD reconciliation;
- namespaces and resource controls;
- repeatable configuration across environments;
- consistent ingress and certificate handling;
- centralized observability;
- worker capacity shared across several small services.
In this k3s migration, the test advertising application is the next practical candidate. Before converting anything, its VM needs a real inventory:
docker ps -a
docker compose ls
systemctl --type=service --state=running
ss -lntup
df -h
lsblk
ps aux --sort=-%mem
The commands are less important than the questions they answer:
- What actually runs?
- Where is the database?
- Are there uploaded files or other persistent data?
- Does the application use Redis?
- Are there cron jobs or background workers?
- Which environment variables and secrets exist?
- Which DNS names and ports are part of the production behavior?
Only after those answers exist should the Kubernetes manifests be designed.
SonarQube: supported, but still stateful
SonarQube is a strong migration candidate because SonarSource provides an official Helm-based Kubernetes installation path. Its current documentation also says that a production deployment should use its own database and that resource, storage and startup behavior need explicit planning. See the official SonarQube Kubernetes prerequisites.
That makes SonarQube suitable for k3s, but not disposable.
The migration still has to account for:
- the existing database and its version;
- database backup and restore;
- Elasticsearch-related filesystem requirements;
- CPU and memory sizing;
- startup, readiness and liveness probes;
- persistent data that must survive a Pod replacement;
- rollback to the original VM if validation fails.
“There is an official Helm chart” is a useful signal. It is not a complete migration plan.
k3s migration boundaries: services that stay on VMs
A successful k3s migration does not need to end with zero virtual machines.
In this environment, Active Directory, DNS, DHCP, NPS, the certificate authority, WSUS and Veeam form part of the infrastructure around the cluster. Moving them into k3s could make Kubernetes depend on services that themselves depend on Kubernetes.
The healthier boundary is:
VM infrastructure
├── identity and directory services
├── DNS and DHCP
├── network authentication
├── certificate authority
├── patch management
├── backup infrastructure
├── GitLab
└── selected vendor appliances
k3s application platform
├── internal web applications
├── APIs
├── application databases where appropriate
├── SonarQube
├── development and test workloads
├── Kubernetes-native runners
└── future application services
This separation also leaves useful recovery tools available when the Kubernetes platform is unhealthy.
Why GitLab stays outside the cluster
GitLab can run on Kubernetes. That fact alone does not make it the right move for this environment.
The official GitLab Helm installation documentation describes a cloud-native deployment with multiple components and external PostgreSQL, Redis and object storage. GitLab’s production guidance also recommends a cloud-native or hybrid reference architecture rather than treating the platform as one simple container.
For a small self-managed installation that already works on a VM, moving it would create considerably more operational surface area.
There is also a more important local reason to leave it where it is. GitLab stores the repositories that describe the k3s desired state:
GitLab
↓
GitOps repository
↓
Argo CD
↓
k3s
If GitLab moves into the same cluster, a complete cluster failure can produce a bootstrap loop:
k3s is unavailable
↓
GitLab is unavailable
↓
Argo CD cannot read the manifests
↓
the manifests needed to rebuild services are inside the failed platform
Keeping GitLab on a separate VM preserves a recovery anchor:
GitLab remains available
↓
restore or rebuild k3s
↓
install and connect Argo CD
↓
synchronize the desired state
↓
restore databases and persistent files
In this case, not migrating GitLab is a resilience decision, not unfinished modernization.
Why the network controller stays on a VM
The network-management controller was another tempting candidate because community container projects existed.
However, the vendor’s documented deployment methods are a bundled installation on supported Linux distributions and a virtual appliance. The official GWN Manager installation guide describes its own Linux installation, services and network ports.
Moving a working network controller to an unofficial container image would remove one small VM while adding a custom packaging and upgrade responsibility. The risk-to-benefit ratio was poor, so the VM stays.
A vendor-supported container release could change that decision later. Until then, a small, boring VM is the better platform.
Legacy runners are temporary exceptions
Some GitLab runners can move into k3s, especially for newer container-based build and test workflows. Others still belong to projects with VM-specific dependencies.
The practical model is mixed:
GitLab
├── Kubernetes runners
│ └── new container-friendly pipelines
└── VM runners
└── legacy projects and host-specific dependencies
There is no value in forcing the VM runners into Kubernetes before the projects they serve are ready. They can disappear naturally as those dependencies are removed.
k3s migration is not the only answer: workloads to retire
The old wiki produced the simplest decision.
If a service is no longer used, Kubernetes does not make it more useful. The correct sequence is:
- take a final backup or snapshot;
- record how it could be recovered;
- power it off;
- leave it off for an agreed observation period;
- delete it after confirming that nobody needs it.
Deleting an obsolete VM saves more resources and maintenance than containerizing it ever would.
Seven practical k3s migration rules
For every remaining VM in the k3s migration backlog, I now want answers to the same seven questions.
1. Is it an application or a platform dependency?
Application workloads are usually better migration candidates. Identity, DNS, backup and bootstrap systems deserve a much higher bar.
2. Is the deployment officially supported?
An official container image or Helm chart is useful. A community image for a critical vendor appliance transfers maintenance responsibility to me.
3. Where does the state live?
Database, uploaded files, queues, encryption keys, scheduled jobs and generated data must all be identified. A container image is only the executable part of the service.
4. Does migration improve disaster recovery?
GitOps improves the recovery of configuration. It does not automatically back up databases or persistent volumes. A migration that makes the restore path less clear is not an improvement.
5. Does it create a dependency loop?
The systems needed to rebuild k3s should not all disappear when k3s disappears.
6. Is the VM actually still needed?
Retirement must remain an available answer.
7. Can the old system remain as a rollback?
The original VM should stay powered off but recoverable until the new workload, backup and restore process have been verified.
The k3s migration sequence
The resulting k3s migration order is:
- Clean up: back up and retire confirmed obsolete VMs.
- Fix the k3s failure domains: distribute worker capacity across both Proxmox hosts.
- Migrate low-risk application workloads: start with the test advertising and finance applications.
- Migrate SonarQube: only after its database, storage and resource requirements are documented.
- Reassess what remains: keep infrastructure and vendor systems on VMs unless the operational case changes.
Backup and restore testing runs alongside this sequence. No original VM is deleted merely because the new Pod starts successfully.
The expected result of the k3s migration
The realistic end state of this k3s migration is not “everything is Kubernetes.” It is a much smaller and more intentional VM estate beside a shared application platform.
k3s becomes the normal destination for internal applications, development services and supported application platforms. Proxmox VMs continue to host the infrastructure, recovery anchors and vendor products that are safer outside the cluster.
That boundary is simpler than either extreme. It avoids one VM per small application without turning Kubernetes into a dependency for every service in the environment.
The main lesson
The most useful outcome of the k3s migration inventory was not a count of VMs that could be removed. It was a reason for every workload to live where it lives.
“Keep it on a VM” is a valid architecture decision when it reduces dependencies, preserves vendor support or improves recovery.
The next post in this series will examine the cloud side of the platform: Azure Arc, a separate AKS cluster, and why stretching the existing on-premises k3s cluster across a WAN connection is not the hybrid architecture I want.