Hybrid Kubernetes sounded, at first, like a single cluster whose worker nodes could live partly on-premises and partly in Azure. Technically, an Azure VM can be made to join an existing k3s server over a VPN. Architecturally, however, that is usually the least attractive interpretation of hybrid.
This field note records the design decision I reached for an existing on-premises k3s environment: keep local and cloud failure domains separate, use Azure Arc when centralized management is useful, and add an independent AKS cluster only when a workload genuinely belongs in Azure. It is an architecture decision, not a claim that the Arc onboarding has already been completed.
Table of contents
What hybrid Kubernetes can mean
The phrase hybrid Kubernetes is overloaded. It may describe centralized management of clusters in different locations, consistent application delivery to those clusters, a Kubernetes platform running on Azure-integrated on-premises infrastructure, or—in the most literal interpretation—nodes from different sites joined to one cluster.
Those designs do not have the same reliability model. Before selecting a product, I separate three questions:
- Do I want Azure to manage and govern the cluster I already have?
- Do I need workloads to run in Azure as well as on-premises?
- Do those workloads truly need to share one Kubernetes control plane?
In my environment, the first two questions may eventually be yes. The third is no. That distinction makes the architecture much simpler.
Option 1: connect the existing k3s cluster with Azure Arc
Azure Arc-enabled Kubernetes attaches an existing Kubernetes cluster to Azure Resource Manager. Agents run inside the cluster and establish an outbound connection to Azure. The cluster remains where it is; its nodes, control plane and workloads do not move into Azure.
For an on-premises k3s cluster, Arc can provide a useful shared management layer:
- Inventory, resource grouping and tags in Azure.
- GitOps-based configuration using Flux or Argo CD.
- Azure Policy for governance.
- Azure Monitor and Defender integrations where appropriate.
- Azure RBAC and cluster-connect capabilities.
The key architectural point is that Arc does not merge the attached k3s nodes with AKS nodes. Each connected cluster remains its own Kubernetes and Azure resource. This is hybrid Kubernetes as centralized visibility and governance, not a stretched data plane.
The connection also has operational requirements. Microsoft’s Arc connectivity documentation explains that the agents use outbound connectivity for configuration and management features. A temporary cloud connectivity loss does not relocate or erase local workloads, but it does affect the Azure-side management capabilities. That dependency belongs in the design and monitoring plan.
When Azure Arc is a good fit
I would use Arc when I want Azure-native inventory, governance or GitOps for the existing cluster without rebuilding it. It is the least invasive of the three options because the local k3s control plane and workload placement remain unchanged.
What Azure Arc does not solve
Arc is not additional worker capacity, a multi-site service network or automatic disaster recovery. It will not make an on-premises application run in Azure merely because the cluster appears in the Azure portal. Workload portability still requires manifests, images, secrets, storage and data-recovery plans.
Option 2: stretch one k3s cluster into Azure
It is technically possible to create a Linux VM in Azure, connect it to the local network with a site-to-site VPN and join it as a k3s agent:
k3s agent --server https://on-prem-control-plane:6443 --token REDACTEDThe short command makes the design look easy. The operational consequences are not.
A remote worker must maintain reliable communication with the Kubernetes API server. Pod networking, service routing, image pulls, storage access, health reporting and application dependencies now cross a WAN or VPN boundary. If that link fails, the Azure node becomes unavailable to the on-premises control plane. Pods may remain running briefly while becoming unreachable, or Kubernetes may try to replace them somewhere that does not have the same data or capacity.
Latency and packet loss also become cluster concerns instead of merely application concerns. Troubleshooting must distinguish a container problem from CNI routing, firewall policy, VPN state, MTU mismatch and cloud routing.
The control plane is the hard boundary
I would not stretch k3s server nodes or the embedded etcd datastore across these sites. Consensus systems are sensitive to latency and partitions, and the WAN would become part of every control-plane decision. A local link problem could then threaten the availability of the entire cluster rather than only a remote workload.
A single control plane across failure domains may be justified in a specialized platform with carefully engineered networking, storage and support. It is not the default I would choose for a small production environment. The apparent simplicity of “one cluster” hides a larger shared failure domain.
Option 3: run separate k3s and AKS clusters
The more robust hybrid Kubernetes pattern is two independent clusters:
- On-premises k3s for internal services, lab workloads and applications that depend on the local network.
- Azure Kubernetes Service (AKS) for workloads that need Azure capacity, public-cloud integrations or an independent cloud failure domain.
Both clusters can consume the same Git repositories and deployment conventions. GitLab CI, Flux or Argo CD can reconcile the appropriate manifests into each target. Common Helm charts and Kustomize bases reduce duplication while overlays keep environment-specific settings explicit.
This model accepts that “same application platform” does not have to mean “same cluster.” Each site retains its own API server, scheduler, networking and upgrade lifecycle. A VPN outage may interrupt application-to-application traffic, but it does not turn a healthy worker node into NotReady merely because its control plane is on the other side of the link.
The data layer still needs a deliberate design
Two clusters do not automatically provide disaster recovery. A stateless deployment is easy to reproduce; a stateful application also needs database replication or restore procedures, object storage, secrets, persistent-volume strategy and tested recovery objectives.
For the workloads already running locally—such as internal tooling and services tied to local infrastructure—the sensible first step is usually to leave them there. An Azure cluster should be introduced for a concrete requirement, not simply to make the diagram look hybrid.
Where AKS on Azure Local fits
AKS enabled by Azure Arc on Azure Local is a separate architecture. It runs AKS-style Kubernetes clusters on Azure Local infrastructure in the customer’s environment and integrates provisioning and lifecycle management with Azure.
That does not mean AKS can be installed as a management layer on any existing Proxmox or small k3s setup. Azure Local is its own distributed infrastructure platform with defined deployment models and hardware ecosystem. It is relevant when an organization is intentionally adopting that platform, not as a lightweight add-on to an already working cluster.
For my current environment, replacing the infrastructure simply to gain the AKS operational model would be disproportionate. Connecting the existing k3s cluster with Arc—or keeping it independent—is the more realistic path.
Hybrid Kubernetes decision matrix
| Option | What it provides | Main risk or cost | My view |
|---|---|---|---|
| Azure Arc + existing k3s | Central inventory, policy, GitOps and Azure integrations | Outbound management dependency and added agents | Good first experiment |
| One stretched k3s cluster | A single API and scheduler across sites | WAN/VPN becomes a cluster dependency and failure domain | Avoid for this environment |
| Separate k3s + AKS | Independent failure domains with consistent delivery | Two lifecycles and explicit cross-cluster data design | Best when cloud workloads are needed |
| AKS on Azure Local | Azure-managed Kubernetes model on Azure Local | Requires adoption of the Azure Local platform | Different infrastructure decision |
The hybrid Kubernetes design I would use
My target design is intentionally boring:
- Keep the existing on-premises k3s cluster as an independent cluster.
- Optionally connect it to Azure Arc for inventory, governance and GitOps evaluation.
- Keep GitLab as the source of truth for application and infrastructure manifests.
- Add a separate AKS cluster only when a production or cloud-native workload justifies it.
- Deploy to both clusters through the same delivery process, with explicit environment overlays.
- Treat databases, persistent storage and secrets as separate recovery workstreams.
This creates a shared operating model without a shared control-plane failure domain. The local cluster can continue serving local workloads if Azure management connectivity is interrupted. The Azure cluster can continue scheduling cloud workloads if the site-to-site VPN is unavailable, provided those workloads do not depend synchronously on local services.
Management plane versus workload plane
This distinction is the core of the decision. Azure Arc may connect the management plane: inventory, policy, deployment and observability. Separate clusters keep the workload planes independent: scheduling, node health, service networking and upgrades.
That is the version of hybrid Kubernetes I trust more. Central management is useful; centralizing every failure is not.
A low-risk adoption path
I would introduce this architecture in small, reversible steps:
- Document the current k3s cluster, namespaces, storage classes, ingress endpoints and external dependencies.
- Connect a non-critical or lab cluster to Azure Arc first.
- Measure agent resource use and confirm outbound firewall requirements.
- Test inventory and read-only visibility before enabling policy enforcement.
- Evaluate GitOps with one non-critical namespace and a clear rollback path.
- Define which future workload would justify a separate AKS cluster.
- Test backup and restore independently from cluster registration.
This sequence provides evidence before commitment. If Arc adds useful governance, it can stay. If it adds complexity without solving a real problem, the k3s cluster remains a valid independent platform.
Frequently asked questions
Can an Azure VM join my on-premises k3s cluster?
Yes, with suitable network connectivity it is technically possible. The harder question is whether the VPN latency, routing and failure behavior are acceptable for the Kubernetes node and the workloads assigned to it. For this environment, I would use a separate cluster instead.
Does Azure Arc move my workloads into Azure?
No. Arc connects management capabilities to the cluster. The workloads continue to run on their existing nodes unless I deliberately deploy them elsewhere.
Is Azure Arc required to run both k3s and AKS?
No. Independent clusters can be managed with GitLab, Flux, Argo CD and normal Kubernetes tooling. Arc is optional when Azure inventory, policy, security or access features provide value.
Is AKS on Azure Local the same as AKS in Azure?
No. Both use Azure-connected lifecycle tooling, but one runs on Azure Local infrastructure in the customer’s environment while standard AKS runs in Azure. It is also not the same as attaching an arbitrary existing k3s cluster to Arc.
What is the biggest risk in a stretched hybrid Kubernetes cluster?
The WAN becomes part of the cluster’s health. A network partition can affect node status, scheduling, service reachability and recovery behavior at the same time. Separate clusters contain that failure more cleanly.
Hybrid Kubernetes does not require one enormous cluster. In this case, the better design is a common source of truth and management approach across two independent platforms: k3s on-premises, AKS when Azure capacity is needed, and Azure Arc only where its governance features earn their operational cost.