Cozystack v1.6.0: Talos Linux Workers, Tenant Firewalls, and the End of Budget Escapes

| |

7 min read

Cozystack logo

Imagine running a hotel where every guest gets their own fully private apartment — complete with a kitchen, living room, and a lock on the door — but you never had to build individual buildings. That is Cozystack in a nutshell. It takes a single Kubernetes cluster and turns it into a self-service private cloud where tenants can spin up their own Kubernetes clusters, virtual machines, databases, and storage — all isolated, all on demand, all managed through a clean API.

The problem it solves is one anyone who has tried to share a Kubernetes cluster across teams knows intimately: multi-tenancy is hard. RBAC only goes so far. Namespace isolation leaks. One team’s noisy neighbor brings down everyone’s workloads. Cozystack handles all of that for you — tenant isolation, resource quotas, network policy, identity, certificates — so platform engineers can stop being babysitters and start being, well, engineers.

Cozystack v1.6.0 is the project’s biggest release since launching, and it fundamentally changes the foundation: tenant Kubernetes workers now boot Talos Linux instead of Ubuntu. That alone would be a headline, but the release also adds OIDC single sign-on, a tenant-managed firewall API, hierarchical resource quotas that actually enforce budget limits, and patches two KVM guest-to-host escape vulnerabilities. If you are running tenant VMs on Kubernetes, you want to read this.

Think of it this way: Cozystack just swapped out the foundation of every tenant’s house while they were still living in it — and nobody had to move out.

What Is New in Cozystack v1.6.0

Talos Linux Replaces Ubuntu for Tenant Workers

The biggest change in this release is a full gut renovation of how tenant Kubernetes worker nodes boot. Previously, Cozystack provisioned workers running Ubuntu and bootstrapped them through kubeadm — the standard Kubernetes bootstrap tool. In v1.6.0, that entire path is replaced with Talos Linux, the immutable, API-driven operating system purpose-built for Kubernetes.

Why does this matter? Talos has no SSH, no shell, no package manager. You manage it entirely through a declarative API. That means smaller attack surface, simpler compliance stories, and worker nodes that behave identically every single time. The migration is driven by Cluster API’s Talos bootstrap provider (CABPT) with a CSR-signing sidecar embedded in the Kamaji control-plane pod.

Existing tenant clusters roll over automatically — old Ubuntu workers are replaced by Talos workers on the first reconcile after upgrade, with no manual intervention required. Worker disks are reprovisioned and container images re-pulled, so plan for a full worker-pool replacement per tenant cluster.

# Existing workers roll over automatically after upgrade
# Plan for: full worker-pool replacement per tenant cluster
# nodeGroups[*].diskSize now sizes the single Talos system disk

The separate kubelet disk is gone — Talos handles its own disk layout on a single system disk sized by diskSize. Kubernetes v1.30 is also dropped from the support matrix, since it falls outside the Talos compatibility window (PR #2931).

etcd-Operator v1alpha2: Zero-Downtime In-Place Migration

etcd is the brain of every Kubernetes cluster, and Cozystack just performed brain surgery on its own etcd management layer — without taking anything offline.

The legacy etcd.aenix.io/v1alpha1 StatefulSet-based model is replaced by the new etcd-operator.cozystack.io/v1alpha2 Membership API, served by a Cozystack-authored chart at appVersion v0.5.2. Platform migration 50 runs as a pre-upgrade hook that rewrites ownership, labels, and custom resources so the new operator takes over the live data plane on its first reconcile.

No data move. No pod restart. No downtime.

Before touching anything, the migration takes a mandatory snapshot of every adopted cluster to the platform-managed backup bucket. If it cannot resolve the snapshot target, it fails loudly rather than proceed unsafely. It also re-issues each cluster’s TLS certificates with the new operator’s native member wildcard SAN first, because an adopted cluster with stale certs would silently fail TLS on the next member replacement while still reporting healthy (PR #2859, PR #3270).

OIDC Single Sign-On for Tenant Clusters and Grafana

Two symmetric features let a tenant opt an individual workload into the platform’s Keycloak identity realm through one flat selector — no platform-level configuration required.

For tenant Kubernetes clusters, each Kubernetes CR gains spec.oidc.mode: System | CustomConfig | None. In System mode, the tenant cluster trusts the platform’s Keycloak realm automatically, and a ready-to-use kubeconfig with OIDC exec configuration is surfaced through the dashboard. A tenant user goes from “cluster exists” to “kubectl works with my SSO identity” without an operator in the loop.

The same pattern applies to per-instance Grafana dashboards: spec.oidc.mode wires a per-instance Keycloak client, and spec.oidc.users lets you reconcile users into Grafana roles (Admin, Editor, Viewer) automatically (PR #3044, PR #3176).

SecurityGroup: A Firewall API Tenants Can Actually Use

A new tenant-facing, namespace-scoped firewall resource — sdn.cozystack.io/v1alpha1 SecurityGroup — lets tenants manage their applications’ network policy without needing access to the underlying Cilium API.

A SecurityGroup is a membership group: it owns a label that gets stamped onto pods of every managed application listed in spec.attachments. The group projects 1:1 onto a CiliumNetworkPolicy in the same namespace, so one SecurityGroup can cover several applications at once. Ingress and egress rules reference other groups by name, and the resolution happens live in the Cilium dataplane.

Before this, tenants had no way to express network policy without platform-level access. Now they can define firewalls as a first-class managed resource, and the platform handles the translation to Cilium under the hood (PR #2922).

Hierarchical Resource Quotas: No More Budget Escapes

Previously, a tenant’s declared resource quota became a plain per-namespace ResourceQuota that ignored the tenant tree entirely. A super-admin inside a quota’d tenant could create a sub-tenant with a larger quota, or no quota at all, and consume more than was ever allocated. That is… not great for a multi-tenancy platform.

In v1.6.0, a tenant’s declared quota is now the budget for its entire sub-tree. A child that declares a quota carves a fixed slice out of the parent’s remaining budget. A child without one shares the parent’s pool instead of being unbounded. And a child quota exceeding the parent’s remaining budget is rejected at admission.

Enforcement is two cooperating layers: a declaration-time gate that deterministically rejects over-budget children, and a runtime enforcer that maintains per-namespace allocated quotas so members collectively stay within the parent’s budget. A parent quota lowered below the sum of its children emits a QuotaOvercommitted event, and a temporary buffer percentage can be set during rollout so existing workloads are not evicted (PR #3086).

Wildcards, Storage Reclamation, and Security Fixes

  • Wildcard certificates, end to end: Operator-provided wildcard TLS certificates are now automatically replicated into every tenant namespace that terminates TLS. No extra operator input needed. A new opt-in publishing.certificates.wildcard mode also issues a single *.root-host certificate via DNS-01, solving Let’s Encrypt rate limit issues at scale (PR #2988, PR #2990).
  • Storage reclamation on deletion: A ten-PR sweep across the catalog means that deleting a managed application now actually releases its storage instead of leaking PVCs. ClickHouse, Qdrant, OpenBAO, Harbor, MariaDB, monitoring, SeaweedFS, etcd, Bucket, and Gateway all gained post-delete cleanup hooks. This makes deletion destructive — which is the entire point (PR #3072 and nine more).
  • Keycloak KMS encryption: An optional KMS-encrypting database proxy provides column-level encryption for PII at rest, backed by either a static KEK or Vault Transit. Off by default. Keycloak also gets a separate admin hostname, S3 backups, and configurable login themes (PR #3003, PR #3239).
  • Immutable tags and rc-to-stable promotion: A stable release is now a byte-identical promotion of its release candidate. No tag is ever force-moved, and stable is never rebuilt. The cron-driven patch-release workflow is gone — stable tags exist only through explicit promotion (PR #3017).

Security: Two KVM Guest-to-Host Escapes Patched

This release ships Talos v1.13.6, which carries Linux kernel 6.18.38 with fixes for CVE-2026-53359 and CVE-2026-46113 — two KVM guest-to-host escape vulnerabilities. These are the kind of bugs that matter enormously in a platform like Cozystack, where tenant virtual machines run on shared KVM hosts. A compromised tenant VM could potentially escape to the host kernel.

The maintainers took this seriously enough to ship an emergency mitigation in talm v0.33.0 that temporarily disabled nested virtualization as a stopgap before the kernel fix landed. That mitigation is now reverted in v0.33.1 since the kernel fix is available. Both the management cluster and tenant worker VMs receive the patched Talos image (PR #3240, PR #3269).

The release also closes CVE-2025-68121 and CVE-2026-24051 in OpenBao, CVE-2025-52881 and CVE-2025-58181 in csi-driver-nfs, and CVE-2026-35469 in KubeVirt’s spdystream dependency.

Closing

Cozystack v1.6.0 is the release where a promising sandbox project starts looking like a production-grade private cloud platform — Talos under the hood, real multi-tenancy enforcement, identity wired end to end, and the kind of security responsiveness that makes you trust the team building it.

Learn More