K3s

Why K3s?

I run a 3-node Proxmox cluster. While I initially used Docker compose for app deployment across VMs, this process required manual node selection and lacked automated failover capabilities.

Though Docker Swarm was an option, I chose k3s - a lightweight k8s distribution - for its maturity, active community support, and perfect fit for my homelab needs.

K3s Install

# install k3s server
# I disable servicelb because I use metallb
curl -sfL https://get.k3s.io | K3S_TOKEN=123 sh -s \
  - server --cluster-init \
  --disable servicelb \
  --kube-proxy-arg proxy-mode=ipvs \
  --kube-proxy-arg ipvs-strict-arp

# install k3s agent
curl -sfL https://get.k3s.io | K3S_URL=SERVER_IP K3S_TOKEN=123 sh -s agent

Argo CD

Argo CD enables GitOps in my k3s cluster - using my git repo as the source of truth where each commit triggers a deployment.

Argo CD Install

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.14.0-rc3/manifests/install.yaml

Example

Here's how I deploy Cloudflare Tunnel with Argo CD:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cf-tunnel
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/lttviet/settings.git
    targetRevision: HEAD
    path: k3s/apps/cf-tunnel/overlays/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: cf-tunnel
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Argo CD checks my repo every 3 minutes and automatically applies any changes in the specified directory. The web UI provides clear visibility into deployment status:

Advanced Configs

Insecure ArgoCD Website

Since Traefik handles TLS termination, I can disable TLS in Argo CD website.

patches:
  - target:
      kind: ConfigMap
      name: argocd-cmd-params-cm
    patch: |-
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: argocd-cmd-params-cm
      data:
        server.insecure: "true"

Helm Support in Kustomization

Most apps give me plain manifests, but for some that only have helm charts, this allows me to install helm charts inside kustomization.yaml.

patches:
  - target:
      kind: ConfigMap
      name: argocd-cm
    patch: |-
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: argocd-cm
      data:
        kustomize.buildOptions: "--enable-helm"

App of Apps Health Check

When I deploy app of apps, I want the parent app to be healthy only when all its children apps are healthy. This feature was removed in a previous Argo CD version, but can be patched in.

patches:
  - target:
      kind: ConfigMap
      name: argocd-cm
    patch: |-
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: argocd-cm
      data:
        resource.customizations.health.argoproj.io_Application: |
          hs = {}
          hs.status = "Progressing"
          hs.message = ""
          if obj.status ~= nil then
            if obj.status.health ~= nil then
              hs.status = obj.status.health.status
              if obj.status.health.message ~= nil then
                hs.message = obj.status.health.message
              end
            end
          end
          return hs

Repo structure

Root level

k3s/
├── apps/          # User-facing applications
├── argocd/        # Argo CD manifests
├── bootstrap/     # Initial cluster setup
├── cluster/       # Cluster-wide applications
└── patches/       # Custom patches for k3s

Inside /k3s/apps

k3s/apps/cf-tunnel
├── base
│   ├── deployment.yaml
│   ├── external-secret.yaml
│   └── kustomization.yaml
└── overlays
    └── dev
        └── kustomization.yaml

Each app follows the kustomize overlay pattern. This is overkill for homelab, but it's a good practice for production.

In production, ideally there are different environments for each app. I can have /overlays/dev, /overlays/staging, /overlays/prod, etc.

Bootstrap Process

The root app monitors the bootstrap directory which bootstraps the cluster through three main applications:

1. Self-managed Argo CD.
2. Cluster infrastructure components
3. User-facing applications

Cluster app and user-facing apps have multiple child apps under them.