Info
Multiple heads are better than one, in theory.
Hardware
A cluster of 2 nodes:
- Ryzen 5560U, 16GB, mini pc
- Ryzen 5560U, 32GB, mini pc
It’s an even number, so the Proxmox cluster will lose quorum if just 1 node goes down. I set 1 node to have 2 votes to avoid this situation for the time being.
WLAN isn’t optimal for Proxmox, so my nodes are connected to the router via Ethernet.
Proxmox
After install, I run this script to correct PVE sources and disable subscription nag.
Cluster
To remove a cluster, run these commands in PVE host:
systemctl stop pve-cluster corosync
pmxcfs -l
rm -rf /etc/corosync/*
rm /etc/pve/corosync.conf
killall pmxcfs
systemctl start pve-clusterUbuntu LXC
Setup to create a new Ubuntu LXC: Ubuntu.
TODO: look into terraform/opentofu to automate the process of creating LXC.
Secrets
I use sops and age to encrypt files before checking them into git repos.
I have a VM dedicated for Ansible and Packer. It can decrypt the secret files before building and deploying.
I’ll look into HashiCorp Vault in the future.
Samba
I have 1 physical machine running a Samba server to share files between VMs and LXC containers. It’s a single point of failure, so I plan to move it out to a dedicated NAS machine in the future.
Packer template: github.
Current setup
I’m only using LXC Ubuntu containers. One container for running Docker compose and the others are apps which I found easier to install directly. All my docker compose files are here.
AdGuard Home
AdGuard Home is set up to block ads as well as a DHCP server. I’m using Hagezi list.
I have 2 instances running in each node. The configs are synced using AdGuard Home Sync.
Cloudflared Tunnel
Tunnels are used to expose some of my internal apps to public, so I don’t need to vpn to access them. They are protected behind Cloudflare Access.
Logs
Fluentd logging driver sends logs over the network to Fluentd collector, which then sends them to Elasticsearch.
Kibana is used to visualize logs.
I limit Elasticsearch memory to 1GB, and it is still working well with my current set up.
sudo hostnamectl set-hostname composer2