Standing Up My Production Environment, Part One: Bootstrapping
A Flux Hero Story

Search for a command to run...
A Flux Hero Story

No comments yet. Be the first to comment.
I've been using containers for a while now without fully understanding what's going on under the hood. This week I went back to the fundamentals; what problem do containers solve, what do namespaces a

A GitOps Build in Three Stages w/FluxCD

The GitOps wars, why Flux won👀️

Week 1 This week marked the beginning of my weekly learning-in-public journey in Cloud and DevOps. It’s amazing how much you can build just by showing up every day and doing one thing at a time. What’s up y’all I’m Angel. This is IT with ANC and welc...

Staging has been stable for a while now. Two apps, one monitoring stack, and Renovate all running as services. Everything reachable internally through ingress, with the apps that need it exposed publicly through my own domain. That stability is what made this the right week to start production, the cluster my friends and family will actually use.
I leaned heavily on documentation I already wrote while building staging, along with the official Flux docs, rather than figuring any of this out from scratch a second time.
This part barely needed thought. My own bootstrapping docs covered it end to end, and the only things that changed from staging were the VM name and the IP. SSH access was set up the same way it always is.
One thing I am deliberately leaving for later: hardening SSH properly. Noting it here so I do not forget.
The install itself was routine. The part that actually took some thought was managing two separate kubeconfigs, one for staging and one for production, WITHOUT one silently overwriting the other.
k3s names its context "default" out of the box, which is a problem the moment you have two clusters. Both configs needed distinct names before merging, and I found that out the hard way the first time I tried it.
The merge itself is simple once the naming is sorted:
export KUBECONFIG=~/.kube/prod-config:~/.kube/staging-config
kubectl config view --flatten > ~/.kube/merged-config.yaml
mv ~/.kube/merged-config.yaml ~/.kube/config
unset KUBECONFIG
# verify
kubectl config get-contexts
or
kubectx
While I was in there, I also dealt with a disk space problem that has been quietly building on the staging VM. Renovate chews through disk space in /var/lib/rancher/k3s/containerd/ over time, and manually pruning old images felt like something I would forget to keep doing. Instead I tuned k3s's built-in image garbage collection directly in the k3s config file:
kubelet-arg:
- "image-gc-high-threshold=60"
- "image-gc-low-threshold=40"
Once disk usage hits 60 percent, k3s starts evicting old images until it drops back down to 40 percent. Restart k3s to take effect.
This was; in my own words while writing the notes, stupid simple. I love Flux for exactly this reason.💎️
Same bootstrap command as staging, same GitHub PAT, same user, same repo environment variables. The only thing that changed was the path:
flux bootstrap github \
--owner=$GITHUB_USER \
--repository=$GITHUB_REPO \
--branch=main \
--path=./clusters/production \
--personal
No need to reinstall Flux unless you are working from a new machine. The one thing worth double checking before running this is that your current context actually points at the new production cluster. I use kubectx to switch quickly between contexts, which makes this a five second check instead of a mistake waiting to happen.
Environment is bootstrapped. Production has k3s, Flux, and a merged kubeconfig that will not fight with staging. Deploying the actual applications is Step 2, and that is tomorrow's work. Part Two once it is done.