I'm trying to bridge a gap in my Python knowledge as I pursue automations and a DevOps job. Refreshing the fundamentals now will solidify what I know, so I'm working through a course that starts with the basics and builds up to APIs and AI agents. The course instructor uses VS Code, I'm sticking with vim. This is also directly useful prep for an upcoming hackathon; RowdyHacks.

Study Sesh One - Fundamentals, Conditionals, and Venv

Breezed through the refresher on variables, data types, and data structures, nothing new, just knocking the dust off.

The example that actually made it click for DevOps work was a simple conditional exercise:

"As a devops engineer you need to check if CPU % is greater than 50%; if CPU is greater than 20% but less than 50%, alert the team."

Code for this lives in cpu_check.py and real_cpu_check in the repo.

Also spent time on virtual environments, since managing packages and dependencies across different Python projects gets messy fast. Instead of installing different versions and conflicting binaries globally, you spin up a venv and install dependencies inside it, keeping everything tidy and decoupled:

python -m venv .venv
source .venv/bin/activate

Using pip for now. Once I move to uv, I won't even need these commands since uv handles venvs natively.

Study Sesh Two - Functions, Loops, and an Actual CPU Check

This session was functions and loops, then putting them to use on a real exercise:

As a DevOps Engineer, check the CPU usage for 5 seconds, take a threshold from the user, and check if CPU is healthy.

This is also where the psutil library came in, which is useful for interacting with local process and system info. I found myself reading a lot of documentation which is a skill in, and of itself.

Study Sesh Three - Reviewing My Own Code

Went back and reviewed how I'd answered the previous session's exercise. My solution and the instructor's were different but landed on the same result, and honestly, mine was a little better. Getting noticeably more comfortable with the logic, syntax, and flow control at this point, and it works. All of it is sitting in commit history in my public lab repo.

Study Sesh Four - Making Code Reusable, and What an API Actually Is

Took the code from the last session and made it reusable, then moved into APIs: what they are, why they exist, when to use them, and how to build one in Python.

The instructor uses the request library for GET, PUT, DELETE, and so on. The metaphor that made APIs finally click for me was, thinking of an API as the middleman between a client and a server, like at a restaurant. You don't walk into the kitchen and order directly from the chef/back-of-house. You look at the menu, the waiter takes your order to the chef, and the waiter brings the food back out to you. The API in that scenario is the waiter, back-of-house/chef would be your "Backend", and the menu or front-of-house would be your "Frontend"

This is also where FastAPI is introduced in the course; a framework for building APIs from scratch. I kinda went of the deep end and started having a lot of fun with FastAPI. Their docs were very useful and I was up and running in no time..

Install:

pip install fastapi "fastapi[standard]"

FastAPI itself is a class (from fastapi import FastAPI), and the appeal is how "Fast" it is to something useful e.g. you define your parameters using standard Python types, and FastAPI takes care of the rest. Including generating two versions of automatic docs for your API without being asked. Feels a little spoiled, if im being honest but I'm not going to complain.

Study Sesh Five - Building the Hello World and Metrics Endpoints

Time to combine the FastAPI basics with the system-check code from session three into a real endpoint, /metrics, that reports actual system info by calling a check_system_info function.

Sounded simple. Spent 20 minutes troubleshooting and couldn't get it working, so I watched how the instructor approached it instead. My attempt:

from fastapi import FastAPI
from system_utils import check_system_info

app = FastAPI(title="Devops Utils API")

@app.get("/hello")
def hello():
    return {"message": "Hello Angel, Welcome To Devops Utils API"}

@app.get("/metrics")
def metrics():
    system_metrics = check_system_info()
    return {"system metrics": system_metrics}

Turns out I was overthinking it. The fix was just calling the function directly and letting FastAPI handle the rest:

from fastapi import FastAPI
from system_utils import check_system_info

app = FastAPI(title="Devops Utils API")

@app.get("/hello")
def hello():
    """
    This API is Hello World, my first api from scratch using FastAPI.
    """
    return {"message": "Hello Angel, Welcome To Devops Utils API"}

@app.get("/metrics")
def metrics():
    """
    This API gets the system metrics of my laptop i.e. cpu, mem, disk
    """
    return check_system_info()

Where I'm At

I feel a lot more confident on how to build my own APIs from scratch with FastAPI, understand how they work, and know what purpose they serve. Both the hello-world endpoint and the metrics endpoint locked it in for me.

All the exercises from these sessions are in my GitHub lab repo, for anyone who wants to follow along or check my work.

Whats Next

Building my own API to interact with the Docker daemon, from scratch, no course this time. That one's going to be its own write-up.

More to come; until next time see ya on the flippity flip.

A few weeks back I had to redeploy my GitOps k3s homelab because of a network migration; new router, new subnet, and (long story, see the last post) no clean way to avoid it. Rather than fight k3s over its IP-bound TLS certs I just reprovisioned the VMs by hand, pointed Flux at them, and let it reapply everything from the repo. It worked pretty well for all intents and purposes.

But it also left me wanting more. To quote Kelsey Hightower I had "left room for greatness," and after that redeploy, the "room" became obvious. Every part of that recovery except the VM provisioning itself was already automated. Flux owned the cluster state. Git owned the source of truth. The only manual part left was me clicking through the Proxmox UI to spin up VMs.

You can see where this is headed. I set out to automate the provisioning of the VMs and k3s, and the hand-off to Flux, using Packer, Terraform, and cloud-init. In theory; this gets my entire GitOps homelab on any hardware running Proxmox. We'll see how theory holds up.

The Mental Model

I work from Larry, my small headless Debian ssh server running on a re-purposed windows surface laptop with Terraform, Packer, kubectl, kubectx/kubens, Flux, my age key, and a GITHUB_TOKEN in the environment. Nothing sensitive ever touches the VMs themselves:

jumpbox (10.10.10.x, Debian 13 headless)
 ├─ terraform, packer, kubectl, kubectx, flux, age key, GITHUB_TOKEN env
 │
 ├─ [1] packer build  ──SSH──> temp build-VM on Proxmox ──> becomes template
 │
 ├─ [2] terraform apply ──API──> Proxmox clones template 
 │        cloud-init on each new VM: hostname, ssh key, swapoff, install k3s ONLY
 │
 └─ [3] terraform's local-exec (still running ON the jumpbox, as me):
          scp kubeconfig off the new node → point it at node's real IP
          flux bootstrap github  (uses my jumpbox's GITHUB_TOKEN, never the VM's)
          kubectl create secret sops-age  (uses my jumpbox's age key, never the VM's)

VMs get just enough to run k3s. Anything that needs a real credential (the GitHub token, the age key for SOPS) runs from the jumpbox via local-exec, never gets baked into an image or handed to a VM(however the k8s secret does live on the cluster - encrypted of course).

How It Fits Together

The repo is split the same way the workflow is:

homelab-automation/
├── packer/
│   ├── debian13-k3s.pkr.hcl      # builds the template
│   ├── http/preseed.cfg          # unattended Debian installer answers
│   └── scripts/provision.sh      # cloud-init/qemu-guest-agent + cleanup
└── terraform/
    ├── versions.tf
    ├── variables.tf
    ├── main.tf                   # clones the template per cluster
    ├── cloud-init.tf             # uploads rendered cloud-init as a Proxmox snippet
    ├── outputs.tf
    ├── terraform.tfvars.example
    └── templates/
        └── user-data.yaml.tftpl  # k3s install + flux bootstrap, per node

1. Packer boots the Debian 13 netinst ISO on Proxmox, feeds it http/preseed.cfg for a fully unattended install (no swap partition, OpenSSH + sudo + curl from the start), runs scripts/provision.sh to install qemu-guest-agent and cloud-init, then generalizes the image (clearing machine-id, SSH host keys, and the cloud-init cache) before converting it into a Proxmox template. This means the template can not be used to manually provision, I created a seperate template for that.

2. Terraform clones that template once per entry in var.clusters (staging and production by default), applying the VM hardware settings from my notes, and attaches a per-node cloud-init config rendered from user-data.yaml.tftpl.

3. cloud-init on first boot of each clone, sets hostname/user/SSH keys, disables swap, installs k3s (INSTALL_K3S_EXEC=--disable=helm-controller, matching my notes). From that point, Flux owns the cluster.

One-time setup, from the jumpbox:

# Packer
cd packer
packer init .
packer build -var-file=variables.pkrvars.hcl debian13-k3s.pkr.hcl

# Terraform
cd ../terraform
cp terraform.tfvars.example terraform.tfvars   # fill in your values
export TF_VAR_proxmox_api_token="root@pam!terraform=xxxxxxxx-xxxx-..."
export TF_VAR_github_token="ghp_xxx"           # never put this in tfvars/git
terraform init
terraform plan
terraform apply

Oldies But Still Goodies; Why Not Ansible

A lot of fellow homelabbers redeploy and manage infra/config with Ansible and custom shell scripts. Nothing wrong with that, but it's not GitOps and it's exactly what I've been trying to move away from. Between Flux and Kubernetes-native tooling, I don't see a need for it in the lab itself.

That said, I thought I'd keep Ansible around for "Day 2" ops. Adding a user to sudo, installing docker-compose, one-off custom builds on a node that already exists. That's a reasonable division of labor, and it's not a bad way to stay grounded in the "traditional" DevOps fundamentals while still leaning GitOps-first for anything that touches cluster state.

Packer: The Golden Image

From the packer/ directory:

packer init .

packer validate -var-file=variables.pkrvars.hcl debian13-k3s.pkr.hcl

packer build -on-error=ask -var-file=variables.pkrvars.hcl debian13-k3s.pkr.hcl

The commands above build a template meant for Terraform only, because the image is generalized and locked down, and it can't be cloned and booted for manual testing as previously mentioned.

To get a version I can poke at by hand, I had to:

• keep the default user credentials in a separate provisioning script

• override the template name and provisioning script via extra Packer variables:

packer build -on-error=ask -var-file=variables.pkrvars.hcl \
  -var 'vm_id=9001' \
  -var 'vm_name=debian13-manual-template' \
  -var 'provision_script=scripts/provision-manual.sh' \
  debian13-k3s.pkr.hcl

Lessons From my Zettel Notes

A few things that came up while getting this far:

• packer validate threw a handful of deprecation warnings and a missing-checksum warning. It still validated, but ignoring those would've just meant hitting the same bugs later so it was worth fixing up front rather than deferring.

• provision.sh had a couple of package names wrong for Debian 13 specifically, which needed a quick correction.

• The Terraform-only template is successful, but by design can't be manually cloned, which is exactly what the separate manual-template build args above are for.

  ---

Wrap Up

Packer's side of this is where I want it. A clean, generalized, golden Debian 13 image that Terraform can clone on demand. Next up is finishing the Terraform side of things, confirming the cloud-init → k3s → Flux bootstrap chain works end to end on a fresh clone, and, if it does, getting to the point where standing up this entire GitOps homelab really is one command away. Checkout my lab repo where I do my testing/experiments: Github

Not to be confused w/my actual homelab gitops repo: Homelab

More to come.

No biggie tho, the network gets built around that limitation instead of fighting against it.

Also I finally got a router so I can actually manage my network now and not be locked down to OEM-harware. The router I got was a Linksys-MXS4300.

[ T-Mobile Gateway ]  <-- Performs cellular NAT, ignores my home devices

         │  Ethernet cable from T-Mobile LAN to Linksys-Router WAN
         ▼

[ Linksys OpenWrt Router ]  <-- The ONLY network my house and lab talk to
   ├── TVs, Phones, Laptops (Wi-Fi)
   └── Proxmox Server ── Multiple k3s Clusters (Ethernet)

The T-Mobile Gateway still does its own NAT in the background, but nothing on my network actually talks to it directly. Everything routes through the Linksys running OpenWrt instead. I also disabled the tri-band radios on the T-Mobile gateway so my devices only see the Linksys Wifi

Step One: Flash the Linksys Router

Standard OpenWrt install; navigated to the router's default address, uploaded the OpenWrt factory firmware built for my specific model, and followed the official OpenWrt wiki guide for the device the whole way through.

Step Two: Flash the Backup Partition Too

This router dual-boots between two firmware partitions, which means a power outage mid-update could drop it back onto stock Linksys firmware. To avoid that I flashed OpenWrt onto the second partition as well, using the Sysupgrade image and not the Factory image. Now there is no corporate firmware left on the device to fall back to, no matter what happens.

Step Three: Set Up the Home Network

At this point in time there were 3 subnets in play; my old AT&T LAN on 192.168.1.1/24, the T-Mobile Gateway on 192.168.12.1/24, and a new 10.10.10.1/24 network for the Linksys router, configured under Network > Interfaces.

The T-Mobile Gateway connects into the Internet port on the back of the Linksys, since it cannot be bridged. During setup I kept my laptop plugged into a different LAN port to make sure I was configuring the router itself and not accidentally sitting on the wrong side of the network.

I then disabled the T-Mobile Gateway's tri-band radios through its companion app, so they would not interfere with the new Linksys radios.

To keep every TV and Wi-Fi device from needing to be reconnected by hand, I set the Linksys SSID and password to match the old network exactly, under the wireless interface settings.

Router and gateway both went to their permanent spot after that, and the new network was live. I could now finally unplug my AT&T router.✌️

Step Four: Migrate Proxmox and the Homelab

First, a new static IP for the Proxmox node:

vim /etc/network/interfaces # Add IP/Gateway Here
systemctl restart networking

Example:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface using DHCP
auto eth0
iface eth0 inet dhcp

# Static IP Address Example
This configuration assigns a fixed IP address, subnet mask, gateway, and DNS server to an interface.

# The primary network interface using a static IP
auto eth0
iface eth0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.1
    dns-nameservers 8.8.8.8 8.8.4.4

And a matching static reservation on the OpenWrt side, in the web-gui under Network > DHCP > Add, then Save and Apply.

The VMs and k3s clusters were big concern for me, but FluxCD had my back. Destroy VMs, redeploy VMs, point Flux at them, and the homelab comes back on its own.💎️

The reason a redeploy was necessary at all was because k3s ties a node's IP address to its local TLS certificates at cluster creation. Change the IP and that binding breaks.

I could have gone in and hand patched certificates to work around it, but that is exactly the kind of old-school manual fix GitOps is supposed to make unnecessary. Destroy the cluster, recreate it, let Flux reapply everything from the repo. Done.

There is one step above even this, worth mentioning for later. Provisioning the VMs themselves with Terraform or OpenTofu, cloud-init, and Packer, instead of doing it by hand. That is the next layer to build toward.

Wrap Up

Now that I can manage my own network the sky is the limit for homelab, and very happy w/my documentation. Every project has had a full write up/post-mortem and I now have a back log of things to write about/publish.

Up next: Data Backups, Dev-Containers, and Finally deploying Production.

I leaned heavily on documentation I already wrote while building staging, along with the official Flux docs, rather than figuring any of this out from scratch a second time.

Provisioning the VM

This part barely needed thought. My own bootstrapping docs covered it end to end, and the only things that changed from staging were the VM name and the IP. SSH access was set up the same way it always is.

One thing I am deliberately leaving for later: hardening SSH properly. Noting it here so I do not forget.

Installing k3s

The install itself was routine. The part that actually took some thought was managing two separate kubeconfigs, one for staging and one for production, WITHOUT one silently overwriting the other.

k3s names its context "default" out of the box, which is a problem the moment you have two clusters. Both configs needed distinct names before merging, and I found that out the hard way the first time I tried it.

The merge itself is simple once the naming is sorted:

export KUBECONFIG=~/.kube/prod-config:~/.kube/staging-config

kubectl config view --flatten > ~/.kube/merged-config.yaml

mv ~/.kube/merged-config.yaml ~/.kube/config

unset KUBECONFIG

# verify
kubectl config get-contexts
or
kubectx

While I was in there, I also dealt with a disk space problem that has been quietly building on the staging VM. Renovate chews through disk space in /var/lib/rancher/k3s/containerd/ over time, and manually pruning old images felt like something I would forget to keep doing. Instead I tuned k3s's built-in image garbage collection directly in the k3s config file:

kubelet-arg:
  - "image-gc-high-threshold=60"
  - "image-gc-low-threshold=40"

Once disk usage hits 60 percent, k3s starts evicting old images until it drops back down to 40 percent. Restart k3s to take effect.

Installing FluxCD

This was; in my own words while writing the notes, stupid simple. I love Flux for exactly this reason.💎️

Same bootstrap command as staging, same GitHub PAT, same user, same repo environment variables. The only thing that changed was the path:

flux bootstrap github \
  --owner=$GITHUB_USER \
  --repository=$GITHUB_REPO \
  --branch=main \
  --path=./clusters/production \
  --personal

No need to reinstall Flux unless you are working from a new machine. The one thing worth double checking before running this is that your current context actually points at the new production cluster. I use kubectx to switch quickly between contexts, which makes this a five second check instead of a mistake waiting to happen.

What's Next

Environment is bootstrapped. Production has k3s, Flux, and a merged kubeconfig that will not fight with staging. Deploying the actual applications is Step 2, and that is tomorrow's work. Part Two once it is done.

History/Context

Applications used to run directly on bare metal servers(some still do). One app, one server, no isolation. This caused problems you would expect; dependency conflicts between applications sharing the same machine, and a single point of failure if that one server went down.

The hypervisor solved part of this. Virtual machines let you run multiple isolated operating systems on the same physical hardware, which fixed the dependency problem and made things more fault tolerant. However every VM needed its own operating system, its own resource provisioning, and boots slowly compared to containers.

Containers were the next logical step. Lightweight, fast to start, efficient with resources, and friendly to work with as a developer. They are not magic and they do not solve every problem, but they solved enough of the right problems to become foundational infrastructure.

What Actually Makes a Container a Container

Two Linux kernel features do the most of the work: "namespaces" and "cgroups". These are not the only things that make a container work e.g. "runc", but for the sake of simplicity these 2 are the most important. Remember the 80/20 rule.

Namespaces handle isolation. Basically tricks a process into believing it is the only thing running on the system. A process inside a container sees its own process list, its own network stack, its own filesystem, even though it is sharing a kernel with everything else on the host.

Cgroups handle resource limits. Control groups constrain what a container is allowed to use: CPU, memory, IO, and more. Without cgroups, one noisy container could starve everything else on the host.

I followed along w/this video building a container with nothing but Go, no Docker involved. It was one of those exercises where you understand enough to see how the pieces fit, without fully understanding every detail underneath. I will be coming back to this one.

https://www.youtube.com/live/2e0dhoPeKfI?si=JGA0FuWOvCajojZ0

docker run, exec, and attach

These three commands look similar but get used for different tasks.

docker run starts a new container from an image.

docker exec starts a new process inside a container that is already running, usually a shell, so you can look around. Exit that shell and the container keeps running like nothing happened.

docker attach connects your terminal directly to the container's existing process, usually PID 1. If that container is running something like nginx, attaching shows you its live output. The one thing to be careful of; hitting Ctrl+C here can send a signal to that process and stop the container. Detach safely with Ctrl+P then Ctrl+Q instead.

TL;DR: run to start something new, exec to open a side door and poke around, attach to watch or interact with main process.

Building My First Docker Container

A simple exercise, but a good one for making a Dockerfile stop feeling like a template you copy without reading.

mkdir myapp && cd myapp

A one-line script, greet:

#!/bin/bash
echo "Hello from my container!"

make executable:

chmod +x greet

and the Dockerfile:

FROM ubuntu:24.04

WORKDIR /app

COPY greet .

CMD ["./greet"]

build with:

docker build -t myapp:1.0.0 .

-t myapp:1.0.0 tags the image with a name and version. The trailing . tells Docker to use the current directory as the build context.

The Dockerfile Instructions, One at a Time

• FROM sets the base image.

• RUN executes commands during the build. Each RUN creates a new layer, so related commands should be bundled together rather than spread across multiple lines.

• COPY brings files from the build context into the image.

• WORKDIR sets the working directory for every instruction that follows it.

• ENV sets environment variables available at runtime.

• EXPOSE documents which ports the application uses. It does not actually publish them, that still requires -p at runtime.

• CMD is the default command the container runs on startup, and it can be overridden at runtime.

• ENTRYPOINT works like CMD but is harder to override, which makes it the better choice for a container that should always run one specific executable.

None of this is complicated once it is written down like this. It just was not obvious to me until I built it myself.

Next Time: backing up my Production applications data and possibly migrating Linkding to Postgres with CNPG.

The rule I try to hold myself to: kubectl is for troubleshooting and ad hoc testing only. Flux is the one making sure the cluster matches what is committed to the repo, no matter what I or anyone else deploys by hand.

Here is how the process went; including the mistakes and troubleshooting.

Stage One: Getting It Running

First step was the basic scaffolding. I created a directory for Audiobookshelf under apps/base, then built four manifests: a namespace, a deployment, a service, and a config map.

The namespace and service were simple:

apiVersion: v1
kind: Namespace
metadata:
  name: audiobookshelf

apiVersion: v1
kind: Service
metadata:
  name: audiobookshelf
spec:
  ports:
    - port: 3005
  selector:
    app: audiobookshelf
  type: ClusterIP

The config map took a bit more digging. I checked Audiobookshelf's documentation for the environment variables it accepts. PORT is all I need for now, but HOST and NODE_ENV could be used later.

apiVersion: v1
kind: ConfigMap
metadata:
  name: audiobookshelf-configmap
  namespace: audiobookshelf
data:
  PORT: "3005"

apiVersion: apps/v1
kind: Deployment
metadata:
  name: audiobookshelf
spec:
  replicas: 1
  selector:
    matchLabels:
      app: audiobookshelf
  template:
    metadata:
      labels:
        app: audiobookshelf
    spec:
      containers:
        - name: audiobookshelf
          image: ghcr.io/advplyr/audiobookshelf:v2.35.1
          ports:
            - containerPort: 3005
          envFrom:
            - configMapRef:
                name: audiobookshelf-configmap

A kustomization file tied the four manifests together, and a matching entry under apps/staging wired it into Flux.

To test, I port forwarded locally from "Larry" (my ssh jump box):

kubectl port-forward deployment/audiobookshelf 3005 3005

and from my laptop:

ssh -N -L 3005:localhost:3005 larry-local

Getting there took more commits than I would like to admit. A typo in the image name. A typo in the config map. A case-sensitivity mistake in the namespace. A missing kustomization entry. Every one of them was a small obvious fix once Flux told me the deployment was not converging, which is exactly the point of running things this way. Nothing silently broke. It just sat there refusing to apply until the YAML was correct. see my commit history

Stage Two: Persistent Storage

With the app running, next up was making sure data survived a pod restart. I added a PersistentVolumeClaim and mounted it into the container using subPaths to separate the /config, /metadata, and /audiobooks directories inside one shared volume.

It worked. I killed the pod, it came back, and the data was still there. The only mistake in this stage was forgetting to add the new storage manifest to the kustomization file, which meant the PVC never actually got applied until I caught it a few minutes later.

The single-volume approach looked fine at this point. It would not stay that way.🥀️🥺️

Stage Three: Running as Non-Root and Going Public

This stage did two things: it locked the container down, and it put Audiobookshelf on the internet.

For the security piece, I set the deployment to run as the node user instead of root with privilege escalation disabled. This is where the subPath approach from Stage Two fell apart. I set the security context to UID and GID 1000, expecting that to line up cleanly with the mounted directories.

Instead I kept seeing GID 0 on the mounted paths, and the fix was not a permissions tweak. It was simply mounting three separate PVCs for /config, /metadata, and /audiobooks instead of subPaths on one shared volume.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: audiobookshelf-config
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: audiobookshelf-metadata
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: audiobookshelf-audiobooks
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

apiVersion: apps/v1
kind: Deployment
metadata:
  name: audiobookshelf
spec:
  replicas: 1
  selector:
    matchLabels:
      app: audiobookshelf
  template:
    metadata:
      labels:
        app: audiobookshelf
    spec:
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
        runAsGroup: 1000
      containers:
        - name: audiobookshelf
          image: ghcr.io/advplyr/audiobookshelf:v2.35.1
          ports:
            - containerPort: 3005
          envFrom:
            - configMapRef:
                name: audiobookshelf-configmap
          securityContext:
            allowPrivilegeEscalation: false
          volumeMounts:
            - mountPath: /config
              name: audiobookshelf-config
            - mountPath: /metadata
              name: audiobookshelf-metadata
            - mountPath: /audiobooks
              name: audiobookshelf-audiobooks
      restartPolicy: Always
      volumes:
        - name: audiobookshelf-config
          persistentVolumeClaim:
            claimName: audiobookshelf-config
        - name: audiobookshelf-metadata
          persistentVolumeClaim:
            claimName: audiobookshelf-metadata
        - name: audiobookshelf-audiobooks
          persistentVolumeClaim:
            claimName: audiobookshelf-audiobooks

With that sorted; the last piece was exposing it outside the cluster. I went for a Cloudflare tunnel, and the workflow followed the same pattern I use everywhere else in this repo:

1. create the credential

2. wrap it in a Kubernetes secret

3. encrypt the secret before it ever touches Git.

1. 
cloudflared tunnel create audiobooks

cd ~/.cloudflared

2.
kubectl create secret generic tunnel-credentials \
  --from-file=credentials.json=<CREDS>.json \
  --dry-run=client -o yaml > cloudflare-secret.yaml

3.
sops --age=$AGE_PUBLIC \
  --encrypt --encrypted-regex '^(data|stringData)$' --in-place cloudflare-secret.yaml

Then in the Cloudflare console add a DNS A-record pointing at the tunnel.

Finally I created a cloudflared deployment, and a config map describing the ingress rule:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudflared
spec:
  selector:
    matchLabels:
      app: cloudflared
  replicas: 2
  template:
    metadata:
      labels:
        app: cloudflared
    spec:
      containers:
      - name: cloudflared
        image: cloudflare/cloudflared:2026.7.2
        args:
        - tunnel
        # Points cloudflared to the config file, which configures what
        # cloudflared will actually do. This file is created by a ConfigMap
        # below.
        - --config
        - /etc/cloudflared/config/config.yaml
        - run
        livenessProbe:
          httpGet:
            # Cloudflared has a /ready endpoint which returns 200 if and only if
            # it has an active connection to the edge.
            path: /ready
            port: 2000
          failureThreshold: 1
          initialDelaySeconds: 10
          periodSeconds: 10
        volumeMounts:
        - name: config
          mountPath: /etc/cloudflared/config
          readOnly: true
        # Each tunnel has an associated "credentials file" which authorizes machines
        # to run the tunnel. cloudflared will read this file from its local filesystem,
        # and it'll be stored in a k8s secret.
        - name: creds
          mountPath: /etc/cloudflared/creds
          readOnly: true
      volumes:
      - name: creds
        secret:
          secretName: tunnel-credentials

      # Create a config.yaml file from the ConfigMap below.
      - name: config
        configMap:
          name: cloudflared
          items:
          - key: config.yaml
            path: config.yaml
---
# This ConfigMap is just a way to define the cloudflared config.yaml file in k8s.
# It's useful to define it in k8s, rather than as a stand-alone .yaml file, because
# this lets you use various k8s templating solutions (e.g. Helm charts) to
# parameterize your config, instead of just using string literals.
apiVersion: v1
kind: ConfigMap
metadata:
  name: cloudflared
data:
  config.yaml: |
    # Name of the tunnel you want to run
    
    tunnel: abs

    credentials-file: /etc/cloudflared/creds/credentials.json

    # Serves the metrics server under /metrics and the readiness server under /ready
    metrics: 0.0.0.0:2000
    no-autoupdate: true

    ingress:
    - hostname: abs.angelchavez.net
      service: http://audiobookshelf:3005

    # This rule sends traffic to the built-in hello-world HTTP server. This can help debug connectivity
    # issues. If hello.example.com resolves and tunnel.example.com does not, then the problem is
    # in the connection from cloudflared to your local service, not from the internet to cloudflared.
    - hostname: hello.example.com
      service: hello_world
    # This rule matches any traffic which didn't match a previous rule, and responds with HTTP 404.
    - service: http_status:404

Commit - push and Flux handled the rest. Audiobookshelf-Staging is live at abs.angelchavez.net now.

One thing worth noting for anyone doing the same as me: Cloudflare tunnels cap uploads at 1GiB, which is worth knowing before trying to upload a large audiobook library through the tunnel instead of loading it onto the volume directly/locally.

Where This Leaves Things

3 stages, a dozen or so small fixes, and the app now runs as an unprivileged user; keeps its data across restarts and is reachable from outside the house. All without a single manual kubectl apply making it into the running state.

Repo: github.com/angel-n-chavez/homelab

Next time: I'm going to walk through deploying my Production environment which will be running the exact same apps as staging, but will be used by family and friends.

Today I'll be discussing why I chose Flux to manage my production homelab cluster, over ArgoCD. TL;DR: "It forces me to stay in the CLI, its k8s native, its simple".

Context

I run a k3s cluster that mirrors real world production infra, on a debian13 proxmoxVM. As of right now I run 4 apps/service in a staging cluster, soon I will deploy the production cluster to another VM. Both clusters will be managed via GitOps from a single monorepo in Github, "git is the source of truth for everything". I do believe this is the modern standard with cloud/multi-cloud, hybrid, and air-gapped infrastructure.

repo: Homelab

Why Flux

I chose flux because it forces me to stay in the CLI as there is no GUI. It also forces me to think in a "Git Ops" way. Because of this, this also makes it much simpler to deploy and manage. Ccompared to ArgoCD it can be overwhelming with how much you have at your disposal in the GUI. Easily integrated with AKS, and EKS; unsure on GCP k8s platform. Lastly Flux relies on Kustomize, which is what I use extensively in my homelab.

eventually I'll try Argo and use Helm more but for now i think its better in the long term do go this route. Beside Flux has Helm support and its honestly better, IPO!

End of the Day

Whether you use Flux or Argo the implacations of Gitops for your infra and team can be very beneficial:

• easier to collaborate

• can make infra/app changes via pull-requests

• easier to automate

• no kubectl

• can set up alerts to ensure correctness and notify when there is divergence

The list goes on and its not without its flaws and drawbacks(be ready to read alot of YAML!😂️), however the reality is real world engineering teams adopt this method because it works and it works for this platform engineer!🔒️

Until next time where I'll either be talking about deploying audiobookshelf, or how I am managing backups for my apps data; thanks for stopping by!

This week marked the beginning of my weekly learning-in-public journey in Cloud and DevOps. It’s amazing how much you can build just by showing up every day and doing one thing at a time.

What’s up y’all I’m Angel. This is IT with ANC and welcome to my blog, let’s get into it!

What I Worked On

Here’s a closer look at what I built, configured, and learned from Sunday through Saturday

Debian 13 Home Server Setup

I repurposed an old Surface Laptop 2 as a home server. I installed Debian 13, set up Docker Compose alongside Portainer for container management, and created a homepage dashboard using “Meet-Homepage”. On top of that I configured it as a Tailscale exit node, which came in handy when the network at my job was acting funny when we tried to access work services behind SSO. It allowed me to route my internet traffic through my home server to access work systems behind SSO when using the my jobs wifi or LAN was not letting me. It also inspired others to do the same.

FOG Imaging Server

Together with my colleague Joseph, I deployed a FOG server to enable imaging/cloning PCs on the floor using iPXE with a “silver image.” This helps us standardize computer setup and speeds up deployments across the floor. It also allows us to create images as well.

KASM Workspaces

I single-handedly deployed a KASM Workspaces server so that our team can stream containerized apps or desktops to their browser. We also demoed its isolation features to representatives of another school. Super useful for forensic research, testing, and ensuring a clean sand boxed environment.

Windows Server 2022

I spun up a Windows Server 2022 instance which assisted Joseph with AD, DNS, and DHCP setup. Even though I’m not a windows guy anymore, It’s always good to get hands-on with core Windows infra to complement the Linux side of things.

Proxmox Cluster Documentation

I started writing documentation for our Proxmox cluster, which is where many of these services (KASM, FOG, etc.) are running. Documentation is becoming one of my priorities, because without it, future work becomes fragile or hard to onboard others.

GitHub Repo & GitOps Workflows

I set up a plan for a professional infrastructure repository in GitHub. Part of that was practicing Git workflows like feature branching for documentation changes to start off. This is laying the foundation for more formal GitOps practices, when we add our IaC and our config playbooks as well as future projects.

Key Takeaway

Small consistent steps are powerful. Even basics like documenting, installing tools, and setting infrastructure support are what compound into real skill and confidence. Also; doing things in public adds accountability, clarity, and momentum.

What’s Next

Here’s what I’ll be focusing on in Week 2:

• Make the Master repo for our cluster infra

• Finalize and clean up the documentation structure and add it to the master repo

• Write the first meaningful Ansible playbook (beyond just pinging nodes)

• I will also be (attempting) installing Arch Linux on a PC i picked up recently

Thanks for following along I’ll be back next Friday with Week 2’s reflection and what I learn!

It’s been quite a while since my last entry where I talked about Terraform. That entry was an intro into using Terraform to automate the provisioning of cloud resources, with hopes of diving deeper into it. This is still the plan, however things change, and currently I’ve been working a lot more with Python than IaC. So today I thought I’d share with y’all my experience learning the syntax of Python over the past month.

So my main resource for learning the syntax has been, you guessed it, “Python Crash Course“ by Eric Matthes from Starch Press. If you don’t know already Starch Press comes out with exceptional resources for a plethora of IT topics. PCC takes you from noob to apprentice in chapters 1 - 12 where you will read some theory and explanations, and do code challenges at the end of each chapter. Followed by projects in the remaining chapters(this is where I am).

What, Why, How

Let’s talk day 0, the book starts off real nice. With the author stating how he has been coding since he was 5 and if you didn’t start then you’re doomed to never getting past “Hello World“…haha nah I’m just kidding.

Matthes starts by saying that his dad was a major reason he started coding. His father worked for a pioneer company in the computing realm, Digital Equipment. Matthes started coding as a child because of this. He then goes on to state who the book is for, why learn Python, what to expect in the book, and some resources. In short this book is for those looking to start building as fast as possible, it’s a book for all levels of expertise, but particularly its good for noobs(like me).

Part I

The book is split into 2 parts. Part 1 is the “basics” starting with creating a programming environment tailored to python whether you’re on a mac, windows, or Linux machine. The book then takes the reader from variables, conditionals, loops, functions, etc, to data-structures, OOP, tests, and best practices. My impression of this part of the book was that it was perfect for someone like me to get started with python. Because I like most, am a hands-on learner, I can’t just read it I have to do it. Matthes has the reader covered by making us do challenge labs at the end of each chapter. no hand holding, only the text and minimal simple examples, the challenges are actually challenging which is something I really appreciated.

Here is an example of one of the challenges where I needed to create a program that checks for stored data(username) and retrieves the data or prompts the user for data if none is stored, and then store that data in a json file:

from pathlib import Path
import json

def get_stored_username(path):
    """get stored username if available"""

    if path.exists():
        contents = path.read_text()
        username = json.loads(contents)
        return username
    else:
        return None

def get_new_username(path):
    """prompt for a new username"""

    username = input("What is your name? ")
    contents = json.dumps(username)
    path.write_text(contents)
    return username

def greet_user():
    """greets user"""

    path = Path('FilesAndExceptions/StoringData/username.json')
    username = get_stored_username(path)

    if username:
        print(f'Welcome back, {username}')
    else:
        username = get_new_username(path)
        print(f'We"ll remember you when you come back, {username}')

greet_user()

Part II

Part 2 is where the fun/building really begins. It consists of 3 different projects each with a specific topic/focus in mind. You get to choose between building a game, a data visualization tool(s), or a Web App all with pure python. This is currently where I am at, and I must say I am proud of how far I’ve come. Now by no means does this book make you a professional, not in the slightest. What it does is give you a strong foundation so that you can get good with the fundamentals and if you choose to and put in the work, become an expert.

If you enjoyed this read or you’re inspired to go give this book a try because of my review, I appreciate that and I’m glad I was able to offer some value from this. Check out my GitHub where I am building my first game in python inspired by the book. I also have some other cool stuff.

My GitHub

See y’all next time✌️

It's been a while, and I felt like I really needed to get back to making my learning journey public again. Today, we're talking about Infrastructure as Code (IaC), more specifically, Terraform!

First, what is IaC? It's actually several things - it can be an ad hoc script you've whipped up, configuration management like Ansible or Chef, or provisioning tools like Terraform or Pulumi. To me, IaC is a method or tool used to automate and provision cloud infrastructure. Here's what a quick Google search says: "Infrastructure as Code (IaC) is a process that uses code to provision and manage infrastructure instead of manual processes."

Overview + Setup

Common patterns include:

• Terraform + Config Management (i.e., Ansible)

• Terraform + Server Templates(i.e., Packer)

• Terraform + Orchestration (i.e., Kubernetes)

To get started, you'll need to download and install the Hashicorp Repository from the CLI using your local package manager. Once completed, download and install Terraform. Verify the installation by checking your local repolist or typing "terraform --version".

If you're using Fedora 40 with the DNF package manager, here are the steps:

dnf install -y dnf-plugins-core
dnf config-manager --add-repo https://rpm.releases.hashicorp.com/fedora/hashicorp.repo
dnf install terraform -y
terraform --version

If you're using a different OS, just google "how to install terraform from the CLI on [your OS]."

Next, create a basic "bare bones" main.tf file somewhere in your filesystem. Here's a basic example:

terraform {
    required_providers {
        aws = {
            source = "hashicorp/aws"
            version = "~> 3.0"
        }
    }
}

provider "aws" {
    region = "us-east-1"
}

resource "aws_instance" "example" {
    ami = "ami-0b0ea68c435eb488d" # Ubuntu Xenial Xerus 16.04 LTS
    instance_type = "t2.micro"
}

In this example, the first code block defines which provider we're going to use (AWS), followed by defining a default region. In the third code block, we're defining an EC2 instance named "example," using an Ubuntu 16.04 AMI and a t2.micro instance type.

Important: Make sure you have the AWS CLI installed with the correct user credentials/secrets and that the user has proper permissions in AWS to create an EC2 instance, or this won't work!

Finally, in the directory containing your main.tf file, run the following commands:

terraform init
terraform plan  # to see what changes you're making
terraform apply

Once done, if you have no error messages, check the AWS EC2 Console to see your instance created and running! This is real power! No longer do you need to create and deploy your cloud infrastructure from the console - the console should only be used for verification purposes!

Avoiding Unwanted Cloud Bills

To avoid acquiring a hefty AWS bill overnight, make sure to run the terraform destroy command to ensure that everything you created is destroyed and not left running:

terraform destroy

Congrats! You just used Terraform to provision AWS resources all from the CLI. You can now build on this, as will I in the coming weeks.

Stay tuned for upcoming blog entries where I'll dive into:

• Language Features

• Variables & Outputs

• Testing

• Developer Workflows

• And more!

See y'all in the next one!

Today we get to talk about the fundamentals of User and Group account management specifically in Linux, since that’s what I use daily and since that’s what the cloud runs on. Whether you're configuring access control for an EC2 instance or organizing users within a secure environment, understanding how Linux handles user and group accounts will lay the foundation for effective cloud administration. We'll be covering everything from creating new users to managing group permissions, ensuring you're well-prepared to handle user management in a Linux/cloud context.

With that being said, hey, I'm Angel. This is IT with ANC, and welcome to my cloud blog. Let's get into it!

Overview

As a system administrator, one of your core responsibilities is managing user and group accounts. This task encompasses everything from creating new users to ensuring proper password security and system access. In this post, we'll dive into the foundational tasks for managing user and group accounts on a Linux system, including working with special purpose accounts, password management, and more. By the end of this guide, you'll have a solid understanding of how to efficiently manage these accounts and tailor them to meet your organization's security policies.

Understanding Key Files for User and Group Management

Managing users and groups involves interacting with several critical system files. Here’s a breakdown of two essential ones:

1. /etc/passwd:
   This file contains details about all user accounts, including special system accounts used to run services. It is readable by all users but only writable by the root user. The information in this file includes usernames, user IDs (UIDs), and the location of user home directories.

2. /etc/shadow:
   For added security, Linux uses the /etc/shadow file to store encrypted passwords, ensuring that only the root user has access. Each entry in /etc/passwd corresponds to an entry in /etc/shadow. This file also contains additional fields, such as password expiration information, allowing you to enforce password policies.

Special Purpose System Accounts

Some system accounts are critical for services but should not be accessible by regular users. These are referred to as special purpose system accounts, and they are generally used to run system services.

Administrators can control the range of regular user accounts through the /etc/login.defs file, ensuring that system accounts have their own UID range, distinct from regular users.

Creating and Managing User Accounts

Adding a user account is a two-step process:

1. Create the user account: To create a new user, you can use the useradd command. This command adds a new entry to /etc/passwd and generates a corresponding entry in /etc/shadow.

    useradd newuser
   

2. Set the user password: After creating the account, you need to set a password for the new user using the passwd command. Users can change their passwords, but only the root user can set or reset other users' passwords.

    passwd newuser
   

   By default, the passwd command will prompt the user to change their password after a specified period, ensuring security compliance.

Using Template Initialization Files for New Users

When a new user account is created, a default set of files from the /etc/skel directory is copied to their home directory. These files provide a basic configuration for the new user’s environment. Admins can customize these files to provide pre-configured settings for new users. For example, you can add default bookmarks in a browser or setup default environment variables.

Note that changes to /etc/skel after user creation won’t affect existing users.

Updating and Managing User Passwords

Linux allows users to change their passwords with the passwd command, but as a system administrator, you may need to manage password policies and lock or unlock user accounts:

• To lock a user account:

    passwd -l username
  

• To unlock a user account:

    passwd -u username
  

  Additionally, you can enforce password policies by controlling how often users must change their passwords using the change command. This command lets you specify the number of days between password changes and when a password expires.

Removing User Accounts

When a user no longer needs access to the system, it's important to remove their account. This can be done using the userdel command:

userdel username

You can also delete a user’s home directory and associated files by adding the -r flag:

userdel -r username

Understanding Group Accounts

Groups are an essential part of managing users in Linux. A group allows you to assign collective permissions to a set of users, making resource management easier. For example, you might have groups like admin, developers, or finance, each with specific permissions to access certain files or directories.

• Primary Group:
  Every user is automatically assigned a primary group when their account is created. Typically, this is a group with the same name as the user.

• Secondary Groups:
  Users can also belong to additional groups, granting them access to other system resources.

The /etc/group file holds information about groups, including which users belong to which group. You can add, modify, or delete groups with the groupadd, groupmod, and groupdel commands.

Best Practices for Managing Users and Groups in Linux

Here are some best practices for managing user and group accounts in Linux:

1. Follow the Principle of Least Privilege:
   Ensure that users are only granted the access they need to perform their tasks, and no more. Use groups to control access to resources efficiently.

2. Regularly Update Password Policies:
   Use tools like chage to enforce regular password changes and ensure that passwords are complex enough to resist brute-force attacks.

3. Monitor User Activity:
   Regularly audit user and group information by reviewing the /etc/passwd, /etc/shadow, and /etc/group files. Remove inactive users or accounts that are no longer in use.

4. Use Automation for Account Management:
   For larger organizations, consider using scripts to automate the creation, modification, and deletion of user accounts. This ensures consistency and reduces the potential for errors.

Cloud-Specific Best Practices for AWS

When working in an AWS environment, user and group management becomes even more crucial for securing your cloud resources. Here are some additional best practices tailored to AWS:

1. Leverage IAM for User Management: Instead of managing individual Linux accounts for each user across multiple EC2 instances, use AWS Identity and Access Management (IAM) to manage permissions centrally. IAM roles can be assigned to EC2 instances, allowing users to interact with other AWS services without needing to manually configure credentials.

2. Use AWS Systems Manager for Access Control: AWS Systems Manager Session Manager allows you to securely connect to your EC2 instances without needing to manage individual SSH keys. This way, you can control user access centrally from the AWS Management Console, which simplifies auditing and improves security.

3. Integrate with AWS Directory Service: If you need centralized authentication, AWS Directory Service can help manage Linux users via Microsoft AD, making it easier to apply uniform user policies across different environments, including on-premises and cloud.

4. Automate with CloudFormation or Terraform: Automate the creation and configuration of EC2 instances, including user and group settings, by using infrastructure-as-code tools like AWS CloudFormation or Terraform. This ensures consistent configurations and simplifies deploying new resources with the correct user and group permissions.

5. Monitor with CloudWatch and CloudTrail: Use Amazon CloudWatch and AWS CloudTrail to monitor user activity across EC2 instances. Set up alarms to notify you of suspicious behavior, such as repeated failed login attempts or unauthorized access to sensitive files.

6. Isolate Workloads with VPC and Security Groups: In addition to Linux group permissions, use Virtual Private Clouds (VPC) and Security Groups in AWS to isolate workloads and control traffic to your instances. This provides an additional layer of access control beyond just user/group management at the OS level.

These cloud-specific practices help extend traditional Linux user and group management into the cloud, ensuring that you follow both the least privilege principle and maintain strong security standards across your AWS infrastructure.

Conclusion

Managing users and groups is a foundational task for any system administrator. Whether you're creating new user accounts, updating passwords, or managing group access, understanding the basics of user and group management ensures that your system runs smoothly and securely. By following best practices and leveraging the right tools, you can streamline these processes and maintain a secure environment. As I continue this blog I am going to keep focusing on the fundamentals, like Linux basics, networking basics, and scripting. You’ll find that being “Mr. Fundamentals” when it comes to cloud will take you far and future proof your career in tech. Additionally as I learn more from my classes at Northwest Vista and start my job as a datacenter technician these posts will continue to get more and more technical.

In this entry, we'll dive into how customizing your shell environment can enhance your productivity in the Cloud, covering everything from managing environment variables to automating tasks with Bash functions, and will bring you closer to being a Linux power user. Which will serve you well since 90% of the servers that make up the cloud run some flavor of Linux.

Hey, I'm Angel and this is IT with ANC, welcome to my cloud blog. Let's get into it!

Overview

As you dive deeper into cloud computing, understanding advanced features like environment variables, custom functions, and profile modifications becomes essential. In this post, we'll explore how to customize your shell environment, making it more efficient and tailored to your specific needs. Whether you're managing EC2 instances, automating deployments with scripts, or configuring environments, mastering shell customization can significantly enhance your efficiency and effectiveness.

Understanding Variables: The Building Blocks of Shell Customization

Variables in the shell are a fundamental concept, serving as placeholders for values that can be used across commands and scripts. In Bash, variable names must start with a letter or underscore and can include letters, numbers, and underscores. In the dynamic environment of cloud computing, variables allow you to create flexible and reusable scripts. Whether you're storing critical configuration data or passing parameters between commands, mastering variable management is essential. Remember, they are case-sensitive!

Local vs. Environment Variables:

• Local Variables: Available only within the shell in which they are created.

• Environment Variables: Passed into all other commands and programs started by the shell, making them globally accessible.

A common convention is to use lowercase for local variables and uppercase for environment variables, helping you distinguish between the two at a glance.

Example:

local_variable="test"
ENVIRONMENT_VARIABLE="TEST"

Displaying Variable Values:

• set: Displays all variables (local and environment).

• env & export -p: Displays only environment variables.

• echo $VARIABLE: Displays the value of a specific variable.

Creating Environment Variables: Flexibility and Control

By default, variables in Bash are local. However, you can easily promote a local variable to an environment variable using a few different methods:

1. Exporting an Existing Variable:

    export LOCAL_VAR
   

2. Creating and Exporting in One Step:

    export NEW_VAR="value"
   

Using declare or typeset: These commands can also declare a variable as an environment variable.

declare -x ANOTHER_VAR="value"

For temporary changes, the env command allows you to set environment variables for the duration of a command or script, without permanently altering your environment.

Unsetting Variables: Avoiding Pitfalls

If you need to delete a variable, use the unset command:

unset VARIABLE_NAME

Be cautious with critical system variables like $PATH—unsetting these can cause serious issues!

To safeguard against errors, you can enable the nounset option:

set -o nounset

This will cause an error if a script tries to reference an unset variable, preventing unexpected behavior.

The PATH Variable: The Backbone of Command Execution

One of the most critical environment variables in your shell is $PATH. It contains a list of directories that the shell searches when you enter a command. Understanding how $PATH works is crucial for customizing your environment and ensuring that commands are executed correctly.

Absolute Path vs. Relative Path:

• Absolute Path: Specifies the exact location from the root directory, starting with /.

• Relative Path: Specifies the location relative to the current directory, making it more flexible but potentially less clear.

Why Shell Customization is Essential for Cloud Professionals

In a cloud environment, your ability to quickly adapt and automate tasks is crucial. Shell customization allows you to create a tailored environment that can streamline repetitive tasks, reduce errors, and optimize your workflow. For instance, setting environment variables, writing custom Bash functions, and understanding critical variables like $PATH can save you time and prevent costly mistakes.

Practical Applications in AWS:

• Managing Environment Variables: In AWS, environment variables are often used to store sensitive information like API keys, database credentials, or configuration settings for applications. Understanding how to properly set and manage these variables ensures that your applications run smoothly and securely.

• Bash Functions for Automation: Automating tasks such as starting/stopping EC2 instances, uploading files to S3, or deploying applications can be streamlined using custom Bash functions. This allows you to execute complex commands with a single, simple command, saving time and reducing the likelihood of errors.

• Maintaining Consistent Environments: When deploying across different environments (e.g., development, staging, production), it's vital to maintain consistency. Customizing your shell profiles and using environment variables can help ensure that your environments are consistently configured, reducing the chance of unexpected behavior when scaling in the cloud.

Key Concepts for AWS Professionals

1. Variables:

• Local vs. Environment Variables: In cloud computing, understanding the scope of variables is crucial. For example, a local variable might be used within a script to perform a specific task, while an environment variable could store a value like a region or instance type that needs to be accessible by multiple scripts or applications.

2. The PATH Variable:

• Navigating AWS CLI: The $PATH variable is particularly important when working with the AWS Command Line Interface (CLI). Ensuring that your PATH includes the directories where the AWS CLI and other essential tools are installed allows you to execute commands from any location within your terminal, improving your workflow efficiency.

3. Unsetting Variables:

• Avoiding Misconfigurations: In AWS, incorrect or missing environment variables can lead to failed deployments, broken connections, or even security vulnerabilities. Understanding how to unset variables safely, and knowing when to do so, is critical for maintaining the integrity of your cloud environments.

Real-World Examples:

• Automating EC2 Instance Management: Suppose you frequently start and stop EC2 instances for development purposes. You can create a Bash function that encapsulates this process, setting necessary environment variables like instance IDs and regions within the function. This not only saves time but also reduces the chance of errors.

• Configuring Multi-Environment Deployments: When deploying applications across multiple AWS environments, you can use environment variables to manage configuration differences between environments. By customizing your shell profile, you can easily switch contexts and ensure that you're deploying to the correct environment with the right settings.

Conclusion

By mastering these shell features, you'll not only improve your efficiency in AWS and gain the skills needed to handle complex cloud environments with confidence, but you will be one step closer to becoming a Linux power user. Start customizing your shell today and experience the difference in your cloud workflows!

Welcome to today’s blog, where we’ll dive into the fascinating world of the Command Line Interface (CLI) and Bash scripting. If you’re on a journey to enhance your cloud computing skills, mastering these tools is a crucial step. Whether you’re just starting out or looking to sharpen your existing knowledge, this post will offer you practical insights through a project you can complete, using the Linux CLI and Bash scripting to interact with AWS services.

What to Expect.

In this post, I’ll walk you through:

1. The Project: How I used the CLI and Bash to create the CloudUploader, a tool for uploading files to cloud storage.

2. Lessons Learned: Key takeaways and best practices for working with the CLI and Bash in a cloud environment.

By the end of this entry, I hope you’ll feel inspired to take on your own projects and gain hands-on experience with these powerful tools. So, grab your preferred selection of caffeine (coffee for me), and let’s get started!

Getting Started: My Experience with the CLI and Bash

First things first, going into this project I already had played around with the CLI and Bash and was somewhat comfortable with both. So, my experience will be different from yours depending on your skill level with these tools. Regardless I still learned a lot and ran into my own problems, mainly with the bash portion of it since I'm still novice. Let me also start off by saying that no matter what level you're at, you need to be comfortable with solving problems on your own, and eventually get good at it. Think about it most System Administrators, Jr and Sr Developers, and even IT Support just get really good at identifying the problem, searching for possible solutions, and testing/implementing. As a Jr Developer or SysAdmin, solving problems independently and documenting them will fast-track your career. You are gonna get ahead a lot quicker and get that promotion or position a lot faster vs the other guy; when the time comes. Be a problem solver not a problem bringer.

Alright enough of that, lets get into it.

The CloudUploader Project: An Overview

The CloudUploader CLI is a Bash based tool created/used for seamless upload of files to a specified cloud storage service. In my case of course I used AWS S3 standard. It's also the capstone project for the 1st phase of the Learn to Cloud study guide. Phase 1 focuses on learning Linux CLI, Bash scripting, and computer networking with good resources for all 3 and obviously a project to top it off.

Learn more details or start your Learn to Cloud journey here: Learn to Cloud

Disclaimer: Prerequisites for this project. Get familiar with the basics of AWS Cloud and working in the console. I recommend studying the material found in the Cloud Practitioner exam. Also get familiar with the AWS docs as this will become your best friend and eventually go-to resource when working in the cloud.

Get familiar with the documentation here: AWS DOCS

Step 1: Provisioning Cloud Resources

Pretty straight forward, to start building this for yourself I would start with provisioning your cloud resources for this project. If you don't already have a free tier AWS account(most cloud providers have this tier) I highly recommend doing that first. There's plenty of tutorials and documentation on how to get started so I'll let you pick a resource for that.

After creating your very own AWS free tier account you will be logged in as "root" and free to start building, but wait a second! I seriously advise that you go ahead and create an admin account to work with as it is not best practice and generally unsafe to work as root(also best to set up Multi-Factor Authentication for both root and admin. DO NOT FORGET YOUR LOGIN CREDENTIALS FOR ROOT). If we keep security in mind first we don't have to deal with more headaches later.

Your next step after creating an admin account will be to create an S3 bucket that you want to use for uploading files. you'll be able to use and upload to more than just one S3 bucket, but for now let's just provision one. call it "cli-uploader" or "cloud-uploader-project" whatever makes sense to you.

Step 2: Installing and Configuring the AWS CLI

Next your going to want to download and install the AWS-CLI. To configure the AWS-CLI you will need IAM credentials(access key, secret key), now you can either use your admin account or create a specific IAM user for this project. Do not, I repeat, please do NOT use the root account for this. Your root account shouldn't even have access key credentials to begin with.

I chose to create an IAM user specifically for this project with only access to the CLI not the console, with admin privileges. Best practice here would be to give the user access to only what you need(least-privelege), but for this case I went ahead and gave admin rights.

Helpful links:

CLI Installation

Configure CLI

Step 3: Bash Scripting and Next Steps

Alright great! You have your cloud environment set up, AWS CLI installed and configured, you are ready to start that Bash script. For this part I recommend doing some of your own research and practicing with Bash before you start coding.

The big issue i ran into during this step was learning the syntax, I had already taken programing I & II in Java so was familiar with most programing concepts. To really get familiar with Bash syntax I followed a YouTube course from Gwyneth Peña S. as well as looking on GitHub for people who did the same project and seeing how they did it.

You can look at my GitHub to help you start but the fun part about this is your only limited to what you know. This makes this project scalable in a sense, since you can always work on bugs, add new features, and refactor as you learn throughout time. Hell if you wanted you can take this project all the way to the finish line; setting up a CICD pipeline, configuring Infrastructure as Code, and even containerize using Docker if you really want. Like I said you're only limited to what you know.

My GitHub

Engage with Me: Share Your Experience!

I’d love to hear from you!

• What’s your experience with CLI and Bash?

• Have you encountered any specific challenges while working with these tools?

Feel free to share your thoughts in the comments below or reach out to me directly. Engaging with fellow learners is a great way to enhance our collective knowledge, so don’t hesitate to join the conversation! If you enjoyed this blog, consider subscribing to my newsletter to stay up-to-date with AWS and IT in general. Let’s continue this journey of learning together!

It’s the middle of 2024, and time seems to be flying by. Looking back, I’m proud of how far I’ve come since April 2023 when I started learning networking as a complete beginner. My journey began with repeated failures on the Juniper Networks Certified Internet Associate (JNCIA) exam—not once, but twice! However, by the end of the year, I turned things around after enrolling in a cloud bachelor’s program and acquiring the CompTIA Network+ certification. I also joined my school’s cybersecurity club, where I participated in capture the flags (CTFs) and earned two more certifications(CCST & AWS Cloud Practitioner). Despite these accomplishments, by early 2024, I still hadn’t landed an entry-level IT job, let alone a cloud position. Every job I applied for required professional experience, which I lacked. Sound familiar?

Enter The Cloud Resume Challenge (CRC), created by Forest Brazeal. The CRC is a project designed to give you hands-on experience with cloud technologies, enhance your resume, and demonstrate your ability to manage and deploy cloud-based solutions. In a nutshell, the challenge involves hosting a static website with a visitor counter (powered by a serverless function) and your resume info, all on a cloud service provider like AWS. The project also includes setting up a CI/CD pipeline, which I accomplished using GitHub Actions. The final piece is configuring Infrastructure as Code (IaC), which I plan to complete with Terraform. This is a simplified overview, but I don’t want to overwhelm you with technical jargon just yet!

You can learn more about the challenge here: Forest's Website

Now that you have some context, let me share my personal experience. This won’t be a step-by-step guide, but I do plan to create something similar in the future, broken down into digestible chunks to avoid information overload. Trust me, there’s a lot more to this challenge than meets the eye, so stay tuned for those tutorials!

The Beginning

I first learned about the CRC from Gwyneth Peña S., a cloud YouTuber I admire. Since she specializes in Azure, my initial attempt was with Azure. Unfortunately, I ran into issues, particularly with the visitor counter. I tried using C#, but quickly realized it wasn’t the right fit for me. Overwhelmed by new technologies and college technical classes in Java and AWS, I had to reassess my approach. Looking back, I see that I had spread myself too thin. So, I made an "executive decision" to push my code to GitHub and recreate/restart the project in AWS.

If you’ve made it this far, kudos to you! (And I guess I’m doing alright so far.) Check out my completed AWS serverless resume website here: My Resume Website

Recreation

Recreating the project in AWS was much smoother. I reused the same HTML and CSS for the frontend so the site looked the same. I uploaded my files to S3, created a CloudFront distribution for dynamic caching and HTTPS support, and registered a domain in Route 53. I then wrote a Lambda function in Python 3.11, which worked seamlessly (C# was definitely not for me). This function interacts with a DynamoDB database to keep track of the visitor counter. I could've used API Gateway for secure communication between the function and the database, though Lambda's auto-created "functionURL" worked fine for me. A bit of JavaScript was needed to invoke the Lambda function and display the updated data. However, when I visited the site, the counter wasn’t displaying due to a timing issue with the DOM content loading. Fortunately, a one-line fix with ChatGPT’s help resolved the issue. (Again, this isn’t a detailed guide, so if you want to be notified when I publish those tutorials, subscribe to my newsletter!)

The final step was creating a CI/CD pipeline for the frontend files using GitHub Actions, which turned out to be surprisingly simple. The project was complete, except for the IaC component, which I plan to finish soon. I also intend to learn Docker and containerize the project for fun.

Final Thoughts and Key Takeaways

Recreating this project in AWS turned out to be a smooth process, thanks to the lessons I learned from my initial attempt in Azure. Here’s a quick summary of what I accomplished:

• Reused Frontend Code: I used the same HTML and CSS, ensuring consistency in design.

• AWS Setup: I uploaded files to S3, set up CloudFront for dynamic caching and HTTPS, and registered a domain with Route 53.

• Serverless Function: I wrote a Lambda function in Python 3.11, which interacts with DynamoDB to manage the visitor counter.

• CI/CD Pipeline: I implemented a straightforward CI/CD pipeline using GitHub Actions for the frontend.

Key Takeaways:

1. Flexibility is Key: Switching from Azure to AWS was a strategic decision that aligned better with my skills and project requirements. Sometimes, changing tools or platforms can make a significant difference.

2. Hands-On Experience Matters: Completing the CRC not only enhanced my technical skills but also provided practical experience that is highly valued by employers.

3. Embrace Learning: Challenges like these are opportunities to learn and grow. Even if things don’t go as planned initially, perseverance and adaptability can lead to success.

Whew! I haven’t written this much in a while(hopefully it doesn't show) lol XD.

Thank you for following along with my journey through the Cloud Resume Challenge. Stay tuned for more posts, beginning with an in-depth, step-by-step guide on the frontend portion of this project. If you’re interested in learning more or have any questions, don’t hesitate to subscribe to my newsletter!