185 lines
7 KiB
Markdown
185 lines
7 KiB
Markdown
# dumpnet-argo
|
|
|
|
GitOps cluster management for dumpnet — a single-node Talos/Kubernetes cluster on AWS, managed via ArgoCD.
|
|
|
|
## Architecture
|
|
|
|
- **Talos Linux** on EC2 (t3.medium) — immutable, API-driven OS
|
|
- **ArgoCD** — GitOps continuous delivery
|
|
- **ingress-nginx** — ingress controller (hostNetwork mode)
|
|
- **App of Apps** pattern — all workloads defined in `manifests/`
|
|
- **Terraform** — cluster infrastructure and bootstrap
|
|
- **SOPS + age** — secret encryption for sensitive cluster files
|
|
|
|
## Prerequisites
|
|
|
|
- `terraform`
|
|
- `kubectl`
|
|
- `talosctl`
|
|
- `sops` + `age`
|
|
- AWS CLI configured (`aws configure`)
|
|
|
|
## Secrets Setup (first time only)
|
|
|
|
Sensitive files (`talosconfig`, `controlplane.yaml`, `worker.yaml`) are **gitignored** and never stored in the repo in any form. They are managed as follows:
|
|
|
|
- **Cluster credentials** (kubeconfig, talosconfig, ArgoCD password) → stored in AWS Secrets Manager at `dumpnet`
|
|
- **Talos machine configs** → encrypted with SOPS + age, stored outside the repo
|
|
- **Age private key** → lives at `~/.age/key.txt` — **back this up securely**
|
|
|
|
To decrypt Talos configs to `/tmp` for one-off `talosctl` use:
|
|
```bash
|
|
make decrypt
|
|
# files appear at /tmp/controlplane.yaml, /tmp/worker.yaml, /tmp/talosconfig
|
|
# they are not persisted after reboot
|
|
```
|
|
|
|
## First-Time Setup
|
|
|
|
### 1. Clone and init
|
|
|
|
```bash
|
|
git clone https://forge.keane.sh/ian/dumpnet-argo.git
|
|
cd dumpnet-argo
|
|
make init
|
|
```
|
|
|
|
### 2. Configure variables
|
|
|
|
```bash
|
|
cp terraform/terraform.tfvars.example terraform/terraform.tfvars
|
|
# edit terraform/terraform.tfvars with your hosted_zone_id
|
|
```
|
|
|
|
### 3. Spin up the cluster
|
|
|
|
```bash
|
|
make apply
|
|
```
|
|
|
|
> **Note:** `make apply` runs two Terraform passes internally. The first pass
|
|
> (`-target=talos_cluster_kubeconfig.this`) brings up the EC2 instance, bootstraps
|
|
> the cluster, and retrieves the kubeconfig. The second pass then uses that live
|
|
> kubeconfig to provision Kubernetes resources (namespaces, ArgoCD Helm release).
|
|
> This two-phase approach is necessary because the Helm and Kubernetes Terraform
|
|
> providers need a reachable cluster to initialize.
|
|
|
|
This will:
|
|
- Allocate an EIP
|
|
- Create a dedicated subnet and security group
|
|
- Launch a Talos EC2 instance (t3.medium) with machine config as user-data
|
|
- Associate the EIP (no auto-assigned public IP)
|
|
- Apply the machine configuration via Talos API
|
|
- Bootstrap etcd
|
|
- Retrieve the kubeconfig
|
|
- Create the `ingress-nginx` namespace with privileged pod security
|
|
- Install ArgoCD via Helm with a generated admin password
|
|
- Create Route53 DNS records pointing to the EIP
|
|
- Store all credentials in AWS Secrets Manager (`dumpnet/cluster`)
|
|
|
|
### 4. Post-apply bootstrap (one-time)
|
|
|
|
```bash
|
|
make post-apply
|
|
```
|
|
|
|
This script:
|
|
- Fetches kubeconfig and talosconfig from Secrets Manager
|
|
- Waits for the node to be ready
|
|
- Imports the ingress-nginx namespace into Terraform state
|
|
- Applies the ArgoCD App of Apps
|
|
|
|
### 5. Get ArgoCD password
|
|
|
|
```bash
|
|
make argocd-password
|
|
```
|
|
|
|
Then log in at https://argocd.dumpnet.chat — ArgoCD will finish deploying ingress-nginx and metrics-server automatically.
|
|
|
|
## Rebuilding from Scratch
|
|
|
|
To fully tear down and recreate the cluster:
|
|
|
|
```bash
|
|
make clean # force-delete secret, empty S3, clear stale kube/talos contexts
|
|
make destroy # tear down all terraform resources
|
|
make apply # recreate everything
|
|
make post-apply # wait for cluster ready, bootstrap ArgoCD
|
|
make bootstrap # apply App of Apps
|
|
```
|
|
|
|
## Day-to-Day
|
|
|
|
- **Add a new app**: add a manifest to `manifests/` and values to `charts/` — ArgoCD picks it up on next sync
|
|
- **Add a DNS record**: add the subdomain to `dns_records` in `terraform/terraform.tfvars` and run `make apply`
|
|
- **Cluster access**: `make kubeconfig` or `make talosconfig` (pulls from Secrets Manager, safe to re-run — automatically clears any stale context from a previous cluster before merging)
|
|
- **Emergency talosctl access**: `make decrypt` to get configs in `/tmp`
|
|
|
|
## Makefile Reference
|
|
|
|
| Command | Description |
|
|
|---------|-------------|
|
|
| `make apply` | Create/update cluster infrastructure |
|
|
| `make plan` | Preview infrastructure changes |
|
|
| `make destroy` | Tear down everything |
|
|
| `make clean` | Pre-destroy cleanup (force-delete secret, empty S3 buckets, clear stale kube/talos contexts) |
|
|
| `make post-apply` | One-time bootstrap after fresh cluster creation |
|
|
| `make bootstrap` | Apply App of Apps only |
|
|
| `make kubeconfig` | Pull kubeconfig from Secrets Manager (clears stale context first) |
|
|
| `make talosconfig` | Pull talosconfig from Secrets Manager |
|
|
| `make argocd-password` | Print ArgoCD admin password |
|
|
| `make decrypt` | Decrypt Talos configs to /tmp |
|
|
|
|
## Repo Structure
|
|
|
|
```
|
|
dumpnet-argo/
|
|
├── Makefile # top-level commands
|
|
├── scripts/
|
|
│ └── post-apply.sh # one-time bootstrap after fresh cluster
|
|
├── apps/
|
|
│ └── apps.yaml # ArgoCD App of Apps root
|
|
├── manifests/ # ArgoCD Application manifests
|
|
│ ├── argocd.yaml
|
|
│ ├── ingress-nginx.yaml
|
|
│ └── metrics-server.yaml
|
|
├── charts/ # Helm values per app
|
|
│ ├── argocd/
|
|
│ │ └── values.yaml
|
|
│ └── ingress-nginx/
|
|
│ └── values.yaml
|
|
└── terraform/ # cluster infrastructure
|
|
├── main.tf
|
|
├── versions.tf
|
|
├── variables.tf
|
|
├── security_group.tf
|
|
├── network.tf
|
|
├── namespaces.tf
|
|
├── argocd.tf
|
|
├── dns.tf
|
|
└── outputs.tf
|
|
```
|
|
|
|
## Forking / Multiple Environments
|
|
|
|
Global per-cluster config lives in **`values.yaml`** at the repo root:
|
|
|
|
```yaml
|
|
clusterName: dumpnet
|
|
domain: dumpnet.chat
|
|
repoURL: https://forge.keane.sh/ian/dumpnet-argo.git
|
|
certEmail: dumpnetcerts@keane.sh
|
|
awsRegion: us-east-1
|
|
```
|
|
|
|
This file is passed as the first `valueFiles` entry to every Helm chart, so `domain`, `certEmail`, `clusterName`, and `awsRegion` are available as `{{ .Values.* }}` in all chart values. Terraform variables in `terraform/variables.tf` mirror these same settings for the infrastructure side.
|
|
|
|
**The one thing that can't be templated** is `repoURL` in the ArgoCD `Application` manifests themselves (under `apps/` and `manifests/`). These are plain YAML consumed by ArgoCD before any Helm rendering happens — there's no way to interpolate them without a [Config Management Plugin](https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/). When forking this repo, do a global find/replace on `forge.keane.sh/ian/dumpnet-argo` with your own repo URL.
|
|
|
|
## Notes
|
|
|
|
- The control plane taint is disabled via `allowSchedulingOnControlPlanes: true` in the Talos machine config — no manual taint removal needed
|
|
- The `ingress-nginx` namespace is created by Terraform with `pod-security.kubernetes.io/enforce=privileged` to allow hostNetwork mode
|
|
- ArgoCD manages itself after initial Helm install — future upgrades go through the repo
|
|
- Sensitive files are gitignored entirely — there is no encrypt/commit workflow, only decrypt-to-tmp when needed
|