Developer access: deploy and debug

For a teammate who ships application changes to node1 and debugs them, without being a cluster admin. Two front doors, and neither one can change what runs on its own:

Tool	Identity	You can	You cannot
Argo CD — https://argocd.tiktuzki.com	your Google account (role:deployer)	see every app; sync, restart and read logs of the application workloads	sync platform apps (Traefik, monitoring, databases, ...), delete apps, sync local manifests
kubectl	client cert, group tiktuzki-developers	in demo: read pods, logs, events, deployments, services, ingresses, configmaps; exec, port-forward, delete a pod. Cluster-wide: list nodes and namespaces	read Secrets; edit or create anything; kubectl debug; any other namespace

What runs is decided by Git. Argo CD applies the main branch, and every application has selfHeal: true: a hand edit made any other way is reverted within minutes.

Onboarding checklist

Things the cluster owner does are marked owner; the rest are yours.

1. owner — adds your Google email to Argo CD's policy and records you in the access registry.

2. Join the NetBird network (the owner sends a setup key). The API server only answers on the LAN and the overlay, so kubectl does nothing without it. Check:

   nc -z -G3 100.66.50.60 16443 && echo "API reachable"

3. owner — mints your kubeconfig (a 90-day certificate) and shares it through Proton Pass. Save it, then:

   chmod 600 ~/.kube/microk8s-<you>.conf
   export KUBECONFIG=~/.kube/microk8s-<you>.conf
   kubectl auth whoami        # Username <you>, Groups [tiktuzki-developers system:authenticated]
   kubectl get pods           # the context defaults to namespace demo

   Never paste the file into chat, commit it, or copy it to another machine. If it leaks, tell the owner at once: a certificate cannot be revoked, only outlived.

4. Sign in at https://argocd.tiktuzki.com with Google. You should see the applications; Sync is enabled on the app workloads and disabled on the platform ones.

5. owner — gives you a way to propose changes to the Git repo (see Deploying).

6. Put the certificate's expiry date in your calendar, two weeks early. Renewal is a new kubeconfig from the owner.

Deploying a change

Each application is a Helm chart in charts/<app>/, deployed with its values-dev.yaml. A release is almost always a new image tag:

charts/<app>/values-dev.yaml

image:
  tag: "0.5.105"     # always a real version — never `latest`, it makes rollback undescribable

1. Check the tag exists in the registry before proposing it. A typo is the most common failed deploy: the pod sits in ImagePullBackOff.

2. Open a pull request to main that changes only what the release needs. Say in the description what changes and how to tell it worked.

3. The owner reviews and merges. Argo CD notices within about three minutes. To skip the wait, open the app in Argo CD and press Refresh, then Sync.

4. Watch it roll out:

   kubectl rollout status deploy/<deployment> -n demo
   kubectl get pods -n demo -w

   In Argo CD the app should return to Synced / Healthy. Progressing for longer than a few minutes means the new pods are not becoming ready — go to Debugging.

5. Rolling back is a revert PR (git revert <commit>), reviewed and merged like any other change. Argo CD's own Rollback button is disabled while auto-sync is on, and a rollback outside Git would be undone by the next sync anyway.

Read the values file's comments first

Some charts encode a deploy order in comments — charts/9router/values-dev.yaml, for example, must not enable its ingress before the admin password is set. Those comments are the runbook; skipping them can publish an unprotected console.

Debugging

kubectl get pods -n demo                              # STATUS and RESTARTS first
kubectl describe pod <pod> -n demo                    # Events at the bottom: pull errors, OOMKilled, probes
kubectl get events -n demo --sort-by=.lastTimestamp   # namespace-wide, newest last
kubectl logs <pod> -n demo -c <container>             # pods here run a linkerd-proxy sidecar too
kubectl logs <pod> -n demo -c <container> --previous  # the crashed container, after a restart
kubectl exec -it <pod> -n demo -c <container> -- sh   # look around inside
kubectl port-forward svc/<service> 8080:<port> -n demo  # reach a ClusterIP service locally
kubectl delete pod <pod> -n demo                      # restart one pod; its Deployment recreates it

To restart a whole Deployment, use Argo CD: the resource's ⋮ → Restart action. kubectl rollout restart is a patch on the Deployment, which you do not have.

Symptom	Usually
ImagePullBackOff	the tag does not exist, or the registry needs a pull secret
CrashLoopBackOff	the app exits on start — logs --previous
OOMKilled in describe	the memory limit is too low for the new version — raise it in values-dev.yaml via PR
Pending	no room on node1, or a volume that does not bind — describe says which
Argo CD OutOfSync that will not settle	something mutates the object after apply — tell the owner

exec is trusted access

You cannot read Secrets through the API, but anything a pod receives as an environment variable or a mounted file is visible to exec. Do not copy those values out of the pod.

Asking for more

Another namespace, kubectl debug, Grafana or kafka-ui are separate grants — ask the owner. Each is recorded in the access registry with a reason and a review date.