In mid-August I was setting up the production cluster and moving my own Gitea server onto it. It was a good moment to tidy up the path code takes from a commit to a running pod. I had been running the loop on the home cluster since 1 August, first with a test app, then with the first real website. Now it had to handle everything.
The loop in four steps
- A push to the app repository starts a Gitea Actions workflow.
- CI builds an image and pushes it to the OCI registry built into Gitea, tagged with the commit SHA.
- The same workflow clones the GitOps repository, swaps the image tag in the manifest and commits “Deploy app SHA”.
- Argo CD sees the difference and marks the app “OutOfSync”. I start the sync.
Step three matters most. The cluster never asks the registry “what is the newest”. The version that runs is written down in git, so the GitOps repository’s git log doubles as the deployment history, and rolling back a deployment is a git revert of one commit.
# .gitea/workflows/build.yml (generalized)
name: Build and deploy
on:
push:
branches: [main]
jobs:
build:
runs-on: homelab
steps:
- uses: actions/checkout@v4
- name: Build and push
run: |
echo "${{ secrets.REGISTRY_TOKEN }}" | docker login registry.example.com -u ci --password-stdin
docker build -t registry.example.com/me/app:${{ github.sha }} .
docker push registry.example.com/me/app:${{ github.sha }}
- name: Bump the image tag in the GitOps repo
run: |
git clone --depth=1 "https://ci:${{ secrets.GITOPS_TOKEN }}@git.example.com/me/gitops.git" gitops
cd gitops
sed -i "s|image: registry.example.com/me/app:.*|image: registry.example.com/me/app:${{ github.sha }}|" app/deployment.yaml
git diff --quiet && exit 0
git -c user.name="app CI" -c user.email="$CI_EMAIL" commit -am "Deploy app ${{ github.sha }}"
git push
The registry token can only write packages, and the cluster pulls images with its own read-only token, sealed as a SealedSecret separately in every namespace.
Why nothing deploys itself
Argo CD can sync automatically and heal any drift from git. On both of my clusters that option is off, and it is a decision, not a to-do. A commit in git is a proposal. A deployment is a separate, deliberate action, and before it I see the exact diff. With one person running everything, that is the cheapest form of change review there is.
Manual sync has its traps. One of them caused an error in production. In Argo CD, both a partial sync (selected resources only) and the apply sync strategy (instead of the default hook) skip hooks. My app ran its database migration as a pre-deploy hook Job. The sync inherited the resource list from the previous operation, the migration never ran, and for a short while the new version asked the database for a column that did not exist yet. Since then I sync such apps whole, with the default strategy, and check that the migration Job actually ran.
Renovate: updates as pull requests, not surprises
A week later Renovate joined in. Once a week it reads the GitOps repositories and opens pull requests with new Helm chart and image versions. The rules are strict:
{
"extends": ["config:recommended"],
"schedule": ["* * * * 1"],
"minimumReleaseAge": "3 days",
"packageRules": [
{
"description": "Releases that change behavior wait a week",
"matchUpdateTypes": ["minor", "major"],
"minimumReleaseAge": "7 days"
},
{
"description": "Cluster networking: the version shows on the dashboard, a PR only after a deliberate tick",
"matchPackageNames": ["cilium", "quay.io/cilium/**"],
"dependencyDashboardApproval": true
},
{
"description": "Our own images are bumped by CI; a Renovate PR would fight it",
"matchPackageNames": ["registry.example.com/**"],
"enabled": false
}
]
}
Nothing merges automatically. The soak after a release (three days for patches, a week for the rest) is a cheap filter for releases that quickly get a fix. And merging a PR changes only git, not the cluster, because sync is still manual. So an update passes two gates: I read the changelog when merging and the diff when syncing.
One more setting, not in the snippet above: by default Renovate waits forever on releases without a publication date, and some registries do not provide one. Setting minimumReleaseAgeBehaviour: "timestamp-optional" keeps the soak wherever a date exists and stops the rest from stalling.
What I learned
- Image tag = commit SHA, never
latest. Then git tells you exactly what is running, and rolling back is an ordinary revert. - Exclude your own images from Renovate. If CI writes the tag into git, a second robot doing the same only produces conflicts.
- Watch your hooks when syncing by hand. A partial sync or the
applystrategy silently skips migrations. The safest path is to sync the whole app with the default strategy and check the Job before calling the deployment done.