Why Server Configs Belong in Git, Not in Your Head
Advertisement
Almost every server I take over has the same quiet problem: the configs work, but nobody can tell you why they look the way they do.
An Nginx vhost has proxy_read_timeout bumped to 300 seconds. A fail2ban jail has a whitelisted IP range. A sysctl tweak doubles a connection limit. Each change was probably the right call at the time. It was made live, under pressure, and never written down anywhere except in the file itself. Six months later, nobody remembers which lines are load-bearing and which are leftover experiments, and nobody wants to be the one to remove them.
The lightest possible fix: etckeeper
For a single server, the smallest useful step is etckeeper. It puts /etc under git and commits automatically whenever a package manager touches a config file:
apt install etckeeper
cd /etc
git log --oneline -10
From that point on, every manual edit you make gets picked up on the next package install, or you can run git add -A && git commit yourself right after editing. The payoff hits the first time something breaks after a config change and you run:
git log -p -- nginx/sites-enabled/example.com
You see exactly what changed, when, and—if you wrote a commit message—why. That last part matters. git commit -m "bump proxy_read_timeout for slow report export endpoint" is the difference between future-you understanding a config and future-you being terrified to touch it.
For multiple servers: a configs repo per client
etckeeper is local to one machine. That's fine for a single VPS, but it doesn't scale when a client has three or four servers with related configs. For those, I keep a separate Git repo structured by host:
client-configs/
├── web-01/
│ ├── nginx/sites-available/example.com
│ └── fail2ban/jail.local
├── db-01/
│ ├── postgresql/postgresql.conf
│ └── pgbackrest/pgbackrest.conf
└── deploy.sh
Changes happen locally. They get reviewed in a diff before going anywhere. Then a small deploy.sh script pushes the relevant files to the right host:
#!/bin/bash
HOST=$1
rsync -av "$HOST/" "deploy@$HOST.internal:/" --dry-run
# remove --dry-run once the diff looks right
The --dry-run default is deliberate. Config deploys are exactly the kind of thing where a misplaced trailing slash in rsync can overwrite more than intended. The default behavior forces you to look at what will change before it actually does.
What this buys you that documentation doesn't
The usual alternative is a wiki page or a README describing "important config changes." These rot almost immediately. Updating a wiki is a separate step from making the change, and separate steps get skipped under time pressure. Git history doesn't have that problem: the record is a byproduct of doing the work itself.
It also forces a workflow shift. When a config lives in git, the natural move is to edit it locally, commit with a reasoning message, and then deploy—rather than SSHing in and hacking the live file directly. Most of the value isn't in having a backup; it's in the habit of writing down why a moment before you do the what.
Three months later, when you're staring at a config wondering if a particular line can be safely removed, that one sentence in a commit message saves an hour of cautious archaeology.
Advertisement