Building network automation that outlives its author

A platform where engineers run automation as a shell command, not a git workflow, NetBox stays the single source of truth, and zero secrets touch the repo.

The numbers on this one are small on purpose: two tools, three automation layers, one source of truth, zero credentials in the repository. Small by design, because the platform underneath them is built to outlive the person who built it, and a wide blast radius is the fastest way to make an engineer afraid to touch what you left behind.

The graveyard nobody inventories

Network automation has a failure mode that never makes the postmortem, because nothing breaks. It just quietly stops existing. A script lives in someone’s personal folder, the credentials it needs live in that same someone’s head, and the day they change teams, the fifth engineer who needs the tool rewrites it from scratch. Finding the fourth engineer’s copy would have been faster, if anyone had known to look.

I have watched this happen enough times to stop treating it as bad luck and start treating it as a design problem. The team did not need another clever script. It needed automation that behaves like a product: one platform, one contribution process, and a way to consume it that does not require every network engineer to become a git user first.

Splitting the audience in two

The architecture decision that carries the whole platform is a hard line between who consumes and who contributes. Engineers and operations teams consume. They run tools on a shared, hardened jump host, and a one-time init script sets each person up: it prompts for a NetBox API token and TACACS credentials, then encrypts both with a passphrase only that engineer knows. After that, automation is a shell command. No clone, no branch, no pull request, just the tool doing the thing.

Developers contribute through the workflow you would expect: local development, feature branch, pull request, CI, review, merge. But the source on the jump host is read-only, owned by a service account, so the running platform can never drift from what is actually in the repository. Nobody hand-edits a script on the box at 2am and forgets to commit it. That gap, between what is running and what is documented, is where most tribal knowledge dies.

NetBox is the only thing allowed to be right

Every tool in the platform treats NetBox as the network’s single source of truth: devices, addressing, VLANs, sites. Tools query it live instead of carrying their own inventory files, which means two things follow almost automatically. The platform gets more accurate exactly as fast as the source of truth does, and it can never quietly disagree with it. A static host list rots the day after someone writes it. A live query against NetBox cannot rot, because there is nothing to go stale.

Three layers, one set of rules

Underneath, it is three layers doing different jobs. Python 3.11+ tools, managed with UV for reproducible environments, do the heavy logic. Ansible playbooks run against NetBox’s dynamic inventory for anything that touches devices at scale. Bash handles host setup and the per-user onboarding script, because the first five minutes of using the platform should not require reading documentation.

The rules that hold it together are not suggestions. No credential of any kind lives in the repository. Vault files are gitignored, with only example templates committed, so a new contributor can see the shape of a secret without ever seeing one. Linting, pre-commit hooks, and CI keep every contribution to the same standard, whether it lands on a Tuesday afternoon or an on-call Saturday. And because AI-assisted development is genuinely part of how this gets built now, project context is checked into the repo too, so every session, human or otherwise, starts already briefed instead of re-discovering the same decisions.

What shipped, and what it’s proving

Two tools carry the pattern so far. netdiagram builds architecture diagrams straight from live NetBox data, and it renders the same network two different ways, one for a technical audience and one for leadership, because those are genuinely different diagrams even when they describe the same racks. argus is a network state observer: it collects topology, routing, and ACL state with change history, which turns “what changed since yesterday” from an investigation into a query.

Around both tools sits the part that actually matters for the platform’s survival: the onboarding script, the NetBox-wired inventory, a contribution guide a new engineer can follow without asking me first, and a CI pipeline that keeps the whole thing releasable on demand instead of whenever someone remembers to check.

The mistake that shaped the platform

I got the ownership model half wrong the first time. My instinct was to make myself the reviewer on everything, which feels responsible right up until you notice you have rebuilt the exact single point of failure you were trying to eliminate. The fix was boring: document the standard well enough that someone else can hold it, then actually let them.

The platform is young, and it is still mostly my two tools proving the pattern rather than a shelf full of contributions from a team. That is the honest state of it. But the pieces that matter are already load-bearing: engineers run commands instead of learning git, credentials never touch a repository, and NetBox stays the one place nobody has to double-check. Every tool added from here inherits the same delivery, the same security posture, and the same documentation standard, which is the whole bet. Automation that outlives its author does not happen by accident. You have to build the part that lets you leave.

← ALL POSTS