Fleet operations: managing many servers
You manage dozens of servers as a fleet rather than one by one: inventory, parallel commands, configuration drift, versioned /etc, golden images and standard builds.
Level 401 · 15 lessons · 12 practice labs
Lessons
Inventory
- Fleet inventory: which server, what job, who owns it
- Collecting server facts automatically
- Naming and tagging standards
- Short task: produce an inventory of twelve servers with a script
Parallel operations
- Parallel SSH tools: pssh and ClusterShell
- Rolling changes: one first, then the rest
- Aggregating output and spotting differences: dshbak
- Short task: restart all web servers in turn without downtime
Drift and standards
- Versioning /etc with etckeeper
- Detecting configuration drift
- Golden images and standard builds
- Automated installs: kickstart and preseed
- Short task: find three servers that drifted from the standard and bring them in line
Wrap-up
- Change records and approval
- Capstone: roll a security setting out across a twelve-server fleet in stages, with evidence
Practice labs
- Collect the inventory of twelve servers: Gather distribution, kernel and memory from every server with an SSH loop into one CSV file.
- Group servers by role and environment: Split the inventory file into role and environment groups and test a selector targeting production web servers only.
- Find the gap between inventory and reality: Compare the recorded inventory with data gathered from the servers and report an unrecorded service, a missing server and a wrong role.
- Run a command on all servers in parallel: Run the uptime command on twelve servers in parallel and show the output tidily by host name.
- Separate unreachable servers and retry: In a parallel run collect failed servers in a separate list, set a timeout and retry only those.
- Apply a change in waves with a stop condition: Apply a config change to one server first, then three at a time, with a script that stops and rolls back when a health check fails.
- Find how one file differs across servers: Collect the hash of the SSH config file from all servers and list the ones that differ from the majority.
- Apply a baseline in a re-runnable way: Write a script applying time sync, log forwarding and SSH settings to all servers that reports no change on the second run.
- Produce a standards compliance report: Check a ten-item standard on all servers, tabulate passes and failures per server and prioritise the deviations.
- Distribute a new admin's key to every server: Add a public key to every server and verify no duplicate line was created and login works.
- Remove a departed admin's access everywhere: Find and remove a departed person's key, account and sudo rule on all servers and report anything left.
- Take over a fleet: tidy inventory, access and standards: On Yakamoz Telekom's twelve servers build the inventory, clear unknown keys, apply the baseline and hand over with a compliance report.
About this course
You manage dozens of servers as a fleet rather than one by one: inventory, parallel commands, configuration drift, versioned /etc, golden images and standard builds.
What you will learn
- Inventory
- Parallel operations
- Drift and standards
- Wrap-up
Prerequisites
You should have practised the topics of the same path up to level 301.
Lab environment
Lessons run in your own Kali Linux terminal in the browser, connected to a private lab network. Nothing to install.
Frequently asked questions
- Is this course right for my level?
- The course is at level 401; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.