Resource control and service isolation with systemd
You set CPU, memory and I/O limits with cgroups v2, isolate services with options such as DynamicUser and ProtectSystem, and tighten them while measuring the effect.
Level 401 · 17 lessons · 15 practice labs
Lessons
cgroups v2
- cgroups v2: hierarchy, controllers and slices
- Observing with systemd-cgls and systemd-cgtop
- Slices and scopes: grouping resources
- Short task: map which service sits in which slice
Limits
- CPU: CPUQuota and CPUWeight
- Memory: MemoryMax, MemoryHigh and OOM behaviour
- I/O limits: IOWeight and IOReadBandwidthMax
- TasksMax: against fork bombs
- Temporary, limited jobs with systemd-run
- Short task: limit a noisy-neighbour service so it can't starve the others
Isolation
- DynamicUser: a throwaway user on every run
- ProtectSystem, ProtectHome and ReadWritePaths
- PrivateTmp, PrivateDevices and PrivateNetwork
- Capabilities and system call filtering
- Scoring with systemd-analyze security
- Short task: lower a web service's exposure score without breaking it
Wrap-up
- Capstone: run five services fairly and in isolation on a shared server
Practice labs
- Find which control group a process is in: Find the cgroup path and slice of a given PID and show its place in the group tree.
- Find the heaviest unit from the group view: Looking at unit level instead of processes, identify the service leading in CPU, memory and I/O consumption.
- Delegate controllers to a subtree: Enable delegation so a service can manage its own subgroups, verify the active controllers and set a limit in a subgroup.
- Put a memory ceiling on a service: Define a memory maximum for a leaking service and show only that service is killed when the limit is reached.
- Balance services with CPU quota and weight: Cap a batch service at fifty percent, give the web service a higher weight and measure the split under load.
- Limit a backup job's disk I/O: Set per-device read and write rate limits on a backup service and show with measurements that database latency recovers.
- Place services in a custom slice: Create a new slice for reporting services, move two services into it and put a shared memory limit on the slice.
- Put resource limits on user sessions: On a shared server define task and memory limits per user session and show one user cannot affect the others.
- Build a two-level resource hierarchy for tenants: Design nested slices for three tenants, enforcing tenant totals and per-service limits separately, and verify with a load test.
- Make a service's file system read-only: Add settings that show system directories read-only to a service and allow writes only to its data directory.
- Isolate a service with its own user and private /tmp: Move a service running as root to a dynamic user, a private temp directory and the single capability it needs.
- Narrow a service with a system call filter: Apply a system call filter and address family restriction to a service, find the denied call in the audit record and tune the rule.
- Decide which service is protected when memory runs out: Lower the database's OOM score, raise the cache service's and show the expected service is killed under memory pressure.
- Observe pressure with a soft memory limit: Put a soft limit on a service, read the slowdown from pressure information and report how it differs from a hard limit.
- Solve a noisy neighbour with limits and early intervention: On Göktaşı Yazılım's build server find the job starving other services from pressure metrics and balance it with slice limits and early OOM handling.
About this course
You set CPU, memory and I/O limits with cgroups v2, isolate services with options such as DynamicUser and ProtectSystem, and tighten them while measuring the effect.
What you will learn
- cgroups v2
- Limits
- Isolation
- Wrap-up
Prerequisites
You should have practised the topics of the same path up to level 301.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 401; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.