Detection as code: rule repository, tests and coverage
You manage detection rules like software: version control, review, automatic tests, validation with attack simulation and an ATT&CK coverage map that shows the gaps.
Level 501 · 13 lessons · 15 practice labs
Lessons
The rule repository
- Rule repository structure and naming
- Rule review process and quality criteria
- Validating Sigma rules automatically
- Short task: move ten rules into the repository and fix the validation errors
Testing and validation
- Unit tests for rules: expected matches and non-matches
- Validating with attack simulation: Atomic Red Team tests
- Publishing rules to the SIEM automatically with CI
- Short task: build a pipeline that stops a rule release when its tests fail
Coverage and measurement
- Building an ATT&CK coverage map
- Detection metrics: precision, noise, time to detect
- Prioritising gaps against the threat model
- Short task: pick the five riskiest gaps on the coverage map
Capstone
- Capstone: build a detection programme that is tested, published automatically and measured
Practice labs
- Fix the naming errors in the rule repository: Find five rules that break the file name, ID and folder convention and rename them to the standard.
- Repair Sigma rules that fail validation: Clear the errors in sigma check output and show every rule passes validation.
- Catch the quality problems in a rule review: Review the rule in a merge request against the criteria and comment on the missing test and the broad condition.
- Write matching and non-matching test events for a rule: Add two sample events the rule should and should not catch and show the test script passes.
- Build a pipeline that stops release when a rule test fails: Add a test stage to the CI configuration and show the pipeline stops on a deliberately broken rule.
- Validate with attack simulation that a rule really fires: Run the atomic test, show the rule matches the produced log and fix the rule that does not match.
- Add the step that publishes rules to the SIEM automatically: Write a pipeline step that converts a rule merged to main and loads it into the SIEM, recording the version tag.
- Build an ATT&CK coverage map from rule tags: Collect the tags of the rules in the repository and tabulate the number of covered techniques per tactic.
- Calculate noise and precision per rule: Calculate each rule’s true alert rate from a month of alert closure data and list the worst three.
- Pick the five riskiest gaps on the coverage map: Compare the techniques in the threat model with existing rules and rank the gaps by likelihood and impact.
- Find the technique that looks covered but lacks data: Find techniques that have a rule but no connected log source and correct the map to reflect reality.
- Catch a malformed rule with the validator: Run sigma check on a repository of five rules, find the two with invalid fields and fix them.
- Open a branch and request review for a rule repository change: Add a new rule on a separate branch, write a clear commit message and show the test evidence in the merge request.
- List the log sources your rules require: Collect the logsource fields in the rule repository and count the rules per source.
- Compare rule sources with the data that really arrives: Match required sources against those with records in the SIEM over the last seven days and flag rules that have no data.
About this course
You manage detection rules like software: version control, review, automatic tests, validation with attack simulation and an ATT&CK coverage map that shows the gaps.
What you will learn
- The rule repository
- Testing and validation
- Coverage and measurement
- Capstone
Prerequisites
You should be experienced in the topics of the same path up to level 401.
Lab environment
Lessons run in your own Kali Linux terminal in the browser, connected to a private lab network. Nothing to install.
Frequently asked questions
- Is this course right for my level?
- The course is at level 501; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.