Writing your own SAST rules: Semgrep and data flow
You write Semgrep rules that catch your organisation’s own mistakes, trace injections with source-to-sink data flow analysis and protect your rules with tests.
Level 301 · 18 lessons · 15 practice labs
Lessons
Rule-writing basics
- Rule anatomy: id, pattern, message, severity
- Patterns and metavariables
- pattern-either, pattern-not and pattern-inside
- Trying a rule on small samples
- Short task: write a rule that catches use of a dangerous internal function
Data flow (taint)
- Sources, sinks and sanitisers
- Tracing SQL injection with taint mode
- Propagators and flow across functions
- Reducing false alarms by defining sanitisers
- Short task: write a taint rule for SSRF
Rule quality
- Rule tests: ruleid and ok annotations
- Adding an autofix suggestion
- CWE, OWASP and confidence metadata
- A rule repository and rule tests in CI
- Short task: build and test a five-rule pack
Organisation-wide use
- From incident to rule: never repeating the same mistake
- Measuring rule success: hits and noise
- Capstone: a security ruleset for an in-house framework
Practice labs
- Read the parts of a Semgrep rule: Find the id, pattern, message and severity fields in a ready rule and state in one sentence what it catches.
- Write a rule that catches use of a dangerous internal function: Use a metavariable to write a rule that finds every call to the organisation’s legacy encryption function and try it on a sample file.
- Narrow a rule with pattern-inside and pattern-not: Write a rule catching file opens only inside request handlers and without a validation call, and test it with three positive and three negative samples.
- Mark source, sink and sanitiser in sample code: Point out where user input enters, the dangerous call and the function that sanitises in between.
- Write a taint rule for SSRF: Write a rule catching data flowing from a request parameter to the HTTP client’s URL argument and define the allowlist check as a sanitiser.
- Follow flow across functions with propagators: Extend a rule with propagators to catch SQL injection where input reaches the query through a helper function and an object field, and cut false alarms with a sanitiser.
- Add ruleid and ok annotations for a rule test: Annotate the lines that must and must not match in a sample file and run the test.
- Add an autofix and metadata to a rule: Write a fix that swaps a weak hash function for a safe one and add CWE, OWASP and confidence fields to the rule.
- Build a five-rule pack and test it in CI: Gather the rules with their tests in a repository, write a pipeline step that runs the tests on every change and show a broken rule blocks the merge.
- Extract from an incident report the pattern to turn into a rule: Read a short incident report and describe in two lines the distinctive shape of the flaw in code.
- Write the rule that catches the same mistake as the incident: Write a rule catching an admin function called without an authorisation check and find how many other places it occurs in the repository.
- Measure and improve a rule’s hit rate and noise: Run the rule on five repositories, label findings true or false, calculate precision and make the change that raises it.
- List the security-relevant functions of an in-house framework: Extract from the framework’s documentation the functions that take input, run queries and check authorisation.
- Write the source and sink definitions for the framework: Prepare a shared taint rule defining the framework’s request object as source and its raw query and template functions as sinks.
- Publish a security ruleset for the in-house framework: Prepare a tested, documented six-rule set with metadata for Göktaşı Yazılım’s framework and present it with measured results on a sample app.
About this course
You write Semgrep rules that catch your organisation’s own mistakes, trace injections with source-to-sink data flow analysis and protect your rules with tests.
What you will learn
- Rule-writing basics
- Data flow (taint)
- Rule quality
- Organisation-wide use
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.