Text processing and regular expressions with Python
You reliably parse, transform and validate logs, reports and free text with string methods, regular expressions and Unicode.
Level 201 · 17 lessons · 15 practice labs
Lessons
String tools
- String methods: split, partition, strip, translate
- Alignment, number formats and tables with f-strings
- Unicode normalisation and Turkish case rules
- string.Template, textwrap and difflib
- Short task: turn a fixed-width report into a table
Regular expressions
- Regex syntax: character classes and quantifiers
- Groups, named groups and backreferences
- Extracting and replacing with findall, finditer and sub
- Flags and readable VERBOSE patterns
- Lookahead and lookbehind
- Slow patterns: backtracking and ReDoS
- Short task: extract IP, path and status code from an access log
Parsing and validation
- Validating formats: e-mail, IP, date
- Address and network work with the ipaddress module
- Writing a simple tokenizer
- Short task: mask passwords and keys in text
Capstone
- Capstone: a tool that converts logs of different formats into one
Practice labs
- Redacting secret values in a log: Write a function that redacts tokens, card numbers and e-mail addresses in log lines without touching innocent numbers such as order ids.
- Aligning receipt lines: Write a line formatter that sets product, quantity and amount into columns with f-string alignment and thousands separators.
- Splitting header lines with partition: Turn "Key: value" lines into a dictionary with partition and strip, keeping colons inside the value.
- Showing a config diff with difflib: Produce a unified diff of two config texts with difflib and return a list summarising only the changed keys.
- Upper-casing Turkish i and ı correctly: Write a function that upper-cases city names by Turkish rules so that "izmir" becomes "İZMİR".
- Comparing look-alike names after normalising: Write a function that compares customer names that fail to match because of different Unicode forms using NFC and casefold.
- Making an ASCII file name from a Turkish title: Produce a safe file name by mapping Turkish letters with translate and normalising accents, erroring on an empty result.
- Pulling order codes out of text: Find order codes shaped like "SP-2024-00173" in an e-mail body with findall and drop duplicates.
- Parsing a log line with named groups: Write a VERBOSE pattern that splits time, host, process and message with named groups, returning None for lines that do not fit.
- Converting date formats with sub: Convert day.month.year dates in text to year-month-day with backreferences, leaving invalid months alone.
- Fixing a pattern that slows down: Rewrite a pattern with nested quantifiers that hangs for seconds on long input and pass the time-limited test.
- Validating an account number format: Write a function that validates an account number's shape by country code, length and character rules with fullmatch.
- Checking whether an IP is in an allowed network: Write a function with the ipaddress module that tells whether an address falls in any allowed CIDR, supporting IPv6 too.
- Hiding the password in a connection URL: Replace the user:password part of connection URLs in log lines with asterisks using a lookbehind.
- Writing a tokenizer for a search filter: Split filter expressions like "status=open AND priority>2" into tokens and raise an error with the position on unknown characters.
About this course
You reliably parse, transform and validate logs, reports and free text with string methods, regular expressions and Unicode.
What you will learn
- String tools
- Regular expressions
- Parsing and validation
- Capstone
Prerequisites
Finishing the level 101 courses of the same path, or knowing those topics, is enough.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 201; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.