Bash internals: parsing, performance and portability
You learn in depth how the shell parses and runs a line, what subprocesses cost, how to stay portable to POSIX sh and which subtleties lead to security bugs.
Level 501 · 17 lessons · 15 practice labs
Lessons
Parsing
- Tokenising and parsing: how the shell reads a line
- Word splitting and the hidden effects of IFS
- eval, indirect expansion and namerefs
- The risk of code execution in arithmetic context
- Short task: predict and prove the output of five surprising lines
The process model
- fork, exec and the cost of a subshell
- Builtins versus external commands: the speed difference
- Lost variables in pipelines and lastpipe
- File descriptors: opening, duplicating and closing with exec
- coproc and named pipes
- Short task: measure a slow script and make it ten times faster
Portability
- POSIX sh versus Bash, and bashisms
- Testing with dash and busybox sh
- Differences between Bash versions
- Short task: port a Bash script to POSIX sh so it runs on Alpine
Security subtleties
- Importing functions from the environment and lessons from past flaws
- Capstone: audit a critical script for parsing, performance and portability
Practice labs
- Predict and prove surprising output: Write the output of five short commands before running them, then run them and explain any misses by expansion order.
- Fix parsing broken by IFS: Find the bug where IFS changed inside a function breaks the rest of the script and fix it with local scope.
- Replace eval with a nameref: Rewrite a script that builds variable names with eval safely using namerefs and indirect expansion.
- Find which startup file sets a risky variable: Trace the files read by a login shell and a non-interactive shell to find the one that prepends a writable directory to PATH and remove the line.
- Detect a function imported from the environment: Find the function imported through an environment variable that changes a script's behaviour and wrap the script to run with a clean environment.
- Close code execution in an arithmetic context: Show that a script using user input in an arithmetic expression can be made to run commands, and close it with validation.
- Remove needless subshells from a loop: Rewrite a loop that calls external commands on every iteration with builtins and parameter expansion and measure the speed-up.
- Measure a slow script and make it ten times faster: Find the bottleneck in a script that processes a big file line by line with external commands and produce the same output ten times faster.
- Audit a critical script for parsing, speed and portability: Find one parsing bug, one performance problem and one portability problem in a backup script, fix them and write up the findings.
- Keep a variable lost in a pipeline with lastpipe: Fix a script where a variable assigned in the last command of a pipeline is lost, using lastpipe or restructuring.
- Make two scripts talk through a named pipe: Connect producer and consumer scripts through a named pipe and show no data is lost when the consumer slows down.
- Redirect file descriptors with exec: Redirect all of a script's output to a log with exec, keep the original stdout on another descriptor and print only the summary to the screen.
- Find the bashisms in a script: Find the Bash-only constructs in a script that fails under sh and list them with line numbers.
- Port a Bash script to POSIX sh: Convert a script using arrays and [[ ]] to POSIX sh and confirm it gives the same output under dash and busybox sh.
- Make a script work on an old Bash version: Rework a script relying on new-version features so it also runs on a server with old Bash, using a version check and fallback.
About this course
You learn in depth how the shell parses and runs a line, what subprocesses cost, how to stay portable to POSIX sh and which subtleties lead to security bugs.
What you will learn
- Parsing
- The process model
- Portability
- Security subtleties
Prerequisites
You should be experienced in the topics of the same path up to level 401.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 501; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.