The command-line toolbox: ripgrep, fd, fzf and parallel
You meet fast modern counterparts of the classic tools, bulk work with GNU parallel and small helpers such as watch and entr, and build your own toolbox.
Level 301 · 18 lessons · 21 practice labs
Lessons
Searching and finding
- Fast code and log searching with ripgrep
- Finding files with fd and how it differs from find
- Interactive picking with fzf: history, files and processes
- Index-based search with plocate and keeping the index fresh
- Short task: find forgotten keys in a code repository within minutes
Bulk and parallel work
- GNU parallel: input sources and placeholders
- Limits, progress and retries in parallel jobs
- watch and entr: re-run when something changes
- timeout and retry loops
- Short task: hash ten thousand files in parallel and compare
Viewing and comparing
- bat, less and syntax highlighting
- Coloured and word-level diffs: diff, wdiff, colordiff
- Progress and speed in a pipeline with pv
- Readable tables with column and numfmt
- Short task: turn raw output into a readable table for a manager
Building your toolbox
- Versioning your personal dotfiles
- Doing the same job with classic tools on a server that lacks them
- Capstone: review a large incident folder with your new toolbox
Practice labs
- Find forgotten keys in a repo with ripgrep: Use ripgrep to find lines matching an API key pattern in the code repository and report their file paths.
- Find recently changed scripts with fd: Use fd to list shell scripts changed in the last two days, including hidden and ignored files.
- Find a file missed by a stale index: Work out why plocate misses a new file, refresh the index and submit the file's path.
- Review changed scripts with highlighting: Review the recently changed scripts with bat showing line numbers and git change marks, and find the suspicious added line.
- Show only the relevant lines of a long file, highlighted: Show the line range named in the error message with bat, numbered and highlighted, choose the syntax by hand for a file with no extension and fix the faulty line.
- Wire bat in as the viewer for manuals and diffs: Set the environment so man pages and git diff output open coloured through bat, confirm it falls back to plain output in a pipeline and add the settings to the shell profile permanently.
- Cut off a hanging command after ten seconds: Limit a command that tries to reach an unresponsive server to ten seconds with timeout and write down the exit code returned when the time runs out.
- Put a time limit on a hanging command: Write a script that wraps an occasionally hanging health check in timeout and a bounded retry loop.
- End a signal-ignoring command in stages: End a backup command that ignores TERM first gently and then by force using timeout's signal and --kill-after options; write a wrapper that tells exit codes 124 and 137 apart and logs them.
- Pick one file out of hundreds with fzf: Feed the list of files in the log directory to fzf, pick the requested log by typing part of its name and write the chosen path to the answer file.
- Find the right config with an fzf preview: Browse the config files in fzf with a content preview, find the one listening on a given port from the preview and write the one-liner that opens the selection in an editor.
- Write an fzf shortcut to pick and kill a process: Write a small shell function that lists processes in fzf and gracefully terminates the one you pick.
- Hash thousands of files in parallel: Compute SHA-256 hashes of every file in the archive directory with GNU parallel and gather them in one file.
- Retry jobs that fail now and then: Run conversion jobs that sometimes fail with parallel using limited concurrency and retries, and list those that still fail.
- Re-run a check automatically on change: Set up entr so the syntax check runs automatically every time the config file is saved, logging the result.
- Find word-level changes between two configs: Use a word-level diff to find the changed values between two configs with long lines and report them.
- Turn raw output into a readable table: Turn disk usage output into an aligned table with human-readable sizes using column and numfmt.
- Show the speed and ETA of a long transfer: Monitor the compression of a large archive with pv, apply a rate limit and submit the estimated finish time.
- Put your dotfiles under version control: Collect your shell, editor and tmux settings in a git repository and link them into place with symlinks.
- Do the job with classic tools when modern ones are missing: On a server without ripgrep and fd, do the same search with find, grep and xargs and confirm the results match.
- Sweep a big incident folder with the toolbox: Find suspicious IPs, altered binaries and leaked keys in an incident folder of thousands of files and summarise them in one report.
About this course
You meet fast modern counterparts of the classic tools, bulk work with GNU parallel and small helpers such as watch and entr, and build your own toolbox.
What you will learn
- Searching and finding
- Bulk and parallel work
- Viewing and comparing
- Building your toolbox
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.