Filtering and counting: grep, sort, uniq, wc, cut
You filter a web server’s access log with grep’s options, split it into fields with cut, rank the top requesters with sort and uniq -c, sort a CSV by a column, show the context of errors, search configs for passwords and transform text with tr; at the end you work out yourself the address attacking the login page and the five busiest addresses. The log differs in every lab.
Level 101 · 50 min
Northwind’s website slowed down last night. You have an access log of a few hundred lines; Daniel asks: "Who made so many requests, what did they ask for, how many failed?"
Linux’s power comes from joining small tools with pipes: grep filters, cut cuts fields, sort sorts, uniq -c counts repeats, wc counts, tr transforms. Each does one job, but together they form a data analysis line. The figure shows the typical "top talkers" pipeline.
The log line format: IP - - [time] "METHOD PATH HTTP/1.1" STATUS SIZE "-" "BROWSER". Fields are separated by spaces: field 1 is the IP, field 9 the status code.
What you will do in this lesson
- How many requests weren’t found? grep -c
- grep’s options: -v, -o, -E
- Cut out fields: cut
- The top talkers
- Sort by a column: sort -t -k
- Around the error: -A, -B, -C
- Search a whole folder: grep -r
- Transform: tr
- The login page attacker
- The top five
- Short quiz
Skills you will earn
- Filtering and counting lines (grep, grep -c)
- grep options (-v -o -E -i -w)
- Cutting fields (cut -d -f)
- Sorting and counting (sort, uniq -c)
- Finding the top requesters in a log
- Showing a match’s context (grep -A -B -C)
- Searching a whole folder (grep -r)
- Transforming characters (tr)
- Writing a short, precise finding
← Redirection, stdout and stderr, and pipes · Linux fundamentals · Exit status and chaining commands →
Start for free