File formats with Python: CSV, JSON, YAML, XML, SQLite
You read, write and convert the data formats you meet at work and handle archives and binary files safely.
Level 201 · 17 lessons · 21 practice labs
Lessons
Text-based formats
- CSV dialects, headers and DictReader
- CSV formula injection and safe export
- JSON: custom encoders, dates and decimals
- Validating JSON data against a schema
- YAML, TOML and INI configuration files
- Short task: migrate an INI configuration to TOML
XML and spreadsheets
- Reading and writing XML with ElementTree
- Safe XML parsing: external entities and defusedxml
- Excel files with openpyxl
- Short task: dump an XML inventory to CSV
Databases, archives and binary files
- An embedded database with sqlite3
- Archives with zipfile and tarfile
- Path traversal when extracting archives (Zip Slip) and how to prevent it
- Compression: gzip, bz2 and lzma
- Reading binary records with struct
- Short task: index the contents of backup archives into SQLite
Capstone
- Capstone: a safe converter between data formats
Practice labs
- A format converter driven by a settings file: Complete a tool that follows a TOML description to convert gzipped CSV to JSON Lines row by row, casting types and writing bad rows to a separate file.
- Reading a semicolon CSV: Read a sales file with semicolon delimiters and comma decimals with DictReader and compute the total amount.
- Handling missing headers and extra columns: Fit a CSV with varying header names and some rows with extra columns to one schema and report the bad rows separately.
- Blocking formula injection on export: Neutralise cells starting with =, +, - or @ in a CSV export of user input and pass the tests.
- Writing a record with dates and Decimals to JSON: Write a custom JSON encoder that handles datetime and Decimal fields and test that values survive a round trip.
- Validating order data against a schema: Write a JSON schema for order records and list which field was rejected and why for bad records.
- Reading TOML settings in layers: Write a loader that merges default, environment and local TOML files in order, deep-merging nested tables.
- Loading YAML safely and checking types: Move a settings reader from yaml.load to safe_load and catch values such as "no" turning into the wrong type.
- Getting totals from XML invoices: Read line amounts from a namespaced invoice XML with ElementTree and compute the total.
- Closing an XML reader to external entities: Switch a function that reads outside XML to defusedxml and test that a sample file with entities is rejected.
- Summarising an Excel attendance file: Read a multi-sheet attendance workbook with merged cells using openpyxl and write absences per person to a new sheet.
- Importing a CSV into a SQLite table: Load a product CSV into a SQLite table with executemany and query the number of products per category.
- Extracting an archive without leaving the target: Write a safe extract function that rejects tar members with ../ or absolute paths.
- Counting a compressed log without unpacking: Read a gzip file line by line without unpacking to disk and count error lines, giving a clear error for a corrupt archive.
- Decoding binary sensor records with struct: Read fixed-size little-endian sensor records with struct.iter_unpack and report a truncated final record as an error.
- Updating a zip archive with verification: Replace a single member of a zip archive without extracting to disk, run the CRC check on every member and reject archives whose declared total uncompressed size exceeds the limit.
- Reading a TOML settings file: Read a settings file with tomllib and write a function that returns the server name, port number and tag list inside a table with their correct types.
- Processing a large XML file as a stream: Read an XML file with hundreds of thousands of records using iterparse and clear processed elements; tests check both the totals and a memory limit.
- Writing records to an Excel file: Produce a sheet with a header row from a list of records using openpyxl, writing dates as date cells and amounts as number cells so the types survive a re-read.
- Packing a fixed header with struct: Produce an eight-byte header made of a version number, a record count and flags with struct.pack and read the same values back with unpack.
- Reading an audio file header with struct: Extract the channel count, sample rate and bit depth from a WAV file header with the correct byte order, rejecting files with a wrong signature.
About this course
You read, write and convert the data formats you meet at work and handle archives and binary files safely.
What you will learn
- Text-based formats
- XML and spreadsheets
- Databases, archives and binary files
- Capstone
Prerequisites
Finishing the level 101 courses of the same path, or knowing those topics, is enough.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 201; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.