FIT2109 Chap.2 Shell Tools and Scripting
Shell Tools and Scripting
Shell Tools and Scripting develops a pipeline from a series of observable stages: discover eligible files, validate records, filter the population, aggregate values and format a stable result.
Assignment 1 grounds the method in a Bash log-analysis tool that accepts command-line arguments, searches recursively for log files, handles invalid use and produces several reports.
The central question is always what one stage consumes and emits. A text search followed by a line counter measures matching records only if the search output contains one line per intended record.
Redirection writes a stream to a file; it does not connect that stream to a second command. Clear contracts make these distinctions easy to test.
What this chapter covers
- 01
Pipeline stages with one observable responsibility each
- 02
Recursive discovery restricted to eligible regular files
- 03
Blank, comment and malformed record policies
- 04
Pipes as producer-to-consumer stream connections
- 05
Redirection as a destination choice for a stream
- 06
Argument validation before data processing begins
- 07
Stable sorting and labelled report sections
- 08
Boundary fixtures for spaces, empty input and invalid paths
Worked example · free
Count exact severity records
- +1Discover only regular files with the required .log suffix under the input directory.
- +1Read records without letting filenames or unrelated file types enter the data stream.
- +1Remove blank and comment lines according to the stated record policy.
- +2Select rows whose severity field equals ERROR rather than lines containing the letters anywhere.
- +1Count selected lines and label the result as valid matching records.
- +1Test with nested files, an empty directory, a filename containing spaces and a message that contains ERROR in a non-severity field.
Key terms
- Pipeline
- A composition where one command's ordinary output becomes another command's input. Each stage should preserve a clear record contract.
- Redirection
- A shell operation that sends a stream to or from a file or another destination. It is not the same as piping into a command.
- Record contract
- The explicit rule for what one valid input row contains, which rows are excluded and what a transformation emits.
- Field match
- A comparison against a defined field rather than an arbitrary substring anywhere in a record. It prevents plausible but semantically wrong counts.
- Stable ordering
- A defined sort and tie-break rule that produces repeatable output for the same logical input.
- Fail fast
- Stopping after invalid usage or a required-stage failure so the program cannot emit a partial result that looks complete.
- Boundary fixture
- A small test input designed around an edge condition such as spaces, emptiness, malformed data or nested paths.
Shell Tools and Scripting FAQ
What is the difference between a pipe and output redirection?
A pipe connects one command's ordinary output to another command's input. Output redirection sends the stream to a file or destination. Using a redirection symbol where a pipe is needed creates a file rather than invoking the intended consumer.
Why can a line count still answer the wrong question?
The counter only measures the stream immediately before it. If selection matches a token anywhere instead of the intended field, or if filenames are included, the syntax can succeed while the population is wrong.
What should happen when a required argument is missing?
The script should detect the omission before scanning, print the relevant usage guidance to the diagnostic stream, return a non-zero status and avoid creating a partial output artifact.
How do I make a report deterministic?
Define file scope, record validation, section calculations, primary sort and tie-break ordering. Run repeated tests on the same fixture and compare the logical or byte output, depending on the report contract.
Assessment move
Start by drawing a four-stage pipeline for each task: discovery, validation, selection or aggregation, and presentation. Build a fixture small enough to calculate by hand, including one blank line, one comment, one malformed record and one valid record whose message contains a distracting token. Stop the pipeline after every stage and inspect its output. Only add the counter after the selected population is visibly correct.
Then practise interface failures: unknown command, missing directory, too few arguments and an unwritable destination. Assert the status, diagnostic and absence of misleading output. For report practice, reconcile section totals against the validated population and enforce stable ordering. Finally, explain why each stage exists and show which test would fail if it were removed.
This mirrors the Assignment 1 rubric's emphasis on reliability, error handling, testing, report quality and a development history that records logical changes. Rebuild the same report from two differently ordered file trees and confirm that the final order follows your explicit rule. Then introduce a failed upstream command and ensure the script cannot turn the missing stream into a plausible zero count.
These drills connect determinism with fail-fast behaviour and make the report safer for another command to consume.
Working through Shell Tools and Scripting in FIT2109? Sia is AskSia’s AI Computer Science tutor — ask any FIT2109 Shell Tools and Scripting question and get a clear, step-by-step explanation grounded in how FIT2109 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
