FIT1043 · Introduction to Data Science
Characterising Data & 'Big' Data
Week 9 of Monash FIT1043 Introduction to Data Science defines what makes data 'big' through the V's — Volume, Velocity, Variety, Veracity (and often Value) — and connects them to the growth laws (Moore's, Koomey's, Bell's, Zimmerman's) and the types of metadata. It also introduces using the BASH shell to inspect and pre-process large files before loading them into an analysis tool. The four V's are a confirmed exam short answer, and this shell/big-data material sets up Assignment 2 and Week 10.
What this chapter covers
- 01Big data defined: any attribute (among the V's) that challenges the constraints of a system's capability or a business need
- 02Volume (size of data) and Velocity (frequency/pace of incoming data to process)
- 03Variety (different types of data) and Veracity (accuracy/truthfulness/reliability; uncertainty of data)
- 04Value as a common fifth V; the original 3 V's as the basis for further V's
- 05Growth laws: Moore's Law, Koomey's Law, Bell's Law, Zimmerman's Law
- 06Metadata types: descriptive (title/author), structural (chapters/XML elements/MPEG containers), administrative (version number/archiving date/DRM)
- 07BASH shell for big data: commands like less and grep to read and manipulate large files easily
- 08Privacy: technology makes surveillance easier unless particular measures are put in place
Explain 'big data' and the four V's, and define veracity
- +1Big data = any attribute (among the V's) that challenges the constraints of a system's capability or a business need. Name the four V's: Volume, Velocity, Variety, Veracity (Value is often added as a fifth).
- +1Define each briefly and veracity in full: Volume = the size of the data; Velocity = the frequency/pace at which data arrives and must be processed; Variety = the different types of data; Veracity = how accurate, truthful and reliable the data is (its degree of uncertainty).
Key terms
- Big data
- Any attribute among the V's that challenges the constraints of a system's capability or a business need — not merely 'a large dataset'.
- Volume / Velocity
- Volume = the size of the data; Velocity = the frequency or pace at which new data arrives and must be processed.
- Variety / Veracity
- Variety = the presence of many different types of data; Veracity = how accurate, truthful and reliable the data is (its uncertainty).
- Value (fifth V)
- A commonly added V capturing whether the data yields worthwhile insight; the original three V's are the basis for developing further V's such as Value.
- Metadata types
- Descriptive metadata (title, author), structural metadata (chapters, XML elements, MPEG containers) and administrative metadata (version number, archiving date, DRM).
- Shell (BASH) for big data
- Command-line tools such as less and grep that let you read and manipulate very large files easily before loading a subset into an analysis tool.
Characterising Data & 'Big' Data FAQ
What are the V's of big data?
The four core V's are Volume (size of the data), Velocity (the pace at which it arrives and must be processed), Variety (the many different types of data) and Veracity (how accurate and reliable it is). Value is often added as a fifth. The unit frames big data as any V-attribute that challenges a system's capability or a business need — so 'big' is relative to what your system and task can handle.
What is veracity?
Veracity is how accurate, truthful and reliable the data is — in other words, its degree of uncertainty. Data can be huge (Volume) and fast (Velocity) but still low-veracity if it is noisy, biased or unreliable, which is why it is called out as its own V. This is the definition the sample exam asks for.
What are the three types of metadata?
Descriptive metadata describes content (e.g. a book's title and author), structural metadata describes how content is organised (chapters in a book, elements in XML, containers in MPEG), and administrative metadata supports management (version number, archiving date, digital rights management). Being able to give an example of each is exam-ready.
How is the BASH shell used with big data?
For very large files it is often faster to inspect and filter them at the command line before loading anything into Python or R. Commands like less (page through a file) and grep (find matching lines) let you manipulate large data files easily, and Assignment 2 uses this shell pre-processing step to reduce a large file to a CSV for analysis in R.
Can AI help me with the big-data concepts in FIT1043?
Yes. Sia can drill you on the V's and their definitions, the metadata types with examples, and the growth laws, step by step, and set fresh short-answer practice in the exam's style. It explains the concepts and checks your reasoning; it does not do graded work for you, and Monash academic-integrity rules apply. Confirm details on Moodle.
Exam move
The four V's are the highest-yield item this week and a confirmed exam short answer, so memorise a one-line definition for each (Volume, Velocity, Variety, Veracity) plus Value as the fifth, and the framing that big data is a V that challenges a system or business constraint. Make an example-per-type card for metadata (descriptive/structural/administrative) and be able to name the four growth laws (Moore's, Koomey's, Bell's, Zimmerman's). For the shell material, know that less and grep let you handle large files at the command line — the same pre-processing Assignment 2 requires. Rehearse define-and-give-an-example answers, since that is how the two marks are earned.
Working through Characterising Data & 'Big' Data in FIT1043? Sia is AskSia’s AI Information Technology tutor — ask any FIT1043 Characterising Data & 'Big' Data question and get a clear, step-by-step explanation grounded in how FIT1043 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
