MM1041 Chap.3 Big Data and the Four Vs
Big Data and the Four Vs
What counts as big
The big data lesson defines big data as a field that treats ways to analyse, systematically extract information from, or otherwise deal with data sets too large or complex for traditional data-processing software. It lists the challenges as capturing, storing, analysing, searching, sharing, transferring, visualising, querying and updating data, together with information privacy and data source.
Size is only part of the story; complexity and speed can defeat ordinary tools just as surely.
Where it comes from
The sources listed are ordinary: posts on social media, photos and video, records that software keeps, recordings from cameras and microphones, sensor readings, scanned government documents, GPS trails from phones, purchase records and traffic.
The lesson's illustration of a single internet minute, with millions of searches, messages and video views, makes the point that most big data is produced as a side effect of everyday activity.
The four Vs
Gartner's definition, quoted in the lesson, treats large, fast and varied data as an asset only when a firm has affordable, new ways to process it that improve insight and decisions.
The IBM version adds veracity. Volume is the scale of data, growing because of huge numbers of devices and sensors. Velocity is both the speed at which data is generated and the speed at which it must be processed, since in online analytics a late decision means a missed opportunity. Variety is the range of forms, from relational tables and text to XML and social network graphs.
Veracity is the uncertainty of data: measurement error, typos, unreliable sources and conflicts between sources.
Structured, semi-structured and unstructured
Variety is often summarised as a spectrum. Structured data fits the rows and columns of a database or spreadsheet. Semi-structured data, such as XML, carries labels but no fixed table.
Unstructured data, such as text, images, audio and video, has no predefined model at all. The lesson's email example shows all three in one message, which is why variety is felt even inside a single application.
Why this matters to a manager
Each V points to a different investment.
Volume calls for distributed storage, velocity for streaming analysis, variety for tools that handle several data forms, and veracity for validation before any model is trained. Naming the V that binds a business problem is the first step toward choosing the right system, which is the subject of the next chapter.
What this chapter covers
- 01
Data sets too large or complex for traditional software
- 02
Everyday sources of big data
- 03
Volume and the scale of data
- 04
Velocity of generation and of processing
- 05
Variety across tables, text, XML and graphs
- 06
Veracity and the trust problem
Worked example · free
Diagnose a retailer's data problems by V
- 3Diagnose the stock complaint.
- 3Diagnose the comments complaint.
Key terms
- Volume
- The scale of data, from terabytes to zettabytes, driven by large numbers of devices and sensors.
- Velocity
- The speed at which data is generated and at which it must be processed to stay useful.
- Variety
- The range of forms data takes, including tables, text, images, audio, XML and graphs.
- Structured Data
- Data organised in fixed rows and columns, such as a database table or spreadsheet.
- Unstructured Data
- Data with no predefined model, such as free text, images, audio and video.
- Semi-structured Data
- Data that carries labels or tags but no fixed table, such as an XML file.
- Streaming Data
- Data that arrives continuously and needs analysis within seconds rather than in batches.
- Data Validation
- Checking records for errors and conflicts before they are used to train a model.
Big Data and the Four Vs FAQ
What are the four Vs of big data?
Volume, velocity, variety and veracity. Volume is the scale of data, velocity is the speed of generation and required processing, variety is the range of data forms, and veracity is the uncertainty in the data. The first three come from the original description of data growth in e-commerce; veracity was added in the IBM version shown in the lesson.
Is a very large spreadsheet big data?
Not necessarily. The course definition turns on whether traditional data-processing software can cope. A large but tidy spreadsheet that ordinary tools handle is simply a large dataset. It becomes big data when its size, speed of arrival, variety of form or uncertainty exceeds what those tools can manage in the time a decision allows.
Why is veracity often called the most dangerous V?
Because the other three multiply the chance of untrustworthy records, and untrustworthy records produce confident wrong answers. Fake reviews, measurement errors and sources that contradict each other all enter a model silently. A system can be fast, large and flexible and still mislead a manager if nobody checked the data first.
Exam move
Draw the four Vs as a two-by-two, with volume and velocity on top and variety and veracity below, and add one example from the lesson and one of your own to each box. Then practise naming the binding V for three short business scenarios until you can justify each choice in one sentence.
Working through Big Data and the Four Vs in MM1041? Sia is AskSia’s AI Information Technology tutor — ask any MM1041 Big Data and the Four Vs question and get a clear, step-by-step explanation grounded in how MM1041 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
