41297 Chap.2 Data Sources, Reliability and Dataset Quality
Data Sources, Reliability and Dataset Quality
Where data come from
The second lecture moves from describing variables to finding them. It contrasts primary data, collected for your own question, with secondary data gathered by others, and points to open repositories and to government, international, healthcare and sports collections.
Each source delivers files in its own format, so preparation is never optional.
Two tests and six dimensions
A measure can be consistent without capturing the right idea, so the chapter separates reliability from validity with business examples.
It then applies the six quality dimensions on the topic selection form: accuracy, completeness, consistency, timeliness, believability and interpretability.
What shape the data take
Finally it compares a dataset with a database and sets out record, matrix, transaction, document, graph and ordered data, explaining why a tidy data matrix is the quickest route to the summary measures of later chapters.
What this chapter covers
- 01
Primary and secondary data collection
- 02
Open repositories and sector data sources
- 03
Reliability versus validity
- 04
Datasets compared with databases
- 05
Record, transaction, document, graph and ordered data
- 06
Six dimensions of dataset quality
Worked example · free
Test three business measures for reliability and validity
- 2Card scans: reliable, because the reader logs every entry the same way, and valid, because a scan is a visit.
- 2Likes: reliable as a count, but not valid for satisfaction, since non-members and casual scrollers also like posts.
- 2Queue guesses: aimed at the right idea, so potentially valid, but unreliable, because two receptionists would estimate differently.
Key terms
- Primary data
- Data collected first-hand by the analyst for the question under study, for example through a survey.
- Secondary data
- Data originally gathered by another party, often for a different purpose, and reused for a new analysis.
- Data matrix
- A table of m rows, one per object, and n columns, one per numeric attribute.
- Transaction data
- Records in which each transaction is a set of items, such as the products in one shopping basket.
- Timeliness
- The quality dimension that asks whether data are recent enough to describe the situation being analysed.
Data Sources, Reliability and Dataset Quality FAQ
Where can I find a dataset for the seminar and assessment work?
The lecture points to large open repositories such as Mendeley Data, IEEE DataPort, Kaggle and Google's dataset search, and to government, economic, international, healthcare and sports collections. You may also choose a dataset from your own interests, provided it passes the quality and size checks the first seminar sets.
Can a measure be reliable but not valid?
Yes, and it is common. A tool can record something with perfect consistency while recording the wrong thing for your purpose, such as counting website visits and calling the count loyalty. Reliability concerns repeatable recording; validity concerns whether the figure stands for the concept you claim to study.
Is a Kaggle file a database?
Usually not. A downloaded file is a dataset: a fixed collection assembled for some analysis and opened in Excel, R, Python or SPSS. A database is an organised system that keeps records current and is normally queried with SQL, like a company's live order system.
What do the six quality dimensions on the topic selection form mean?
Accuracy asks whether values are correct, completeness whether any are missing, consistency whether units and formats agree, timeliness whether the data are recent enough, believability whether the source is credible, and interpretability whether each column's meaning is clear. Support each verdict with evidence from the file.
Does missing data mean I should reject a dataset?
Not necessarily. Note how many values are missing and in which columns, then judge whether the gap damages your research questions. The Week 6 lecture introduces data pre-processing, including the handling of missing values, so a few gaps can often be repaired rather than avoided.
Assessment move
Take one candidate dataset and write a single page about it before you commit. Record where it came from, whether it is primary or secondary, its date of collection and its shape: a matrix, transactions, documents or something else. Then argue reliability and validity for the two variables your research questions lean on most, and give each of the six quality dimensions one sentence of evidence.
If more than one dimension comes out weak, try a second dataset and compare the two pages side by side; the stronger page becomes the justification on your topic selection form.
Working through Data Sources, Reliability and Dataset Quality in 41297? Sia is AskSia’s AI Statistics tutor — ask any 41297 Data Sources, Reliability and Dataset Quality question and get a clear, step-by-step explanation grounded in how 41297 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.