DATA1001 Chap.2 Exploratory Data Analysis
Exploratory Data Analysis
Exploratory data analysis describes variables before a model or test is chosen. Begin with variable type because it determines which summaries and plots are meaningful. Categorical variables are described by counts and proportions, often with bar charts. Quantitative variables are described by centre, spread and shape, using histograms, boxplots and appropriate numerical summaries.
The mean and standard deviation are sensitive to extreme observations. The median and interquartile range are resistant, so they are preferred for skewed or outlier heavy distributions. A histogram reveals modes, gaps and tail direction. When bins have unequal widths, area rather than height represents proportion.
A boxplot compresses the five number summary and uses fences at the first quartile minus 1.5 times IQR and the third quartile plus 1.5 times IQR to flag potential outliers. Canvas Topic 2 covers data and graphical summaries, while Topic 3 covers numerical summaries.
Together they ask students to choose a legal display, compare groups, interpret plotting code, recognise tidy structure and track how transformations change centre and spread. Adding a constant shifts mean, median and quartiles but does not change SD or IQR. Multiplying by a constant rescales centre and spread, with SD using the absolute multiplier.
A complete comparison names shape, centre, spread and unusual observations in context. Do not call a value unusual merely because it lies beyond a whisker without checking the defined rule, and do not compare groups using centre alone when their variability differs. In R code, read aesthetic mappings first and geometry second to recover the variables and intended comparison.
What this chapter covers
- 01
Topic 2: Data and Graphical Summaries
- 02
Topic 3: Numerical Summaries
- 03
Match variable type to a legal display
- 04
Compare shape, centre, spread and unusual values
- 05
Choose resistant summaries under skew
- 06
Read boxplot, histogram and scatterplot code
Worked example · free
Choose summaries for a skewed distribution
- 1IQR is 28 minus 16, which equals 12 minutes.
- 1The upper fence is 28 plus 1.5 times 12, which equals 46 minutes.
- 1Values above 46 are flagged as potential outliers.
- 1Report median 20 and IQR 12 because the high tail makes resistance valuable.
Key terms
- Mean
- The arithmetic average, a centre measure that responds to every value and is sensitive to extremes.
- Median
- The middle ordered value, a resistant measure of centre.
- Standard deviation
- A measure of typical distance of observations from their mean, expressed in the original units.
- Interquartile range
- The third quartile minus the first quartile, measuring spread of the middle half.
- Resistance
- The property of changing little when a small number of observations become extreme.
- Outlier fence
- A threshold 1.5 interquartile ranges below the first quartile or above the third quartile used to flag unusual values.
Exploratory Data Analysis FAQ
Mean or median on the exam?
Use median when a distribution is skewed or contains influential outliers because it is resistant. Use mean for a roughly symmetric clean distribution when every value should contribute to centre. Explain the choice from the observed shape rather than treating either summary as universally superior.
What must a distribution comparison include?
Compare shape, centre, spread and unusual observations, using the variables and units in the story. If one feature cannot be assessed from the display, say so. Avoid declaring one group better from a single median when spread or skew gives a different practical picture.
How are boxplot outliers found?
Compute IQR as the third quartile minus the first. The lower fence is the first quartile minus 1.5 times IQR and the upper fence is the third quartile plus 1.5 times IQR. Values beyond the fences are flagged, while whiskers extend to observed values within them.
How do transformations change summaries?
Adding a constant moves centre and quartiles by that amount but leaves SD, IQR and range unchanged. Multiplying by a constant rescales centre and spread, with nonnegative spread measures multiplied by the absolute value. Work in the transformed measurement units.
How do I read ggplot code?
Start inside aes and identify each variable mapped to x, y, colour or group. Then read the geometry. A histogram describes one quantitative variable, a boxplot can compare a quantitative response across categories, and a scatterplot examines two quantitative variables.
Exam move
Run a fixed pipeline on each EDA prompt: variable type, legal display, shape, unusual observations, centre, spread and contextual sentence. Practise choosing between mean with SD and median with IQR from a sketch before doing arithmetic. Calculate several outlier fences by hand and check that IQR is nonnegative and measured in the original units. Drill linear transformations by separating shifts from scale changes.
For code questions, hide the plot and predict what the aesthetic mapping and geometry will produce, including which axis contains each variable. Then reverse the task by choosing code for a stated comparison. Build a one page error log for wrong graph types, height versus area in histograms, forgotten units and claims based on centre alone.
End the session with the chapter exam lens and explain why every wrong multiple choice option fails in that particular setting. The aim is not a memorised description template but a precise account of the visible distribution.