7AAVDH10 Chap.5 Data Work for Humanities Questions
Data Work for Humanities Questions
Build an auditable research chain
The applied syllabus moves through pandas, cleaning, exploration, visualisation, natural language processing, APIs and web scraping. This chapter organises those tools around a humanities question.
It defines the unit of analysis, records inclusion and cleaning rules, validates transformations and keeps denominators beside computed patterns.
Interpret at the supported scale
Code can reveal a pattern in a selected dataset without proving a universal cultural explanation.
The chapter follows known records through the pipeline, checks boundary cases and writes conclusions that name source coverage, method choices and plausible alternatives.
What this chapter covers
- 01
Data scope
- 02
Cleaning rules
- 03
Pandas workflow
- 04
Visualisation checks
- 05
API boundaries
- 06
Qualified interpretation
Worked example · free
Compare unequal collections
- 3Print counts and denominators. The marks shown are a study allocation, not an official University marking scheme.
- 3Calculate comparable proportions.
- 3State source and method limits.
Key terms
- Unit of Analysis
- The unit of analysis is the entity represented by each observation in a study.
- Cleaning Rule
- A cleaning rule states how raw values are transformed, retained or excluded.
- Denominator
- A denominator defines the eligible total against which a count is compared.
Data Work for Humanities Questions FAQ
Why are cleaning choices analytical?
Removing, merging or normalising values changes what the dataset represents. Record the rule, compare row counts before and after, inspect examples and explain why the transformation serves the research question.
What should a visualisation prove?
A chart should accurately display a defined comparison and match the underlying table. It can show a pattern, but interpretation still requires source context, validation and a statement of what the data cannot establish.
How do I handle an unstable result?
Locate the instability in source coverage, cleaning, classification or comparison. Report it directly, test a specific alternative and propose the next evidence needed instead of forcing a stronger conclusion.
How can I audit a chart without trusting its appearance?
Choose one bar, point or category total and reproduce it from printed source rows. Confirm the filter, group, numerator, denominator and label. Follow a known record through cleaning to the plotted value and include a known non-match that should stay outside the category. If the manual result disagrees, repair the earliest transformation rather than changing the visual.
A plausible shape is not validation; agreement between trace, table and chart is.
Exam move
Create a method ledger with research question, unit of analysis, source boundary, inclusion rule, transformation, validation output and permitted claim. Fill it before selecting a package. Choose one known record and follow it from raw source through cleaning, analysis and visualisation. Confirm that its identifying fields and category survive as intended. Add a known non-match, a missing case and an ambiguous boundary case.
State expected outcomes before running the pipeline so validation cannot be rewritten around the observed result. Print row counts before and after every filter, then inspect excluded examples. When comparing groups of different sizes, display numerator and denominator together and calculate a justified rate rather than relying on raw counts.
Reproduce one plotted bar or point manually from printed rows and verify its label, grouping and value. For text processing, document how tokens, case, punctuation and categories are treated. For APIs or web scraping, record retrieval scope and test a record known to exist so an empty response cannot be mistaken for cultural absence.
Maintain separate notes for source coverage, cleaning, classification, computation and interpretation. If a result changes, locate the earliest stage whose output differs instead of adjusting the final chart. Compare the computational finding with a plausible alternative explanation and name the evidence that would separate them.
In project writing, use language that matches the source scale: this catalogue shows, these sampled texts contain, or these captured records indicate. Close each session by rerunning the pipeline from a fresh state and comparing outputs with the notebook narrative. A reproducible weak or negative result is acceptable when the design, checks and limitations are explicit; an unsupported broad claim is not.
Add a weekly audit that begins from the final visual rather than the raw data. Select one bar, point or category total and trace it backward to the grouped table, cleaned rows and original records. Recalculate that value by hand and compare the denominator with the figure label. Next, remove one cleaning rule temporarily and record which outputs change.
Explain whether the difference reveals noise, bias or a meaningful alternative representation. Finish by writing two conclusions: one that overstates the result and one that matches the documented scope. Underline the words that make the second version more defensible and list the additional evidence required before the stronger version could be considered.