DATA4207 · Data Analysis in the Social Sciences
Communication Analyses: Content, Networks and Text as Data
Week 12 introduces methods for analysing communication and unstructured data: content analysis of media, social network analysis to map relationships and information flow, feature selection to keep the most relevant variables, and text-as-data with tidytext. The emphasis is how these blend qualitative and quantitative insight from large digital datasets — turning text and ties into something you can model, and choosing which variables actually matter.
What this chapter covers
- 01Content analysis: systematic examination of communication patterns to track opinion and values over time (tidytext)
- 02Text-as-data: tokenise → bag-of-words / document-term matrix → count or tf-idf → analyse
- 03Social network analysis (SNA): nodes and edges; degree, centrality, density, communities
- 04How SNA reveals group dynamics and information flow between actors
- 05Feature selection: identifying the most relevant variables to reduce complexity while keeping information
- 06Feature-selection method families: correlation, mutual information (mRMR), LASSO/ridge/elastic net, random-forest importance, stepwise
- 07Aggregating and ranking variable importance (mean/median/Borda rank aggregation)
- 08Combining qualitative and quantitative insight from communication data
From text to a document-term matrix
- +1Tokenise each document and fix a shared vocabulary in order: {policy, jobs, tax}. Every document becomes a vector indexed by this vocabulary.
- +1Count term frequencies: Document A = (policy 1, jobs 2, tax 0) → (1, 2, 0); Document B = (policy 1, jobs 0, tax 2) → (1, 0, 2). Stacking these rows gives the document-term matrix.
- +1Now each document is a numeric vector, so you can compute similarity (e.g. cosine similarity), weight terms by tf-idf to down-weight ubiquitous words, or feed the matrix into a model — this is what 'text as data' means.
- +1Feature selection: with many terms/variables, keep only the most informative ones — via LASSO, random-forest (permutation) importance, or a mutual-information method such as mRMR — to cut redundancy while retaining signal.
Key terms
- Content analysis
- The systematic examination of communication patterns in media or text to track changes in public opinion and values over time, done in R via tidy text mining (tidytext).
- Bag-of-words / document-term matrix
- A representation of text as vectors of term counts over a shared vocabulary (word order discarded). Rows are documents, columns are terms; the values can be raw counts or tf-idf weights.
- Social network analysis (SNA)
- A framework for mapping social structures as nodes (actors) and edges (ties), using measures like degree, centrality, density and communities to reveal group dynamics and information flow.
- Centrality
- A family of SNA measures (degree, betweenness, closeness, eigenvector) capturing how important or well-connected a node is within the network.
- Feature selection
- Choosing the most relevant variables to reduce complexity while keeping essential information, using correlation, mutual information (mRMR), LASSO/ridge/elastic net, random-forest importance or stepwise methods.
- tf-idf
- Term frequency-inverse document frequency, a weighting that boosts terms distinctive to a document and down-weights terms common across all documents, sharpening a document-term matrix for analysis.
Communication Analyses: Content, Networks and Text as Data FAQ
How does text become something I can analyse statistically?
You tokenise the text, fix a shared vocabulary, and count how often each term appears in each document, producing a document-term matrix — each document is now a numeric vector. From there you can measure similarity, apply tf-idf weighting, cluster documents, or model the matrix. The unit keeps this at a conceptual level with tidytext, but the core idea is that text is data once it is vectorised.
What does social network analysis actually measure?
It represents actors as nodes and their relationships as edges, then quantifies structure: how connected each node is (degree), how central or influential it is (betweenness, closeness, eigenvector centrality), how dense the network is, and which clusters or communities form. This reveals group dynamics and how information flows between people or organisations.
Why do I need feature selection?
Because real datasets have many variables, most of which add noise or redundancy. Feature selection keeps the most informative ones — via LASSO, random-forest importance, mutual information and similar methods — which reduces overfitting, improves interpretability, and makes large social datasets manageable. Importance rankings can be aggregated across methods for a more robust choice.
Can AI help me with text and network analysis in DATA4207?
Yes, as a study aid. Sia can explain content analysis, the bag-of-words and tf-idf ideas, SNA measures, and feature-selection methods, and walk you through a small example step by step. It teaches the method and checks your reasoning; it does not do graded work, and University of Sydney academic-integrity rules apply — confirm on Canvas whether AI is permitted for a task.
Assessment move
Keep this week conceptual but concrete. For text-as-data, build a tiny document-term matrix by hand so the 'text becomes a vector' idea is solid, then note how tf-idf and cosine similarity extend it. For SNA, learn the vocabulary — nodes, edges, degree, centrality, density, communities — and what each reveals about information flow. For feature selection, be able to name a few method families (LASSO, random-forest importance, mutual information/mRMR) and say why trimming variables helps. These methods rarely form a whole assessment on their own, but they widen the toolkit you can justify in the individual report, so link each to a question it could answer. Confirm what is assessable on Canvas.
Working through Communication Analyses: Content, Networks and Text as Data in DATA4207? Sia is AskSia’s AI Statistics tutor — ask any DATA4207 Communication Analyses: Content, Networks and Text as Data question and get a clear, step-by-step explanation grounded in how DATA4207 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
