FIT2094 Chap.8 NoSQL, Warehousing and Responsible Data
NoSQL, Warehousing and Responsible Data
Later database topics compare relational and non-relational structures, operational and decision-support data, and technical possibility with responsible use. Operational databases protect current transactions and support controlled updates. Document databases such as MongoDB can store bounded aggregate-shaped records with flexible nested fields.
Embedding works best when values share ownership, lifecycle and access; referencing helps when facts change independently or have their own identity.
Data warehouses integrate historical data for analysis and business intelligence. Integration requires stable identifiers, shared definitions, time handling and lineage back to operational sources. Useful analysis also requires privacy, legal and ethical appraisal.
Data minimisation, purpose, access, retention, fairness and explainability shape what should be stored or linked, not just how efficiently it can be queried.
Evaluate an architecture through a decision record. Name the operational event, the ownership boundary, the dominant reads and updates, the consistency rules and the growth limit.
If embedding is proposed, test what happens when a nested fact changes independently or the array becomes large. For a warehouse measure, define its numerator, denominator, time grain and lineage to validated operational events. Then review whether linkage creates a new privacy risk, whether the purpose justifies every attribute and who may see the result.
A conditional recommendation is strongest: state the evidence that supports the current choice and the workload or governance change that would trigger redesign.
What this chapter covers
- 01
Relational and non-relational model trade-offs
- 02
Document embedding, referencing and aggregate boundaries
- 03
Operational data and decision-support data
- 04
Data warehouses, integrated history and lineage
- 05
Business intelligence measures and interpretation
- 06
Privacy, access control and data minimisation
- 07
Legal and ethical evaluation of database use
Worked example · free
Choose storage for equipment inspections
- +2Keep assets, work orders and technicians in the relational system because their keys, references and state transitions require strong cross-entity consistency.
- +2Store one bounded inspection response as a document when its nested readings are created and retrieved together, while retaining stable asset and inspection identifiers.
- +2Feed validated events into a warehouse with conformed site and date definitions so historical measures remain comparable and traceable.
- +1Exclude personal free text not needed for analysis, restrict analytical access and document the transformation from operational events to each dashboard measure.
- +1State a review condition: reconsider embedding if readings grow without a bound or require independent updates and cross-inspection queries.
Key terms
- Document database
- A non-relational database that stores records as flexible documents, often with nested values representing one aggregate.
- Embedding
- Storing related nested values inside a parent document when they share ownership, lifecycle and common access.
- Referencing
- Linking separately stored records through identifiers when they have independent identity, change or reuse.
- Operational data
- Current detailed data used to execute and control day-to-day transactions.
- Decision-support data
- Integrated and often historical data organised for analysis, comparison and organisational decisions.
- Data lineage
- A trace of where data originated and how it was transformed into a derived field, table or measure.
- Data minimisation
- The practice of collecting and retaining only information necessary for a defined and legitimate purpose.
NoSQL, Warehousing and Responsible Data FAQ
When is embedding a good document-model choice?
Embed when nested values belong to one parent, share its lifecycle, remain bounded and are usually created or read together. Reference when the related record changes independently, is reused or needs separate access and identity.
How does a warehouse differ from an operational database?
An operational database supports current controlled transactions and frequent updates. A warehouse integrates historical data under shared definitions for analysis. It prioritises consistent measures, lineage and scans across time rather than individual transaction speed.
What makes a business-intelligence measure trustworthy?
Define the event and denominator, reconcile source identifiers, document transformations, test missing or duplicated data and explain limitations. A chart can be visually clear while the measure is biased or based on inconsistent definitions.
What should a responsible-data answer include?
State the purpose, necessary attributes, authorised access, retention period, linkage risk and possible unfair or misleading uses. Explain who could be harmed and what control or review would reduce that risk.
Assessment move
Use decision tables instead of product slogans. Compare relational and document models on identity, constraints, access pattern, update pattern and growth. For each proposed document, mark which fields belong to the aggregate and which change independently. Sketch the flow from an operational event into a warehouse measure and label every transformation or shared definition.
Review an analytical question for purpose, data minimisation, access, retention, linkage quality and fairness. Finish architecture practice with a conditional recommendation that names the evidence that would trigger reconsideration. Keep detailed NoSQL command practice tied to the current task instructions because the Week 11 teaching page is unavailable in the materials reviewed.
Add one adverse scenario involving unbounded growth, false linkage or harmful interpretation and state the technical and governance control that would detect or limit it.
