INFS3822 Chap.8 Deep Learning for Structured Data
Deep Learning for Structured Data
Neural networks transform inputs through learned weights, biases and activations. This chapter traces one forward pass before scaling to fully connected networks, recurrent networks, LSTMs and transformers. It explains transfer learning and fine-tuning, compares deep learning with traditional baselines, and treats data volume, computation, transparency and deployment as reasons that must justify added depth.
What this chapter covers
- 01
Weights and biases
- 02
Activation functions
- 03
Fully connected networks
- 04
RNN and LSTM sequences
- 05
Transformer attention
- 06
Transfer learning
- 07
Fine-tuning choices
- 08
Deep versus traditional models
Worked example · free
Compute one sigmoid neuron
- +1z = 1.5 x 0.6 + (-0.5) x 0.2 + 0.1 = 0.9 - 0.1 + 0.1 = 0.9.
- +1sigmoid(z) = 1 / (1 + exp(-0.9)) = approximately 0.711.
- +1At threshold 0.70 the case is positive; at 0.75 it is negative. The threshold remains a decision policy.
Key terms
- Weight
- A learned parameter controlling the strength and direction of a connection between units.
- Bias
- A learned additive parameter that shifts a neuron's pre-activation value.
- Activation function
- A transformation applied to a neuron's weighted sum to produce its outgoing signal.
- Fine-tuning
- Updating selected parameters of a pretrained model for a new task or domain.
Deep Learning for Structured Data FAQ
When should I consider a deep network?
Consider it when the data volume and pattern complexity support representation learning that simpler baselines cannot capture adequately. The proposal should also address computation, latency, interpretability and monitoring. Compare against a credible conventional model before claiming that depth adds value.
Why do RNNs and LSTMs suit sequences?
They are designed to carry information across ordered time steps, allowing earlier observations to influence later predictions. Use time-respecting evaluation and compare with lag-based baselines. Sequence architecture is justified by temporal structure, not simply by the presence of many columns.
How does transfer learning help with limited labels?
A pretrained model provides a representation learned from a larger source task. Replace the output for the local task, freeze earlier layers, train the new head and evaluate on held-out local evidence. Fine-tune selectively when validation shows the representation needs adaptation.
What does a one-neuron calculation teach?
It separates inputs, weights, bias, activation and threshold so every stage can be checked. Changing one weighted input alters the pre-activation, while a nonlinear activation means the probability change is not constant. The exercise explains the mechanics of a forward pass without turning a weight sign into a causal claim about the business.
Assessment move
Compute a one-neuron forward pass until weights, bias, activation and threshold are distinct. Make an architecture table linking fully connected networks to tables, recurrent networks to sequences and transformers to attention-based dependencies. For each deep model, name a simpler baseline and the evidence needed to justify complexity. Begin with arithmetic.
Write the weighted sum, substitute every input and weight, add the bias, apply the activation and only then compare the output with a decision threshold. Change one input while holding the others fixed and observe that a sigmoid probability does not change by a constant amount across its range. Explain why a weight sign describes the model calculation but does not establish a causal effect.
Scale the picture from one neuron to a layer by tracking input shape, number of units and output shape. Draw arrows showing which parameters are learned and which settings, such as architecture or threshold, are selected through validation. For architecture choice, start from data structure.
A fully connected network can model structured tables; recurrent or long short-term memory networks explicitly process order; transformers use attention to represent dependencies; convolutional networks exploit local spatial structure. Treat these as design matches, not a ladder of prestige. Compare each with a simpler regression, tree or engineered-feature baseline on identical evidence.
For transfer learning, write a staged plan: replace the task-specific head, freeze earlier representation layers, train on local labels, inspect validation, then test limited fine-tuning if a domain gap remains. Record which layers change and protect the final test set from repeated selection. Diagnose overfitting from the relationship between training and validation behaviour, not from depth alone.
Consider regularisation, early stopping, simpler architecture, more representative data and leakage checks before celebrating a lower training loss. Close every deep-learning answer with cost and control: computation, data demand, inference latency, interpretability, subgroup errors and monitoring. Complexity is justified only when its unseen benefit matters to the task and survives those constraints.
Working through Deep Learning for Structured Data in INFS3822? Sia is AskSia’s AI Computer Science tutor — ask any INFS3822 Deep Learning for Structured Data question and get a clear, step-by-step explanation grounded in how INFS3822 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
