 Figure: MLP Training vs Validation AUC Curve
MLP Training vs Validation AUC
0.9998
0.9996
0.9994 -
0.9992 -
- train_auc
AUC
val_auc
0.9990
0.9988
0.9986
0.9984
0
1
2
3
4
5
Epoch
1.OverallTrend:
From epoch 0 to epoch 2, the training-set AUC (blue) gradually rises from about 0.9984 to around 0.9985 After epoch 2, it fluctuates slightly between 0.9984 and 0.9985 through epoch 5.Meanwhile, the validation-set AUC (orange) begins at approximately 0.9998 in epoch 0, dips slightly to roughly 0.9997, and then remains stable between 0.9997 and 0.9998 for the remaining epochs.
2 Gap Between Training and Validation
The validation AUC consistently sits about 0.0013 higher than the training AUC.This can occur because, after SMOTE oversampling, the model is very sensitive to fraud examples in the smaller validation split, while it is still fitting more varied data in the larger training set, leading to a slightly lower training AUC.
3 Need for EarlyStopping
The validation AUC peaks around epoch 0-1 (~0.9998) and then shows no meaningful improvement afterward In fact, it dips marginally by epoch 2 and only recovers slightly later.By using EarlyStopping(monitor='val_auc', mode='max', patience=5), we stop training as soon as the validation AUC stops improving This prevents overfitting in later epochs and saves compute time (we don't need to run all 50 epochs).
I
4 Key Takeaways
Because both training and validation AUCs are extremely close to 1.0, the network distinguishes fraud vs non-fraud almost perfectly The flat lines indicate that our chosen architecture and hyperparameters are appropriate.
Overall, the MLP converges quickly on the SMOTE-balanced data; only a few epochs are needed EarlyStopping ensures we halt training before any overfitting occurs and keeps the workflow efficient.I
3 Insights and takeaways
Lightweight gradient boosters perform strongly in tabular high-dimensional data.
Logistic regression still has value:
with an area under the curve of about 0.9670, it is a reliable choice It is fast, performs well and has clear interpretations.
Deep learning (multi-layer perceptual machines) needs more tuning:
area under the curve is about 0.9636, and it did not outperform the gradient boosting tree model this time Additional hidden layers, enhanced regularisation and further optimisation of hyperparameters are needed to outperform the light gradient boosting machine.
I
1
The model is sensitive to category imbalance: we balanced the training set using a synthetic few oversampling technique, and the results reflect a careful trade-off between recall, precision, and false positive rates

,The subject operating characteristic curves clearly show the differences in the models' abilities in this regard.
I
4.Summary
The subject operating characteristic curve visualisation clearly presents the comparison of the three models on the test set:
Lightweight Gradient Booster is the most accurate (area under the curve ~0.9808), maintaining a high true-positive rate at different false-positive rate thresholds.
Logistic regression was a close second (area under the curve ~0.9670), suitable for obtaining interpretable results quickly.
Multi-layer perceptron (neural network) with an area under the curve of about 0.9636 requires further tuning to achieve gradient boosted tree performance, but still captures some of the non-linear patterns in the data.
This visual display effectively achieves the goal of presenting key visual results, model performance, and sharing insights gained from model training and evaluation.Credit Card Fraud Detection
ROC Curve (Logistic vs LightGBM vs MLP)
ROC Curves: Logistic vs

 LightGBM vs MLP
1.0
0.8
True Positive Rate
0.6
0.4
0.2-
Logistic Regression (AUC=0.9670)
(AUC=0.9808)
- LightGBM
- MLP (Neural Net) (AUC=0.9636)
0.0
--- Random Guess (AUC=0.5000)
0.0
0.2
0.4
0.6
0.8
1.0
False Positive Rate
1.Content and meaning of the visualisation chart
The graph displays the subject work characteristic curves for the three models The blue curve is logistic regression, the orange curve is a light gradient lifter, and the green curve is a multilayer perceptron with the same test set The black dashed line represents the random guess baseline, corresponding to an area under the curve of 0.5.
The horizontal axis is the false positive rate, which refers to the proportion of normal transactions that are misclassified as fraud The vertical axis is the true positive rate, which refers to the proportion of fraudulent transactions that are correctly identified.
As seen in the graph, the orange curve performs best It is consistently closer to the upper left corner in almost all ranges of false positive rates and has the highest area under the curve at 0.9808.2.Interpretation of model performance
Light Gradient Booster:
the area under the curve is 0.9808, which is better than the other models The curve is near the upper left corner, indicating that it maintains a high rate of true positives while controlling a low rate of false positives When the false positive rate is very low (less than 0.05), the Lightweight Gradient Booster has a true positive rate of about 0.90 This means that it captures most fraud cases with few false positives The model is the most stable and has the best overall performance.
Logistic regression:
with an area under the curve of 0.9670, the performance is also strong At low false positive rates, the curve is close to the Light Gradient Booster, indicating that logistic regression also captures most fraud cases with low false positives.
Multilayer Perceptron:
The area under the curve, 0.9636, is slightly lower than logistic regression As a neural network, it is able to capture some non-linear features, but did not outperform the Light Gradient Booster in this task.
Stochastic guessing:
area under the curve 0.5, representing a completely random model that cannot distinguish between positive and negative samples This is the baseline above which all model curves should be significantly higher.
6
I
3 Insights and takeaways
Lightweight gradient boosters perform strongly in tabular high-dimensional data Logistic regression still has value:
with an area under the curve of about 0.9670, it is a reliable choice It is fast, performs well and has clear interpretations.
Deep learning (multi-layer perceptual machines) needs more tuning:
area under the curve is about 0.9636, and it did not outperform the gradient boosting tree model this time Additional hidden layers, enhanced regularisation and further optimisation of hyperparameters are needed to outperform the light gradient boosting machine.
The model is sensitive to category imbalance:
we balanced the training set using a synthetic few oversampling technique, and the results reflect a careful trade-off between recall, precision, and false positive rates The subject operating characteristic curves clearly show the differences in the models' abilities in this regard.

AskSia

Plus

SUMMARY