COMPUTER VISION / ACADEMIC PROJECT
Small images.
Bigger questions.
What can a model learn from just 48 × 48 pixels? A comparison of classical machine learning, a custom residual CNN, and transfer learning on seven facial expression labels.
Facial expression classification · Model design, evaluation, and error analysis.
View code on GitHub- Test images
- 7,178
- Expression labels
- 7
- CNN test accuracy
- 59.5%
- CNN macro F1
- 0.552
01 / THE CHALLENGE
Seven labels.
Unequal evidence.
FER2013 contains small grayscale face images with labels such as happy, sad, and neutral. The task is to predict those dataset labels. An image label is not a reliable measurement of a person’s internal emotional state.
The loaded data contains 28,709 training images and 7,178 test images. A stratified split reserves 5,742 training images for validation, leaving 22,967 for fitting. The test set is kept separate.
Class imbalance makes accuracy incomplete: “happy” accounts for almost a quarter of the test set, while “disgust” has only 111 examples. Macro F1 weights each class equally.
A closer look at the data.
Class share within the selected partition. Training pool includes the later validation split. Each bar uses a fixed 0–30% scale.
02 / THE APPROACH
Build a baseline.
Preserve the pixels.
- Establish a floor. A majority-class dummy classifier, logistic regression, and a random forest use flattened 2,304-pixel inputs. Classical-model tuning uses eight randomized trials and three-fold cross-validation on a stratified 3,500-image subset, scored by macro F1.
- Keep spatial structure. The custom CNN processes 48 × 48 grayscale images through four residual blocks, with 32, 64, 128, and 256 filters. Batch normalization, spatial dropout, L2 regularization, and training-time augmentation help control overfitting.
- Train with validation feedback. The CNN uses class weights, Adam, batches of 64, and up to 70 epochs. Early stopping restores the best validation-loss weights; a plateau reduces the learning rate.
- Compare transfer learning. ImageNet-pretrained ResNet50V2 receives images resized to 128 × 128 RGB. Training first fits a classification head, then fine-tunes the last 20 backbone layers while keeping batch normalization frozen.
INSIDE THE CUSTOM CNN
03 / THE COMPARISON
A smaller model.
A stronger result.
Five models evaluated on the same 7,178-image test set. Recorded notebook results, with a fixed 0–1 scale.
The custom CNN leads at 0.552 macro F1, compared with 0.472 for ResNet50V2.
The custom CNN is 6.3 percentage points ahead of ResNet50V2 on test accuracy, with about 18.5 times fewer total parameters. Its 1.29 million parameters compare with 23.83 million in the transfer-learning model.
This is a result from one training setup. It does not establish that a custom CNN will generally beat transfer learning. Input resolution, preprocessing, and the limited fine-tuning schedule also differ.
View all recorded scores
| Model | Accuracy | Macro F1 | Total parameters |
|---|---|---|---|
| Custom CNN | 59.50% | 0.552 | 1,288,135 |
| ResNet50V2 | 53.22% | 0.472 | 23,828,487 |
| Random Forest | 42.87% | 0.412 | Not reported |
| Logistic Regression | 32.07% | 0.295 | Not reported |
| Dummy Baseline | 24.71% | 0.057 | Not reported |
04 / THE ERRORS
Where the model
loses the distinction.
The largest recorded CNN confusion is sad → neutral: 329 images, or 26.4% of the sad test class. Fear also frequently becomes sad or neutral. A single headline score hides these different failure patterns.
Top five CNN confusion pairs by count. Percentages are shares of the true class, not of all errors.
05 / REFLECTION
Evaluate the limits.
Then iterate.
The random forest’s later fit diagnostic shows 91.8% training accuracy but only 42.0% validation accuracy, a substantial generalization gap. The CNN’s corresponding gap is much smaller, although its test performance still leaves considerable room for improvement.
Next steps would be repeated runs, controlled comparisons of augmentation and class weights, and testing on an external dataset. Grad-CAM and pixel feature importance are included in the notebook as exploratory inspection tools, not proof that the models understand emotions.
These results concern a facial-expression benchmark. They do not validate emotion inference in real-world settings, especially across different people, cultures, lighting conditions, and contexts.