Day 35: Pooling & batch normalization
Two layers that make deep CNNs work
Pooling: shrink and summarize
Max pooling takes a small window (say 2×2) and keeps only its maximum value, halving the spatial resolution. This does two things: it reduces computation as the network deepens, and it grants a little translation invariance — the exact pixel position of a feature matters less, only that it's present in the region. It's how CNNs stay tractable while their receptive field grows.
Batch normalization: stabilize the signal
As data flows through many layers, the distribution of activations can drift, making training slow and unstable. Batch normalization re-centers and re-scales each layer's activations using the current batch's statistics, keeping the signal in a healthy range. In practice it lets you train deeper networks, use higher learning rates, and converge faster — and it's why the batch-statistics-vs-running-averages distinction (Day 29) matters at eval time.
import torch.nn as nn
block = nn.Sequential(
nn.Conv2d(32, 64, kernel_size=3, padding=1),
nn.BatchNorm2d(64), # stabilize activations
nn.ReLU(), # nonlinearity
nn.MaxPool2d(2), # halve spatial size
)
# (batch, 32, 112, 112) -> (batch, 64, 56, 56)The canonical block
conv → batchnorm → ReLU → pool is the repeating motif of nearly every classic CNN. Once you see it, architectures like ResNet stop looking exotic — they're this block, repeated, with one clever addition you'll meet tomorrow (skip connections).
Key terms
- Max pooling
- Downsampling by keeping the maximum value in each small window, reducing resolution and adding translation invariance.
- Batch normalization
- Normalizing a layer's activations using batch statistics to stabilize and speed up training.
- Translation invariance
- The property that a feature is recognized regardless of its exact position in the image.
What is a primary benefit of adding batch normalization layers to a deep CNN?
