Key Convolutional Neural Network Architectures

Why This Matters

Understanding CNN architectures isn't just about memorizing layer counts—it's about recognizing the design problems each network solved and the innovations that made deeper, faster, and more accurate models possible. You're being tested on your ability to explain why certain architectural choices work: how skip connections prevent vanishing gradients, why depthwise separable convolutions reduce computation, and what trade-offs exist between speed and accuracy in detection networks.

These architectures form the backbone of modern computer vision systems, from medical imaging to autonomous vehicles. When you encounter questions about feature extraction, gradient flow, computational efficiency, or real-time inference, you need to connect specific networks to the concepts they exemplify. Don't just memorize that ResNet has skip connections—understand why that innovation enabled training networks 152 layers deep when previous attempts failed.

---

Foundational Architectures: Establishing the CNN Paradigm

These early networks proved that convolutional approaches could outperform traditional methods and established the core building blocks—convolution, pooling, and nonlinear activation—that every subsequent architecture builds upon.

LeNet-5

• First successful CNN architecture—developed by Yann LeCun in 1998 for handwritten digit recognition on the MNIST dataset
• 7-layer structure introduced the conv-pool-conv-pool-FC pattern that became the standard CNN template
• Tanh activation and average pooling were later replaced by ReLU and max pooling, but the architectural blueprint persists in modern networks

AlexNet

• 2012 ImageNet winner that proved deep learning could dominate large-scale image classification, achieving 15.3% top-5 error rate
• ReLU activation replaced tanh/sigmoid, dramatically accelerating training by avoiding saturation in positive regions
• Dropout regularization and GPU training introduced two techniques now considered essential for training deep networks effectively

---

Depth-Focused Architectures: Going Deeper with Small Filters

These networks explored whether stacking more layers with smaller filters could capture increasingly abstract features. The key insight: two 3×3 convolutions have the same receptive field as one 5×5 but with fewer parameters and more nonlinearity.

VGGNet

• Uniform 3×3 filters throughout—proved that consistent small kernels with increased depth (16-19 layers) outperform larger, shallower alternatives
• Simple, repeatable structure makes it ideal for transfer learning; VGG features remain popular pretrained backbones
• 138 million parameters in VGG-16 demonstrated a key limitation: depth alone creates massive computational and memory costs

GoogLeNet (Inception)

• Inception module applies multiple filter sizes (1×1, 3×3, 5×5) in parallel, capturing multi-scale features simultaneously
• 1×1 convolutions for dimensionality reduction—bottleneck layers cut parameters from 138M (VGG) to just 4M while maintaining accuracy
• Global average pooling replaced fully connected layers, eliminating overfitting-prone dense connections at the network's end

---

Gradient Flow Innovations: Solving the Depth Problem

As networks grew deeper, vanishing gradients made training increasingly difficult—early layers received negligible updates. These architectures introduced structural innovations that allow gradients to flow unimpeded through hundreds of layers.

ResNet

• Skip connections (residual learning) let the network learn $F(x) + x$ instead of $F(x)$, providing a gradient highway that bypasses problematic layers
• Extreme depth becomes possible—ResNet-152 trains successfully where plain 152-layer networks fail completely due to degradation
• Identity mappings mean layers can learn to "do nothing" if needed, making deeper networks at least as good as shallower ones

DenseNet

• Dense connectivity connects each layer to all subsequent layers, creating $\frac{L(L+1)}{2}$ direct connections in an L-layer network
• Feature reuse means later layers access all previously computed features, reducing redundant computation and parameter count
• Growth rate hyperparameter controls how many new feature maps each layer contributes, enabling fine-grained efficiency tuning

---

Task-Specific Architectures: Beyond Classification

These networks adapt CNN principles for specialized tasks—semantic segmentation requires pixel-level predictions, while object detection demands localization and classification simultaneously.

U-Net

• Encoder-decoder symmetry with skip connections preserves spatial information lost during downsampling, enabling precise boundary localization
• Designed for limited data—the architecture combined with aggressive data augmentation achieved state-of-the-art medical image segmentation with ~30 training images
• Skip connections concatenate encoder features to decoder layers, combining high-resolution spatial detail with deep semantic information

Faster R-CNN

• Region Proposal Network (RPN) generates candidate bounding boxes using the same convolutional features as classification, enabling end-to-end training
• Two-stage detection: RPN proposes regions, then a second network classifies and refines each proposal—accurate but computationally expensive
• Anchor boxes at multiple scales and aspect ratios allow the RPN to detect objects of varying sizes without multi-scale image pyramids

---

Efficiency-Optimized Architectures: Speed and Resource Constraints

Real-world deployment often requires inference on mobile devices or real-time processing. These architectures sacrifice some accuracy for dramatic improvements in speed, memory footprint, and computational cost.

MobileNet

• Depthwise separable convolutions factor standard convolution into depthwise (spatial) and pointwise (channel) operations, reducing computation by $\frac{1}{N} + \frac{1}{D_K^2}$ where $N$ is output channels and $D_K$ is kernel size
• Width and resolution multipliers allow tuning the accuracy-efficiency trade-off for specific deployment constraints
• ~4.2 million parameters (MobileNetV1) enables real-time inference on smartphones while maintaining competitive ImageNet accuracy

YOLO (You Only Look Once)

• Single-shot detection treats localization and classification as one regression problem, predicting bounding boxes and class probabilities in a single forward pass
• Grid-based prediction divides images into $S \times S$ cells, each predicting $B$ bounding boxes—enabling 45+ FPS real-time detection
• Speed-accuracy trade-off favors real-time applications; YOLOv4/v5 iterations improved accuracy while maintaining the single-shot paradigm

---

Quick Reference Table

---

Self-Check Questions

1. Both ResNet and DenseNet solve the vanishing gradient problem through shortcut connections. What is the fundamental difference in how they combine features, and what are the memory implications of each approach?

2. You need to deploy an object detection model on a drone with limited computing power that must process video in real-time. Which architecture would you choose and why? What accuracy trade-offs would you accept?

3. Compare the role of skip connections in U-Net versus ResNet. How does the purpose of these connections differ despite their structural similarity?

4. GoogLeNet and VGGNet achieved similar ImageNet accuracy in 2014, yet GoogLeNet used 35× fewer parameters. Identify two specific architectural innovations that enabled this efficiency.

5. If asked to design a system for detecting tumors in medical CT scans where precise boundary delineation matters more than processing speed, which architectures would you combine and why? Consider both detection and segmentation requirements.