Deep Learning Systems Unit 12 ReviewDeep Learning for Computer Vision

Key Concepts and Foundations

• Deep learning leverages artificial neural networks with multiple layers to learn hierarchical representations of data
• Vision tasks involve processing and analyzing visual data (images, videos) to extract meaningful information
• Neural networks consist of interconnected nodes or neurons organized into layers (input, hidden, output)
• Each neuron applies a nonlinear activation function to its weighted input and passes the result to the next layer
• Common activation functions include sigmoid, tanh, ReLU (Rectified Linear Unit), and its variants (Leaky ReLU, ELU)
• Training a neural network involves optimizing the weights to minimize a loss function using techniques like gradient descent and backpropagation
  • Gradient descent iteratively adjusts weights in the direction of steepest descent of the loss function
  • Backpropagation efficiently computes gradients by propagating errors backward through the network
• Convolutional Neural Networks (CNNs) are specifically designed for processing grid-like data such as images

Neural Network Architectures for Vision

• LeNet-5 is a pioneering CNN architecture that demonstrated the effectiveness of CNNs for handwritten digit recognition
• AlexNet popularized deep CNNs by achieving breakthrough performance on the ImageNet classification challenge
• VGGNet introduced a deeper architecture with smaller convolutional filters (3x3) and achieved state-of-the-art results
• GoogLeNet (Inception) introduced the concept of Inception modules, which perform convolutions at multiple scales and concatenate the results
  • Inception modules help capture features at different spatial resolutions and reduce computational complexity
• ResNet introduced residual connections that allow gradients to flow directly through identity mappings, enabling training of very deep networks (100+ layers)
• DenseNet builds upon ResNet by introducing dense connections, where each layer receives inputs from all preceding layers
• EfficientNet achieves state-of-the-art accuracy with significantly fewer parameters by systematically scaling network width, depth, and resolution

Convolutional Neural Networks (CNNs)

• CNNs are designed to process grid-like data such as images by exploiting spatial locality and translation invariance
• Convolutional layers apply learned filters to input data, capturing local patterns and features
  • Filters slide over the input, performing element-wise multiplications and summing the results
  • Multiple filters are used to detect different features (edges, textures, shapes)
• Pooling layers downsample the spatial dimensions of the feature maps, reducing computational complexity and providing translation invariance
  • Max pooling selects the maximum value within each pooling window
  • Average pooling computes the average value within each pooling window
• Fully connected layers follow convolutional and pooling layers to perform high-level reasoning and classification
• CNNs can learn hierarchical representations, with earlier layers capturing low-level features (edges) and later layers capturing high-level concepts (objects)
• Data augmentation techniques (rotation, flipping, cropping) are commonly used to increase the diversity of training data and improve generalization

Image Classification Techniques

• Image classification involves assigning a class label to an input image from a predefined set of categories
• Softmax activation is commonly used in the output layer for multi-class classification, producing a probability distribution over classes
• Cross-entropy loss is often used as the objective function, measuring the dissimilarity between predicted and true class probabilities
• Evaluation metrics for image classification include accuracy, precision, recall, and F1 score
  • Accuracy measures the overall correctness of predictions
  • Precision quantifies the proportion of true positive predictions among all positive predictions
  • Recall quantifies the proportion of true positive predictions among all actual positive instances
• Techniques like one-hot encoding and label smoothing can be used to represent class labels and regularize the model
• Class imbalance can be addressed through techniques like oversampling minority classes, undersampling majority classes, or using class weights

Object Detection and Segmentation

• Object detection involves identifying and localizing multiple objects within an image
• Popular object detection architectures include R-CNN, Fast R-CNN, Faster R-CNN, YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector)
  • R-CNN uses selective search to generate region proposals and applies a CNN for feature extraction and classification
  • Fast R-CNN improves upon R-CNN by sharing computation and using ROI (Region of Interest) pooling
  • Faster R-CNN introduces a Region Proposal Network (RPN) to generate region proposals, making the process end-to-end trainable
• YOLO divides the image into a grid and predicts bounding boxes and class probabilities for each grid cell, enabling real-time detection
• SSD uses a single CNN to directly predict bounding boxes and class scores at multiple scales
• Semantic segmentation assigns a class label to each pixel in an image, providing a dense classification map
  • Fully Convolutional Networks (FCNs) adapt classification CNNs for pixel-wise prediction by replacing fully connected layers with convolutional layers
  • U-Net is a popular architecture for semantic segmentation that uses an encoder-decoder structure with skip connections
• Instance segmentation extends semantic segmentation by distinguishing individual instances of objects
  • Mask R-CNN builds upon Faster R-CNN by adding a branch for predicting object masks in parallel with bounding box regression and classification

Transfer Learning and Pre-trained Models

• Transfer learning leverages knowledge gained from solving one task to improve performance on a related task
• Pre-trained models, trained on large-scale datasets (ImageNet), can be used as feature extractors or fine-tuned for specific tasks
  • Feature extraction: Use pre-trained model's learned features as input to a new classifier
  • Fine-tuning: Unfreeze some or all layers of the pre-trained model and retrain on the target task
• Transfer learning reduces the need for large labeled datasets and accelerates training by starting from a good initialization
• Popular pre-trained models include VGG, ResNet, Inception, and EfficientNet
• Domain adaptation techniques can be used to bridge the gap between the source and target domains when applying transfer learning

Advanced Topics and Current Research

• Attention mechanisms allow models to focus on relevant parts of the input, improving performance and interpretability
  • Self-attention (Transformers) has shown promising results in vision tasks by capturing long-range dependencies
  • Vision Transformers (ViT) apply Transformers directly to image patches, achieving state-of-the-art performance on image classification
• Generative models, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), can synthesize realistic images
  • GANs consist of a generator and a discriminator trained in a two-player minimax game
  • VAEs learn a latent representation of the data and can generate new samples by sampling from the latent space
• Few-shot learning aims to learn from a small number of labeled examples per class
  • Meta-learning approaches learn a learning algorithm that can quickly adapt to new tasks
  • Metric learning approaches learn a distance metric to compare query and support examples
• Unsupervised and self-supervised learning methods aim to learn meaningful representations without explicit labels
  • Contrastive learning maximizes the similarity between augmented views of the same image while minimizing the similarity between different images
  • Clustering and autoencoders can be used to learn compressed representations of the data

Practical Applications and Case Studies

• Image classification has numerous applications, including object recognition, scene understanding, and medical image analysis
  • Classifying plant species from leaf images to aid in biodiversity research
  • Identifying skin lesions in dermatological images for early detection of skin cancer
• Object detection is crucial for applications like autonomous driving, surveillance, and robotics
  • Detecting pedestrians, vehicles, and traffic signs in real-time for self-driving cars
  • Monitoring crowds and detecting abnormal activities in video surveillance systems
• Semantic segmentation enables precise understanding of scene semantics, useful for autonomous navigation and image editing
  • Segmenting road scenes into categories like road, sidewalk, and obstacles for autonomous vehicles
  • Extracting specific objects or regions from images for editing or compositing
• Instance segmentation is valuable for applications that require distinguishing individual objects, such as robotics and medical image analysis
  • Segmenting individual cells or nuclei in microscopy images for quantitative analysis
  • Identifying and tracking individual products on a conveyor belt for automated quality control
• Transfer learning has been successfully applied to various domains, including medical imaging, remote sensing, and agriculture
  • Adapting pre-trained models for disease diagnosis from medical scans (X-rays, CT scans)
  • Classifying land cover types from satellite imagery for environmental monitoring and urban planning