👁️Computer Vision and Image Processing Unit 6 – Machine Learning in Computer Vision

Key Concepts and Foundations

• Computer Vision (CV) focuses on enabling computers to interpret, understand, and process visual data from the world
• CV involves capturing, processing, analyzing, and understanding digital images or videos
• Draws from various fields including mathematics, physics, signal processing, and artificial intelligence
• Aims to automate tasks that the human visual system can perform (object recognition, scene understanding)
• Key challenges include variations in lighting, viewpoint, scale, and occlusion
  • Occlusion occurs when objects are partially or fully hidden by other objects in the scene
• CV systems often follow a pipeline: image acquisition, preprocessing, feature extraction, and high-level understanding
• Low-level vision deals with early processing stages (filtering, edge detection) while high-level vision focuses on semantic interpretation (object recognition, scene understanding)

Machine Learning Basics for CV

• Machine Learning (ML) involves building systems that can learn and improve from experience without being explicitly programmed
• ML algorithms automatically learn patterns and rules from data to make predictions or decisions
• Three main types of ML: supervised learning, unsupervised learning, and reinforcement learning
  • Supervised learning uses labeled data to train models for classification or regression tasks
  • Unsupervised learning discovers hidden patterns or structures in unlabeled data (clustering, dimensionality reduction)
  • Reinforcement learning trains agents to make decisions based on rewards and punishments from the environment
• In CV, ML is used for tasks like image classification, object detection, and semantic segmentation
• Training data consists of input images and corresponding labels or annotations
• Models learn to map input features to output predictions by minimizing a loss function that measures the difference between predicted and true outputs
• Overfitting occurs when models memorize training data and fail to generalize to new data
  • Regularization techniques (L1/L2 regularization, dropout) can help prevent overfitting

Image Preprocessing Techniques

• Preprocessing aims to enhance image quality, remove noise, and standardize data for further analysis
• Image resizing adjusts the spatial dimensions of an image while maintaining its aspect ratio
  • Resizing is often necessary to match the input size of ML models or to reduce computational complexity
• Image normalization scales pixel values to a standard range (e.g., [0, 1] or [-1, 1]) to improve convergence during training
• Contrast enhancement techniques (histogram equalization, gamma correction) improve the visual appearance and highlight important details
• Noise reduction methods (median filtering, Gaussian smoothing) remove unwanted noise while preserving edges and structures
• Data augmentation artificially increases the training set by applying random transformations (rotation, flipping, cropping) to existing images
  • Augmentation helps improve model robustness and generalization to new data
• Image segmentation separates an image into multiple segments or regions based on pixel characteristics (color, texture, intensity)

Feature Extraction and Representation

• Features are distinctive properties or patterns in an image that can be used for recognition or matching
• Hand-crafted features are manually designed based on domain knowledge (SIFT, HOG, LBP)
  • Scale-Invariant Feature Transform (SIFT) detects and describes local features that are invariant to scale and rotation
  • Histogram of Oriented Gradients (HOG) captures the distribution of gradient orientations in local regions
  • Local Binary Patterns (LBP) encodes local texture information by comparing each pixel with its neighbors
• Learned features are automatically discovered by ML models during training (CNN features)
• Convolutional Neural Networks (CNNs) learn hierarchical features from raw pixel data
  • Lower layers capture simple patterns (edges, corners) while higher layers capture more complex and abstract features (objects, parts)
• Feature descriptors compactly represent the extracted features for efficient storage and comparison
  • Bag-of-Visual-Words (BoVW) quantizes local features into a fixed-size vocabulary and represents images as histograms of visual word occurrences
• Dimensionality reduction techniques (PCA, t-SNE) project high-dimensional features into a lower-dimensional space while preserving important information

Popular ML Algorithms in CV

• Support Vector Machines (SVM) find the optimal hyperplane that maximally separates different classes in a high-dimensional feature space
  • SVMs are effective for binary classification tasks and can handle non-linearly separable data using kernel tricks
• Random Forests (RF) are ensemble models that combine multiple decision trees trained on random subsets of features and data
  • RFs are robust to overfitting and can handle high-dimensional data with correlated features
• K-Nearest Neighbors (KNN) classifies a new sample based on the majority class of its k nearest neighbors in the feature space
  • KNN is a non-parametric method that requires no training but can be computationally expensive for large datasets
• Naive Bayes (NB) is a probabilistic classifier that assumes independence between features given the class label
  • NB is simple, fast, and works well with high-dimensional data but may underperform when the independence assumption is violated
• Logistic Regression (LR) models the probability of a binary outcome as a linear function of input features
  • LR is often used as a baseline model and can be extended to multi-class problems using one-vs-all or softmax approaches
• Decision Trees (DT) recursively partition the feature space based on the most informative features at each node
  • DTs are interpretable and can handle both numerical and categorical features but may overfit if grown too deep

Deep Learning for Computer Vision

• Deep Learning (DL) refers to ML models with multiple layers that can learn hierarchical representations from raw data
• Convolutional Neural Networks (CNNs) are the most popular DL architecture for CV tasks
  • CNNs consist of convolutional layers that learn local features, pooling layers that downsample the feature maps, and fully connected layers that perform classification or regression
  • Popular CNN architectures include LeNet, AlexNet, VGGNet, GoogLeNet, and ResNet
• Transfer Learning leverages pre-trained models on large datasets (ImageNet) to solve related tasks with limited training data
  • The pre-trained model can be used as a feature extractor or fine-tuned on the target task
• Object Detection aims to localize and classify multiple objects in an image
  • Region-based methods (R-CNN, Fast R-CNN, Faster R-CNN) generate object proposals and then classify each proposal
  • Single-shot methods (YOLO, SSD) directly predict object bounding boxes and classes in a single forward pass
• Semantic Segmentation assigns a class label to each pixel in an image
  • Fully Convolutional Networks (FCN) adapt CNNs for dense pixel-wise prediction by replacing fully connected layers with convolutional layers
  • U-Net is a popular architecture for biomedical image segmentation that uses skip connections to combine high-level and low-level features

Performance Evaluation and Metrics

• Evaluation metrics quantify the performance of ML models on test data
• Accuracy measures the overall correctness of predictions but can be misleading for imbalanced datasets
  • Accuracy = (True Positives + True Negatives) / (Total Samples)
• Precision measures the proportion of true positive predictions among all positive predictions
  • Precision = True Positives / (True Positives + False Positives)
• Recall measures the proportion of true positive predictions among all actual positive samples
  • Recall = True Positives / (True Positives + False Negatives)
• F1 Score is the harmonic mean of precision and recall, providing a balanced measure of model performance
  • F1 Score = 2 * (Precision * Recall) / (Precision + Recall)
• Intersection over Union (IoU) measures the overlap between predicted and ground truth bounding boxes or segmentation masks
  • IoU = Area of Intersection / Area of Union
• Mean Average Precision (mAP) is a common metric for object detection that computes the average precision across different recall levels and object classes
• Confusion Matrix summarizes the model's performance in a table, showing the counts of true positives, true negatives, false positives, and false negatives for each class

Practical Applications and Case Studies

• Face Recognition involves identifying or verifying individuals based on their facial features
  • Applications include security systems, surveillance, and social media tagging
• Autonomous Vehicles rely on CV techniques for tasks like lane detection, obstacle avoidance, and traffic sign recognition
  • Cameras, LiDAR, and other sensors provide visual data for perception and decision-making
• Medical Image Analysis applies CV to diagnose diseases, segment anatomical structures, and guide surgical procedures
  • Examples include tumor detection in MRI scans, retinal image analysis for diabetic retinopathy, and cell segmentation in microscopy images
• Augmented Reality (AR) overlays virtual content on top of the real world using CV techniques like object tracking and pose estimation
  • AR applications range from gaming (Pokémon Go) to education (interactive learning experiences) and industrial maintenance (remote assistance)
• Retail and E-commerce use CV for product recognition, visual search, and recommendation systems
  • Customers can upload images to find similar products or receive personalized recommendations based on their visual preferences
• Agriculture and Precision Farming employ CV to monitor crop health, detect pests and diseases, and optimize resource allocation
  • Drones equipped with cameras can capture aerial imagery for analysis and decision support
• Robotics and Manufacturing integrate CV for tasks like quality inspection, defect detection, and object grasping
  • CV enables robots to perceive and interact with their environment in real-time