Course Overview

This course provides a comprehensive introduction to the principles and techniques of computer vision, specifically tailored for robotics applications. It covers the essential concepts, algorithms, and tools necessary to equip students with the knowledge to design and implement robust vision systems for robots.

​

Course Objectives

Upon completion of this course, students will be able to:

• Understand the basic principles of image formation and processing.

• Apply fundamental image processing techniques such as filtering, thresholding, and edge detection.

• Implement feature detection and description algorithms (e.g., SIFT, SURF, ORB).

• Utilize camera calibration and 3D reconstruction techniques.

• Employ object detection and tracking algorithms.

• Integrate computer vision techniques with robot control and navigation.

​

Course Curriculum

Module 1: Introduction to Computer Vision

• Image Formation and Sensors:

  • Pinhole camera model

  • Camera calibration

  • Types of sensors

• Image Processing Fundamentals:

  • Image acquisition and representation

  • Image enhancement techniques

  • Geometric transformations

Module 2: Feature Detection and Description

• Interest Point Detection:

  • Harris corner detector

  • Shi-Tomasi corner detector

  • Scale-invariant feature transform (SIFT)

  • Speeded Up Robust Features (SURF)

  • Oriented FAST and Rotated BRIEF (ORB)

• Feature Description:

  • Local binary patterns (LBP)

  • Histogram of Oriented Gradients (HOG)

 

Module 3: Stereo Vision and 3D Reconstruction

• Stereo Matching:

  • Disparity estimation techniques (e.g., block matching, dynamic programming)

  • Depth map generation

• Structure from Motion (SfM):

  • 3D reconstruction from multiple images

  • Bundle adjustment

• Simultaneous Localization and Mapping (SLAM):

  • Building 3D maps of an environment

  • Tracking robot's position and orientation

​

Module 4: Object Detection and Tracking

• Traditional Object Detection:

  • Template matching

  • Haar cascades

  • Histogram of Oriented Gradients (HOG) + Linear SVM

• Deep Learning for Object Detection:

  • Convolutional Neural Networks (CNNs)

  • Region-based Convolutional Neural Networks (R-CNN)

  • You Only Look Once (YOLO)

• Object Tracking:

  • Template matching

  • Kalman filters

  • Particle filters

  • DeepSORT

​

Module 5: Running OpenCV in ROS

• Installing OpenCV in ROS

• Running your vision code in ROS
• Bonus Activity:
  • Mount a camera onto a robot arm
  • Stream its data, and
  • Process the information for object detection or calibration

​