Lab #3: Kinematics and Vision

       Introduction

The objective of this lab is to pratice kinematic control of the Turtlebot and to better understand vision processing on the Turtlebot. You are welcome to copy code from the internet, but make sure to credit where it came from. You can work in teams if you'd like.

Objectives

Upon successful completion of this lab, you will be able to:

  1. Have the simulated turtlebot trace a triangle under open-loop control
  2. Have the simulated turtlebot identify and respond to different objects in the environment
Vision

To help you get started with vision, I've provided three files:

To start, I suggest you run the default world with the turtlebot. Then, you can run:
python turtle_vision.py
This will open two new windows. One will show you what the turtlebot is seeing. The other will show you a window that has been filtered. It will pick up the bookshelf but not the other objects in the default world. If you wanted, you could add other objects (such as the soda can) that will also be picked up by the filter.
image_converter.py converts files from a ROS format to a format you can manipulate with OpenCV, and vice versa. You shouldn't need to modify this file.
turtle_vision.py is the main file that shows how to process RGB images and depth images (not needed at this point).
object_detection.py shows how you can process an image. process_target() sets the upper and lower bounds for an HSV filter. process_image() uses lower and upper bounds to filter based on HSV and then shows the image.
You will need to modify these files so that you not only filter to correctly identify different objects, but also you will have to find the width of images and the center of images. Remember to post to Piazza if you have questions / get stuck!

Assignment
  1. Your goal is to have the turtlebot drive in an equilateral triangle that is 2m per side. First, use inverse kinematics to figure out the speeds you would want to drive the wheels and for how long for (a) a turn and (b) an edge.
  2. Now using twist messages, have your turtlebot trace out the triangle as calculated above. After the turtlebot finishes, how close did it come to ending at the correct location?
  3. Place an object in the environment (I'd recommend a sphere). Use openCV to find the center of the image. Write a reactive controller that can drive the robot towards the sphere.
  4. Enhance your code so that the turtlebot stops some known distance from the sphere (e.g., by counting the number of pixels wide the sphere is).
  5. By placing three objects in the environment, write a reactive controller so that the turtlebot traces out an equilateral triangle, 3m per side. How far away from the start point does the turtlebot end up? Is this triange you traced better or worse than when you used inverse kinematics, and why?
  6. For extra credit, as the turtlebot drives towards an object, display the turtlebot's view of the object. Draw the midpoint of the object on this image and update it as the turtlebot moves. Also add some text saying 1) whether the robot is to the right, to the left, or centered on the object and 2) the estimated distance from the object.
Hand in
In Blackboard, please submit the following:
  1. A text file with:
    • Your name
    • The answers regarding how close to the start state the turtlebot ended for both controlers, and your speculations about why one is better than the other.
    • The names of any people you worked with (optional)
    • What you thought the hardest part of the assignment was
    • Any on-line references / websites you found particularly useful
  2. A video or screen capture of your turtlebot tracing the triangle via inverse kinematics and via a reactive vision controler.
  3. The calculations you performed for the inverse kinematics.
  4. The code you wrote for the assignment and any instructions needed to execute it.
Grading
  1. Text file content: 10
  2. Inverse kinematics calculations: 10
  3. Code for kinematics-based driver: 20
  4. Code for vision-based driver: 30
  5. Videos of both triangles:30
  6. Extra Credit: 10