Camera Operations

This guide covers camera operations, setup, and troubleshooting for the RCR Common Robotics Platform.

Overview

The robot uses a camera for visual perception, navigation, and object detection. The camera provides real-time image data for various applications.

Camera Specifications

Model: USB Camera
Resolution: 640x480 (configurable)
Frame Rate: 30 FPS
Interface: USB 2.0
Field of View: 60° (approximate)

Prerequisites

Discovery Server Running: Ensure a ROS discovery server is running (see Discovery Server Setup)
Camera hardware properly connected
ROS2 environment sourced
Sufficient battery charge

Camera Setup

1. Hardware Connection

Physical Mounting
- Mount camera on designated mounting point
- Ensure clear field of view
- Secure mounting to prevent vibration
Power Connection
- Connect USB cable to robot controller
- Verify power requirements
- Check for proper grounding
Data Connection
- Connect USB cable to robot controller
- Verify USB communication
- Test connection stability

2. Software Configuration

Open a new terminal window 📟

Launch Camera:

# Launch camera node
cd ~/repos/common_platform/common_platform_ws
source install/setup.bash
ros2 launch sensors camera.launch.py

You will need to leave this window open for as long as you want the camera running. If you want to verify that the camera data is streaming you can echo the topic, to do so open a new terminal window 📟

Check camera data stream:

ros2 topic echo /${ROS_NAME}/camera/image_raw

Monitor data rate (in hertz):

ros2 topic hz /${ROS_NAME}/camera/image_raw

Once you have checked the data stream, you can close this window as this is just for testing and diagnosis. You want to get in the habit of looking for the ros topic to verify data is streaming. You can do this with:

View all ROS topics

ros2 topic list

Camera Applications

Computer Vision

In order to collect images to annotate in Label Studio, use our custom data_recorder node. This is the first step in fine tuning an existing neural network. Check the Data Annotation document for information on labeling.

Recording Image Files

Open a new terminal window 📟.

If you have never recorded images before, you will need to first build the data-recorder package. After running this once you should not need to do it again in the future.

cd ~/repos/common_platform/common_platform_ws/
colcon build --packages-select data_recorder

Launch recorder

source install/setup.bash
ros2 launch data_recorder ramdisk_recorder.launch.py

Now go check your ~/teleop_data/images folder and you should see some images in there. Recording images will fill up your RAM on your Pi relatively quickly - keep it to about 1-2 minutes total of image capture before annotating your data.

Stop recorder The recorder needs to be stopped manually using Ctrl+C. It should automatically copy the recorded frames from the RAM disk to the following location on your micro SD card: ~/teleop_data/ . Check for a folder called session__ .

Next step: Annotating your data with Label Studio

Recording Video Files

Open a new terminal window 📟

ros2 run image_view video_recorder --ros-args -p filename:=/tmp/robot_video.avi

The video file will be saved to /tmp/robot_video.avi on your Raspberry Pi.

Real-time Monitoring

# View camera feed - you need a virtual environment running to do this - more details soon but in the meantime run this on Joe's computer.
ros2 run rqt_image_view rqt_image_view

Camera Calibration

1. Intrinsic Calibration

Calibration Process:

Print calibration pattern
Launch camera
Run calibration tool
Save calibration parameters

Calibration Command:

# Run camera calibration
# Command to be added after verification

2. Extrinsic Calibration

Coordinate Frame Alignment:

Align camera with robot frame
Verify coordinate transformations
Test with known objects

Transform Setup:

# Check transform tree
ros2 run tf2_tools view_frames

# Verify camera transform
ros2 run tf2_ros tf2_echo base_link camera_link

Camera Data Analysis

1. Image Data

Image Message:

std_msgs/Header header
uint32 height
uint32 width
string encoding
uint8 is_bigendian
uint32 step
uint8[] data

2. Data Quality

Check Image Quality:

# View camera feed
ros2 run rqt_image_view rqt_image_view

# Check for:
# - Clear image
# - Proper exposure
# - No distortion
# - Consistent frame rate

Troubleshooting

Common Issues

Check the camera hardware is present

cam -l

No Camera Data:

Check USB connection
Verify Teensy is powered and running
Ensure a discovery server is running
Connect Cursor to your Pi via SSH and ask for diagnosis help

Poor Image Quality:

Clean camera lens
Check for obstructions
Verify focus
Test in different lighting

Inconsistent Frame Rate:

Check USB bandwidth
Verify power supply
Test with different resolution
Check for interference

Diagnostic Commands

# Check USB devices
lsusb | grep -i camera

# Check video devices
ls /dev/video*

# Test camera
ros2 run usb_cam usb_cam_node --ros-args -p video_device:=/dev/video0

# Monitor data quality
ros2 topic echo /${ROS_NAME}/camera/image_raw
ros2 topic hz /${ROS_NAME}/camera/image_raw

Error Codes

Common Error Messages:

“Failed to open video device” - Check USB connection
“No frames received” - Check power and connections
“Invalid image data” - Check for interference or damage

Maintenance

Regular Maintenance

Visual inspection
Check for obstructions
Verify image quality
Clean camera lens
Check mounting stability
Test focus

Cleaning Procedures

Camera Lens:

Power off robot
Use soft, lint-free cloth
Clean gently to avoid scratches
Remove dust and debris
Verify clear view

Mounting Area:

Clean mounting surface
Check for loose connections
Verify stability

Performance Optimization

Image Processing

Resolution:

Adjust resolution for performance
Balance quality vs. speed
Consider bandwidth requirements

Frame Rate:

Adjust frame rate as needed
Balance performance vs. accuracy
Consider computational load

Parameter Tuning

Camera Parameters:

# In launch file
image_width: 640
image_height: 480
framerate: 30

Processing Parameters:

Adjust processing algorithms
Optimize for real-time performance
Balance accuracy vs. speed

For LiDAR operations, see LiDAR Operations For IMU operations, see IMU Operations For troubleshooting, see Troubleshooting