Robotics / Embedded vision
A robot that could find a person, follow them, and keep its distance.
I built a four-wheel robot that paired camera-based human tracking with independent distance sensing. A Raspberry Pi 5 handled perception; an STM32 handled the motors.
The build
The goal was to detect a person, keep them centered in the camera view, and maintain a set distance as they moved. I used the Pi Camera 3 for visual input and a time-of-flight sensor for distance feedback, then connected that information to the drivetrain through the STM32.
The demonstration also had a mechanical response: after the person stopped for two seconds, a flywheel mechanism launched foam balls. That made the project about more than following a bounding box—it also involved coordinating perception, movement, and a timed behavior.
System architecture
I separated the camera and sensor processing from the physical motor-control layer. The Pi interpreted what the robot was seeing; the STM32 operated the drivetrain and mechanism.
Pi Camera 3 over its ribbon connection.
VL53L1X ToF sensor over I²C.
Human detection, tracking information, and interpretation of range measurements.
Target presence, horizontal error, distance, measurement quality, and command information.
Motor-control outputs through the driver boards, plus coordination of the demo mechanism.
The project presentation identifies TB6612FNG motor drivers in the STM32 subsystem. The diagrams below are the original presentation artifacts, not a verified wiring guide.
Computer vision
My presentation lists Detector.py → MobileNet-SSD. This was the person-detection part of the system: camera frames went through the detector, and the detected person's bounding box supplied position information for tracking.
MobileNet is a lightweight convolutional neural-network architecture designed for mobile and embedded vision. SSD stands for Single Shot MultiBox Detector; it predicts object categories and bounding boxes in one network pass. Together, they provide a compact object-detection approach.
Background: original MobileNets paper ↗ · original SSD paper ↗
JOSH MURZELLODetector.pyMobileNet-SSD detectionTracker.pyTracking layerTof_sensor.pyTime-of-flight sensingMain.pyProgram coordinationFeedback & behavior
The horizontal offset of the detected person's bounding box from the center of the camera view supplied a centering error. As the person moved across the frame, that error supplied the information needed to correct the robot's heading.
The ToF sensor added a direct distance measurement. It let the robot maintain a following distance using range feedback, instead of treating the size of a person's image as a distance measurement.
I used PID control to adjust motor response as the tracking error changed. The STM32 operated the motors while the Pi handled the camera and sensor processing. Getting turning corrections to behave well was part of joining those two layers.
What took work
The Pi 5 needed more current than expected during integration. I used external power for the Pi, making power distribution a separate problem from controlling the motors.
Only certain STM32 pins could be configured for the required UART connection. The communication link had to fit the board's available peripheral configuration.
Tracking needed to be accurate and robust without becoming too computationally demanding. Running the vision work on the Pi made that tradeoff part of the system design.
The ToF sensor was not always aligned with the person seen by the camera. The presentation also records stale readings on poorly reflective surfaces—finding a person in an image did not automatically mean getting a useful range measurement.
Project documentation
Expand a slide, then select it to view the full-size image.
