Vision-Based Autonomous Quadrotor Navigation
Developed a vision-based autonomous navigation pipeline combining monocular perception, pose estimation, trajectory optimization, and feedback control for waypoint navigation.
A Crazyflie has to fly waypoint trajectories — circles, lemniscates, U-turns — using a monocular camera and IMU, not a perfect state oracle. Motion-capture ground truth exists in the lab, but the onboard stack has to estimate pose, see gates, plan, and track.
The gap between a CasADi trajectory that is feasible in FalconGym and a PD controller that holds that trajectory on a real quadrotor is sensor delay, visual-odometry drift, and the fact that a 30-gram platform punishes lag immediately.
- Built a ROS2 monocular stack fusing IMU, visual odometry, and YOLOv8 detection for adaptive pose estimation in Nerfstudio.
- Planned trajectories with CasADi in FalconGym, then tracked them with PD control on the real Crazyflie.
- Verified 100% trajectory success against Vicon, with mean goal error below 8 cm on the evaluated waypoint sets.
Perception
Monocular camera plus YOLOv8 detection of gates and landmarks. Visual odometry supplies the pose the planner and controller consume when Vicon is not the runtime source of truth.
Localization / State Estimation
IMU, visual odometry, and detections are fused for real-time adaptive pose estimation. Vicon is used as ground truth for evaluation, not as a crutch in the reported success numbers.
Planning
CasADi generates dynamically feasible trajectories in FalconGym (circle, lemniscate, U-turn) before the same references are tracked on hardware.
Control
A PD controller tracks the planned path in position and yaw. On the vision-based circle and lemniscate demos, success rate was 100% with mean goal error under 8 cm.
Sim-to-real on a quadrotor is a delay problem as much as a dynamics problem. A trajectory that looks smooth in FalconGym still has to survive VO drift and a control loop that cannot wait for a perfect pose.
Gate detection is only useful if it is expressed in the same frame as the trajectory. Most of the integration work was keeping perception, pose, and PD control on one clock.