Vision-Based Control of Unmanned Aerial Systems Swarm (NSF-REU)
Completed
- Quadrotor manual control in simulation
- Modifying 3D model to fit needs for task
- Green blob/ball bit mask using OpenCV
- Find edges of object to calculate radius and centroid
- Threshold boundaries for object tracking with movement policies for following
- Relative localization using radius size to maintain a consistent distance
- Multi-agent proof of concept by manipulating threshold boundaries instead of symmetrical by default (move the center square and change its size)
- Threading in simulation for better performance
- Created a movement prediction model which uses a calculated ‘momentum’ using a history of center points
- Implementing multithreaded solutions for more snappy visual feedback for dji tello implementation
- Formation Change in Simulation
- Trained Drone detection YOLOv8 model on 16,000 drone pictures, and fine-tuned on hundreds of DJI Tello Drones at angles likely for a follower to see Dataset1 Dataset2
- YOLOv8 has much lower latency for real-time applications than YOLO-NAS, another state of the art model
- Implemented new 3D model that is easier to be recognized by Drone Detection Model
- Implemented tracking based on detected drone in Microsoft Airsim. Uses BoT-SORT
- Physical Implementation of YOLO model with formation change demonstrated successfully
- Line Follower system for Leader Drone
Before and After Fine-Tuning YOLOv8 Model
Physical Demo Videos, click images to open youtube link (top is most recent)
YOLOv8 Monocular Leader-Follower Drone Swarm with Formation Change
Momentum based prediction for monocular drone formation
Demo Program running, relative localization of a follower agent, using momentum based prediction
Simulation Demo Videos, click images to open youtube link (top is most recent, more to come)
Drone Following using YOLOv8 Microsoft Airsim Demo
Formation Change to Maneuver Swarm Through a Narrow Cooridor
momentum predictor implemented
Early Videos (Archive)
Weighted vs unweighted momentum prediction test
Momentum Test using DJI Tello Camera
Leader follower demo
With threading
Without threading
