Computing, School of
School of Computing: Dissertations, Theses, and Student Research
First Advisor
Marilyn Wolf
Second Advisor
Santosh Pitla
Date of this Version
10-2025
Document Type
Dissertation
Citation
A dissertation presented to the faculty of the Graduate College at the University of Nebraska in partial fulfilment of requirements for the degree of Doctor of Philosophy
Major: Computer Science
Under the supervision of Professors Marilyn Wolf and Santosh Pitla
Lincoln, Nebraska, December 2025
Abstract
Uncrewed Aerial Vehicles (UAVs) are increasingly deployed in dynamic, GPS degraded, and cluttered environments, yet their autonomy remains fundamentally constrained by limitations in onboard perception and real-time control. This dissertation addresses these challenges by proposing a unified framework that co-designs deep learning-based perception and model-based control, organized around three core thrusts: Learn to Track, Learn to Localize, and Learn to Evade.
Learn to Track develops dynamic and adaptive perception control mechanisms that optimize CNN inference for target tracking. A control-aware CNN framework dynamically adjusts inference frequency based on UAV motion, reducing latency while maintaining visual lock. An adaptive CNN with filter pruning achieved the lowest tracking RMSE of 0.8 meters and significantly reduced inference latency and energy consumption. Furthermore, a TD3-based co-design strategy for joint perception and motion control maintained a 2-meter RMSE while reducing filter usage by 44% compared to baselines, achieving robust tracking in evasive scenarios.
Learn to Localize presents a cooperative UAV-UGV localization framework that fuses GNSS data with deep learning-based detections, reducing uncertainty by over 1 meter in XY and 1.5 meters in Z. To improve visual reliability, we propose a perception failure recovery mechanism using LSTM-based recurrent networks that generalize across YOLOv8–YOLOv11. Additionally, this work introduce a 30,000-image UAV-UGV dataset, including 7,500 sequential frames suitable for training Markovian failure recovery models.
Learn to Evade proposes a probabilistic sensor fusion framework that models and mitigates sensor-specific perception failures. Ray-miss events from ToF sensors are handled with grid-based belief propagation, while CNN-based false negatives are smoothed with a Hidden Markov Model. Event camera limitations are addressed with diffusion-based belief retention. These uncertainty models are integrated into an obstacle-aware MPC framework capable of safely navigating around both static and moving UAV obstacles even under degraded sensing conditions. The proposed system achieves accurate trajectory tracking and safe avoidance, making it one of the few efforts to treat UAVs themselves as dynamic obstacles. Collectively, this dissertation advances the reliability, adaptability, and resource efficiency of autonomous UAVs through the synergistic design of perception and control. It provides new methods, datasets, and real-world validation to support robust aerial autonomy in real-time and safety-critical applications.
Advisors: Marilyn Wolf and Santosh Pitla
Comments
Copyright 2025, Krishna Muvva. Used by permission