Sim-to-Real Reinforcement Learning Framework for Autonomous Aerial Leaf Sampling

Authors

Ashraful Islam, University of Nebraska - LincolnFollow

First Advisor

Carrick Detweiler

Date of this Version

5-2023

Citation

Ashraful Islam. Sim-to-Real Reinforcement Learning Framework For Autonomous Aerial Leaf Sampling. Master's thesis, University of Nebraska-Lincoln, 2023

Comments

A THESIS Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfilment of Requirements For the Degree of Master of Science, Major: Computer Science, Under the Supervision of Professor Carrick Detweiler. Lincoln, Nebraska: May, 2023

Copyright © 2023 Ashraful Islam

Abstract

Using unmanned aerial systems (UAS) for leaf sampling is contributing to a better understanding of the influence of climate change on plant species, and the dynamics of forest ecology by studying hard-to-reach tree canopies. Currently, multiple skilled operators are required for UAS maneuvering and using the leaf sampling tool. This often limits sampling to only the canopy top or periphery. Sim-to-real reinforcement learning (RL) can be leveraged to tackle challenges in the autonomous operation of aerial leaf sampling in the changing environment of a tree canopy. However, trans- ferring an RL controller that is learned in simulation to real UAS applications is challenging due to the risk of crashes. UAS crashes pose safety risks to the operator and its surroundings which often leads to expensive UAS repairs.

In this thesis, we present a Sim-to-Real Transfer framework using a computer numerical control (CNC) platform as a safer, and more robust proxy, before using the controller on a UAS. In addition, our framework provides an end-to-end complete pipeline to learn, and test, any deep RL controller for UAS or any three-axis robot for various control tasks. Our framework facilitates bi-directional iterative improvements to the simulation environment and real robot, by allowing instant deployment of the simulation learned controller to the real robot for performance verification and issue identification.

Our results show that we can perform a zero-shot transfer of the RL agent, which is trained in simulation, to real CNC. The accuracy and precision do not meet the requirement for complex leaf sampling tasks yet. However, the RL agent trained for a static target following still follows or attempts to follow more dynamic and changing targets with predictable performance. This works lays the foundation by setting up the initial validation requirements for the leaf sampling tasks and identifies potential areas for improvement. Further tuning of the system and experimentation of the RL agent type would pave the way to autonomous aerial leaf sampling.

Adviser: Carrick Detweiler