Formal Language Constraints in Deep Reinforcement Learning for Self-driving Vehicles

Authors

• Tyler Bienhoff, University of Nebraska-LincolnFollow

First Advisor

Stephen D. Scott

Date of this Version

7-30-2020

Document Type

Thesis

Citation

@masterthesis{Bienhoff20,

author = {Tyler Bienhoff},

title = {Formal Language Constraints in Deep Reinforcement Learning for Self-Driving Vehicles},

school = {University of Nebraska-Lincoln},

year = {2020},

month = {July}

}

Comments

A thesis presented to the faculty of the Graduate College at the University of Nebraska in partial fulfillment of requirements for the degree of Master of Science

Major: Computer Science

Under the supervision of Professor Stephen D. Scott. Lincoln, Nebraska, August 2020

Copyright 2020 Tyler Bienhoff

Abstract

In recent years, self-driving vehicles have become a holy grail technology that, once fully developed, could radically change the daily behaviors of people and enhance safety. The complexities of controlling a car in a constantly changing environment are too immense to directly program how the vehicle should behave in each specific scenario. Thus, a common technique when developing autonomous vehicles is to use reinforcement learning, where vehicles can be trained in simulated and real-world environments to make proper decisions in a wide variety of scenarios. Reinforcement learning models, however, have uncertainties in how the vehicle acts, especially in a previously unseen situation that can lead to dangerous situations with humans onboard or nearby. To improve the safety of the agent, we propose formal language constraints that augment a standard reinforcement learning agent while being trained in a simulated self-driving environment. The constraints help the vehicle navigate turns and other situations by penalizing the agent when an action is chosen that could lead to a dangerous situation such as a collision. Empirically, we show that the agent, with these constraints, has a slight performance improvement as well as a significant decrease in collisions. Future work can expand upon the current constraints and evaluate using different reinforcement learning algorithms with constraints for training the self-driving agent.

Advisor: Stephen D. Scott