Agricultural Machinery Operator Monitoring System (Ag-OMS): A Machine Learning Approach for Real-Time Operator Safety Assessment

Authors

Terence Irumva, University of Nebraska-LincolnFollow
Herve Mwunguzi, University of Nebraska-LincolnFollow
Santosh Pitla, University of Nebraska-LincolnFollow
Bethany Lowndes, University of Nebraska Medical CenterFollow
Aaron Yoder, University of Nebraska Medical CenterFollow
Ka-Chun Siu, University of Nebraska Medical CenterFollow

Document Type

Article

Date of this Version

2023

Citation

Journal of Agricultural Safety and Health 29(2): 85-97

https://doi.org/10.13031/jash.15357

Comments

© 2023 ASABE. Used by permission.

Abstract

The 2015 CS-CASH (Central States Center for Agricultural Safety and Health, 2015) Injury Surveillance Surveys showed that around 19% of injuries to agricultural producers are related to tractors or large agricultural machinery, yet only a limited number of studies are found that address tools and methods for monitoring safety behaviors of agricultural machinery operators in real-time. The current safety behavior monitoring approaches require an in-person presence, which can be both time- and cost-inefficient, and the other available methods lack a feedback element to alert operators in realtime. As a result, the research presented in this study aimed to develop an automated approach to monitoring tractor operators' safety behaviors through the use of a trained machine learning (ML) model and a feedback system to alert operators when they engage in unsafe practices. For the ML model development, a skeleton-detecting algorithm called OpenPose was used to detect real-time human postures in a livestreaming video feed from a camera installed in the tractor cab. The model was then trained on three separate categories of tractor operators’ safety operating behaviors, and this trained classifier was used to label operators’ safety behaviors in real time based on the three safety classes. A feedback mechanism controlled by an onboard microcontroller was then used to alert the operators when unsafe operating behavior was detected to facilitate safe practices. This monitoring system, named Ag-OMS (Agricultural Machinery Operators Monitoring System), monitored the ingress/egress operators’ behaviors in real-time entering and exiting the tractor cab. The Ag-OMS successfully identified the ingress/egress operators’ behaviors with an accuracy of 97% on the testing datasets for all safety risk categories.