A Webcam Eye Tracking Infrastructure for Software Engineering Tasks

Authors

• Zachary M. Kozak, University of Nebraska-LincolnFollow

First Advisor

Bonita Sharif

Committee Members

Robert Dyer, Mike Dodd

Date of this Version

5-2026

Document Type

Thesis

Citation

A thesis presented to the faculty of the Graduate College at the University of Nebraska in partial fulfilment of requirements for the degree of Master of Science

Major: Computer Science

Under the supervision of Professor Bonita Sharif

Lincoln, Nebraska, May 2026

Comments

Copyright 2026, Zachary M. Kozak. Used by permission

Abstract

Performing eye tracking utilizing commodity webcams has been explored for over a decade, but limited camera quality and sensitivity to head movements have hindered its adoption in research settings. Recent advances in consumer-grade webcams and machine learning methods present an opportunity to improve the accuracy of webcam eye tracking and extend the feasibility of studies beyond controlled laboratory environments.

Current popular webcam eye tracking methods restrict implementations to the browser and rely on continuous user interactions for calibration, limiting the kinds of studies that can be conducted. This thesis presents a feature-based gaze prediction system that incorporates eye geometry and head pose, specifically the pitch and yaw of the head, into a machine learning model, addressing the sensitivity to natural head movement not addressed in most existing solutions. An ablation study identifies the horizontal iris offset as the most informative feature for gaze prediction, while the vertical component of user gaze requires a more nuanced combination of eyelid distance and openness features. Under free head movement conditions, the proposed tracker outperforms WebGazer, the most widely used open-source webcam tracker, by more than a degree of visual error.

The webcam tracker is integrated into iTrace, an eye tracking infrastructure for software engineering environments, and evaluated in a pilot study ($N=6$) across four software engineering tasks: diagram comprehension, design document reading, code comprehension, and code comparison. The webcam tracker is evaluated against a Tobii Pro Spectrum research-grade eye tracker for these tasks. The webcam tracker achieved a median gaze error of just over 2 degrees of visual angle under controlled conditions. AOI-level behavioral agreement between the two trackers varied by task, with the code comparison task (largest areas of interest - AOIs) yielding the strongest agreement and the diagram task (smallest areas of interest - AOIs) yielding the weakest.

These findings suggest that webcam eye tracking may be a viable tool for software engineering research when areas of interest are large enough to absorb the tracker's error, but it is not yet suitable for studies requiring token-level gaze resolution or higher-level metrics such as fixations or saccades.

Advisor: Bonita Sharif