MobileEYE: deep learning based mobile device eye tracking solution for dynamic visuals

Abstract

With the growing use of mobile devices in dynamic visual environments, such as gaming and augmented reality, the demand for accurate and real-time eye tracking has become increasingly important. Existing commercial eye trackers are often limited by high costs, calibration requirements and the inability to work efficiently on mobile platforms. This research focuses on developing an appearance-based eye-tracking algorithm, MobileEYE, designed for mobile devices to track human gaze accurately while interacting with dynamic visual content, such as video clips or games.

The first part of this research identifies the most suitable deep-learning architecture for mobile eye tracking. Several lightweight CNN architectures were evaluated on static visual stimuli, with MobileNet-V3 reducing error by 16.96% and improving inference speed by 93.61% on a Samsung Galaxy S9 compared to AlexNet. Using depthwise and pointwise convolutions from MobileNet-V3, one Convolutional Neural Network (CNN) architecture and two hybrid architectures were developed.

The second aspect of the research involved integrating mobile edge computing to im- prove the performance of the MobileEYE algorithm on resource-constrained devices. According to the results, transitioning from on-device to edge-based inference increased inference time by 38.07%, considering both inference and communication time, but reduced memory and energy consumption on the mobile device. Deploying the models on edge devices reduced inference time to 167.33 ms per frame on the Intel NUC, with the most energy-efficient configuration consuming 0.1104 mWh per frame on the Odroid N2+. Quantisation and pruning techniques were also employed to optimise the model, reducing inference time by 27.91% at the cost of an 11.03% increase in error.

Lastly, sensitivity analysis compared MobileEYE to the Tobii Pro Nano across conditions such as lighting, age and device types. MobileEYE showed competitive performance in high-light environments, with a 13.9% lower error in high-light conditions (15.81 mm) than in low light (18.01 mm) and on iPhone 14 devices, with a 10.3% lower error compared to Samsung S22. The findings highlight the potential of MobileEYE for applications in gaming, healthcare and education while also identifying areas for future improvements in real-time performance and adaptability across diverse conditions.

Date of Thesis submission	2025
Original language	English
Awarding Institution	Western Sydney University
Supervisor	Anupama Ginige (Supervisor)