Eye tracking

Eye tracking in AR (augmented reality) and VR (virtual reality) refers to the process of measuring and responding to where and how a user's eyes are directed.

It improves the interactivity and immersion of AR and VR experiences by enabling systems to understand and respond to the user's gaze.

How does eye tracking work?

• Sensors and cameras: Eye tracking systems use infrared sensors and cameras to monitor the position and movement of the eyes.
• Data processing: The data collected by the sensors is processed in real time to determine where the user is looking in the virtual environment.
• Software algorithms: Advanced algorithms analyze the eye movement data to interpret the user's intentions and interests.

Tobii develops eye tracking solutions, including for the HTC Vive Pro Eye.

Eye Tracking Methods for XR Devices

In Extended Reality (XR) devices, eye-tracking technologies are becoming increasingly important to improve user interaction. The most common eye tracking methods used in XR devices are

Pupil-centered corneal reflex (PCCR)

• How it works: This method measures the distance between the center of the pupil and the reflection of light (usually infrared light) on the cornea.
• How it is used: This is one of the most commonly used eye tracking methods in XR devices due to its accuracy and robustness. It is used in high-end devices like the HTC Vive Pro Eye.
• Benefits: High accuracy and stability even with small head movements.

Dark and light pupil detection

• How it works: This method uses infrared light to distinguish between the reflection of the iris and the pupil. In dark pupil detection, the pupil is detected as a dark spot; in light pupil detection, the reflected light makes the pupil appear lighter.
• Application: This method is used in XR devices to detect fast movements and eye movements. The concept is used in devices with an integrated camera and infrared light.
• Advantages: Efficient for fast eye movements, useful in dynamic environments.

Foveated rendering

• How it works: This method uses eye tracking to determine the direction of gaze and renders only the area the user is focusing on in high resolution.
• Applications: Foveated rendering is increasingly used in modern VR headsets such as PlayStation VR2 and Oculus Quest Pro.
• Benefit: Better performance and graphics quality because less resources are used for unimportant areas of the image.

EOG (Electro-oculography)

• How it works: This method measures the electrical potentials produced by eye movements. Electrodes are placed near the eyes to pick up these signals.
• How it's used: EOG is used in some specialized research equipment, but it is not as widely used as optical methods in XR equipment.
• Advantages: Works in environments without adequate lighting because it does not rely on optical detection.

Video Oculography (VOG)

• How it works: This method uses cameras to track eye movements. Specially placed cameras in the headset detect eye movements by analyzing the position of the pupil and the eye.
• Uses: VOG is one of the most commonly used eye tracking methods in VR and AR headsets because it is an accurate and cost-effective solution for eye tracking.
• Advantages: Good accuracy and relatively inexpensive implementation in VR devices.

Optical Eye Tracking Systems Using Infrared Light

• How it works: Infrared light is used to create reflections on the eye that are captured by cameras. The method uses both the light reflected from the cornea and the movement of the pupil to determine the direction of gaze.
• Application: This method is often used in XR headsets because it is independent of ambient light and works reliably even in low light conditions.
• Benefits: High accuracy, robust to ambient light and head movement.

Applications of eye tracking in XR

Advanced user interfaces: In VR and AR, eye tracking enables hands-free interaction with the environment.

• Immersion and realism: Thanks to eye tracking, VR and AR systems can adapt to the environment.
• Performance Optimization: Foveated rendering focuses graphics processing on areas where the user's gaze is directed, reducing the amount of computing required to render peripheral vision in detail.
• Analysis and research: Eye tracking provides valuable data on user engagement and behavior.

Eye tracking significantly improves the functionality and user experience of AR and VR technologies by making systems more responsive to the user's natural behavior and visual focus.