Apple files patent for mixed reality headsets that can track your facial expressions

Apple has filed a patent application for a mixed reality headset, which seems to include futuristic features like a head-mounted display, facial expression tracker, and a controller to combine augmented reality and virtual reality. The application, titled “Display System Having Sensors”, was first spotted by Variety, which reported that Apple’s mixed reality headset would provide “3D virtual views of a user’s environment augmented with virtual content.” The application was first filed in March 2019 and published last week. The filing details Apple’s plans to use a range of sensors to gather data from the wearer of a mixed reality headset. Per the report, the headset could come with left and right displays, light sensors to collect information like color, intensity and direction, eyebrow sensors to track facial expressions, world mapping sensors to track movement and location, eye-tracking sensors, lower jaw sensors to track expression, head pose sensors to track a user’s orientation and motion, hand sensors to track position, movement and gestures, an inertial measurement unit to augment the sensor information, and left and right cameras. [caption id=“attachment_6762541” align=“alignnone” width=“1024”]  Apple CEO Tim Cook speaks during Apple’s annual Worldwide Developers Conference in San Jose, California. Reuters[/caption] The patent application notes that the eye-tracking cameras can also be used for biometric authentication. Apple may also use cameras for gesture tracking, according to the patent application. As in the case of **Animoji** , Apple would likely combine data from these sensors to more realistically reproduce a user’s facial expression in **mixed reality** . [caption id=“attachment_7040721” align=“alignnone” width=“1280”]  Image: Patent Application Publication[/caption] If you are wondering why Apple is filing a patent for this technology now if it has already been using the same tech behind emoji, you are forgetting that the challenge with the headset is that you can’t simply film a user’s face because they will have the headsets on. Apple was first **reported to be working on a mixed-reality headset back in April last year** . Apple was then reported to be working on a headset with an 8K display for each eye. The project was expected to be released in 2020. This is interesting considering how Tim Cook has always maintained that  **AR is the future, and not VR** . But hey, maybe Mixed Reality is a good compromise.