EP 3252566 B1 20210106 - FACE AND EYE TRACKING AND FACIAL ANIMATION USING FACIAL SENSORS WITHIN A HEAD-MOUNTED DISPLAY
Title (en)
FACE AND EYE TRACKING AND FACIAL ANIMATION USING FACIAL SENSORS WITHIN A HEAD-MOUNTED DISPLAY
Title (de)
GESICHTS- UND AUGENVERFOLGUNG SOWIE GESICHTSANIMATION MIT GESICHTSSENSOREN IN EINER KOPFMONTIERTEN ANZEIGE
Title (fr)
SUIVI DES YEUX ET DU VISAGE ET ANIMATION FACIALE À L'AIDE DE CAPTEURS DE VISAGE DANS UN VISIOCASQUE
Publication
Application
Priority
- US 201615172484 A 20160603
- US 201615172473 A 20160603
- US 2016046375 W 20160810
Abstract (en)
[origin: EP3252566A1] A head mounted display (HMD) in a VR system includes sensors for tracking the eyes and face of a user wearing the HMD. The VR system records calibration attributes such as landmarks of the face of the user. Light sources illuminate portions of the user's face covered by the HMD. In conjunction, facial sensors capture facial data. The VR system analyzes the facial data to determine the orientation of planar sections of the illuminated portions of face. The VR system aggregates planar sections of the face and maps the planar sections to landmarks of the face to generate a facial animation of the user, which can also include eye orientation information. The facial animation is represented as a virtual avatar and presented to the user. A facial tracking system generates a virtual rendering of a portion of a face of a user wearing a head-mounted display (HMD). The facial tracking system illuminates portions of the face inside the HMD. The facial tracking system captures a plurality of facial data of the portion of the face using one or more facial sensors located inside the HMD. A plurality of planar sections of the portion of the face are identified based at least in part on the plurality of facial data. The plurality of planar sections are mapped to one or more landmarks of the face. Facial animation information is generated based at least in part on the mapping, the facial animation information describing a portion of a virtual face corresponding to the portion of the user's face.
IPC 8 full level
G06F 3/01 (2006.01); G02B 27/01 (2006.01); G06K 9/00 (2006.01); H04N 5/74 (2006.01)
CPC (source: EP)
G02B 27/017 (2013.01); G06F 3/011 (2013.01); G06F 3/012 (2013.01); G06F 3/013 (2013.01); G06V 20/20 (2022.01); G06V 40/165 (2022.01); G02B 2027/014 (2013.01); G02B 2027/0187 (2013.01); G06F 2203/011 (2013.01)
Designated contracting state (EPC)
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
DOCDB simple family (publication)
DOCDB simple family (application)
EP 16200100 A 20161122