BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Seoul
X-LIC-LOCATION:Asia/Seoul
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:KST
DTSTART:18871231T000000
DTSTART:19881009T020000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20230103T035307Z
LOCATION:Auditorium\, Level 5\, West Wing
DTSTART;TZID=Asia/Seoul:20221206T100000
DTEND;TZID=Asia/Seoul:20221206T120000
UID:siggraphasia_SIGGRAPH Asia 2022_sess153_papers_191@linklings.com
SUMMARY:Video-driven Neural Physically-based Facial Asset for Production
DESCRIPTION:Technical Papers\n\nVideo-driven Neural Physically-based Facia
 l Asset for Production\n\nZhang, Zeng, Zhang, Lin, Cao...\n\nProduction-le
 vel workflows for producing convincing 3D dynamic human faces have long re
 lied on an assortment of labor-intensive tools for geometry and texture ge
 neration, motion capture and rigging, and expression synthesis. Recent neu
 ral approaches automate individual components but the corresponding latent
  representations cannot provide artists with explicit controls as in conve
 ntional tools. In this paper, we present a new learning-based, video-drive
 n approach for generating dynamic facial geometries with high-quality phys
 ically-based assets. Two key components are well-structured latent spaces 
 due to dense temporal samplings from videos and explicit facial expression
  controls to regulate the latent spaces. For data collection, we construct
  a hybrid multiview-photometric capture stage, coupling with ultra-fast vi
 deo cameras to obtain raw 3D facial assets. We then set out to model the f
 acial expression, geometry and physically-based textures using separate VA
 Es where we impose a global multi-layer perceptron (MLP) based expression 
 mapping across the latent spaces of respective networks, to preserve chara
 cteristics across respective attributes while maintaining explicit control
 s over facial geometry and texture generation. We also introduce the idea 
 to model the delta information as wrinkle maps for the physically-based te
 xtures in our texture VAE, achieving high-quality 4K rendering of dynamic 
 textures. We demonstrate our approach in high-fidelity performer-specific 
 facial capture and cross-identity facial motion transfer and retargeting. 
 In addition, our multi-VAE-based neural asset, along with the fast adaptat
 ion schemes, can also be deployed to handle in-the-wild videos. Besides, w
 e motivate the utility of our explicit facial disentangling strategy by pr
 oviding various promising physically-based editing results like geometry a
 nd material editing or wrinkle transfer with high realism. Comprehensive e
 xperiments show that our technique provides higher accuracy and visual fid
 elity than previous video-driven facial reconstruction and animation metho
 ds.\n\nRegistration Category: FULL ACCESS, EXPERIENCE PLUS ACCESS, EXPERIE
 NCE ACCESS, TRADE EXHIBITOR\n\nLanguage: ENGLISH\n\nFormat: IN-PERSON
URL:https://sa2022.siggraph.org/en/full-program/?id=papers_191&sess=sess15
 3
END:VEVENT
END:VCALENDAR