BEGIN:VCALENDAR VERSION:2.0 PRODID:Linklings LLC BEGIN:VTIMEZONE TZID:Asia/Seoul X-LIC-LOCATION:Asia/Seoul BEGIN:STANDARD TZOFFSETFROM:+0900 TZOFFSETTO:+0900 TZNAME:KST DTSTART:18871231T000000 DTSTART:19881009T020000 END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTAMP:20230103T035309Z LOCATION:Room 324\, Level 3\, West Wing DTSTART;TZID=Asia/Seoul:20221207T140000 DTEND;TZID=Asia/Seoul:20221207T153000 UID:siggraphasia_SIGGRAPH Asia 2022_sess159_papers_302@linklings.com SUMMARY:UniColor: A Unified Framework for Multi-Modal Colorization with Tr ansformer DESCRIPTION:Technical Communications, Technical Papers\n\nUniColor: A Unif ied Framework for Multi-Modal Colorization with Transformer\n\nHuang, Zhao , Liao\n\nWe propose the first unified framework UniColor to support color ization in multiple modalities, including both unconditional and condition al ones, such as stroke, exemplar, text, and even a mix of them. Rather th an learning a separate model for each type of condition, we introduce a tw o-stage colorization framework for incorporating various conditions into a single model. In the first stage, multi-modal conditions are converted in to a common representation of hint points. Particularly, we propose a nove l CLIP-based method to convert the text to hint points. In the second stag e, we propose a Transformer-based network composed of Chroma-VQGAN and Hyb rid-Transformer to generate diverse and high-quality colorization results conditioned on hint points. Both qualitative and quantitative comparisons demonstrate that our method outperforms state-of-the-art methods in every control modality and further enables multi-modal colorization that was not feasible before. Moreover, we design an interactive interface showing the effectiveness of our unified framework in practical usage, including auto matic colorization, hybrid-control colorization, local recolorization, and iterative color editing. Our code and models are available at https://luc kyhzt.github.io/unicolor.\n\nRegistration Category: FULL ACCESS, ON-DEMAND ACCESS\n\nLanguage: ENGLISH\n\nFormat: IN-PERSON, ON-DEMAND URL:https://sa2022.siggraph.org/en/full-program/?id=papers_302&sess=sess15 9 END:VEVENT END:VCALENDAR