Artistic character generation technique using a controllable diffusion model

다국어 초록 (Multilingual Abstract)

With the recent advent of Metaverse, the character industry that reflects the characteristics of users' faces is drawing attention. there is a hassle that users have to select face components such as eyes, nose, and mouth one by one. In this paper, we propose a diffusion-based model that automatically generates characters from content human photographs. Our model generates user artistic characters by reflecting content information such as face angle, direction, and shape of a content human photo. In particular, our model automatically analyzes detailed information such as glasses and whiskers from content photo images and reflects them in artistic characters generated. Our network generates the final character through a three-step: diffusion process, UNet, and denoising processes. We use image encoders and CLIP encoders for the connection between style and input data. In the diffusion process, a collection of noise vectors is gradually added to a style vector to enable lossless learning of the detailed styles. All input values except for the style images are vectorized with CLIP encoders and then learned with noise style vectors in the UNet. Subsequently, noise is removed from the vectors through the UNet to obtain the artistic character image. We demonstrate our performance by comparing the results of other models with our results. Our method reflects content information without loss and generates natural high-definition characters.

참고문헌 (Reference)

1 리징하오 ; 이병춘, "자동 생성 애니메이션에 효율적인 토폴로지 연구" 한국만화애니메이션학회 (71) : 31-52, 2023

2 최종인 ; 강신진, "앙상블 엔진: 비주얼 노벨 게임 제작을 위한 프레임워크 설계" 한국컴퓨터정보학회 24 (24): 11-17, 2019

3 최정우 ; 김지훈 ; 최인호 ; 서경진, "스캔 기반의 게임 캐릭터로 만든 실사형 얼굴 랜드마크 데이터세트" 한국디지털콘텐츠학회 23 (23): 2259-2268, 2022

4 전효경 ; 조동민, "근로환경 변화에 따른 딥러닝 클라우드 서비스 기술을 활용한 캐릭터 애니메이션 리타겟팅에 관한 연구" 한국멀티미디어학회 26 (26): 753-760, 2023

5 Jae-Yong Seo, "Stable Diffusion-based Automatic Game Avatar Generation and Recommendation Method" 739-742, 2023

6 Rombach, R., "High-resolution image synthesis with latent diffusion models" 10684-10695, 2022

7 Jang So Young, "Conditional Music Composition Model Leveraging the Latent Space based on Music VAE" 410-411, 2023

8 Feng Wang, "Application of Stable Diffusion in Game Character Design and Improvement Strategies" 99-100, 2023

9 백종열, "3D애니메이션제작의 효율성 향상을 위한 오토 리깅 툴의 활용에 대한 연구" 한국만화애니메이션학회 (49) : 247-265, 2017

10 김윤정, "3D 캐릭터 얼굴 모델의 폴리곤 리덕션 연구" 한국애니메이션학회 14 (14): 138-152, 2018