RISS 학술연구정보서비스

검색

인기 검색어

    다국어 입력

    http://chineseinput.net/에서 pinyin(병음)방식으로 중국어를 변환할 수 있습니다.

    변환된 중국어를 복사하여 사용하시면 됩니다.

    예시)
    • 中文 을 입력하시려면 zhongwen을 입력하시고 space를누르시면됩니다.
    • 北京 을 입력하시려면 beijing을 입력하시고 space를 누르시면 됩니다.
    닫기
    KCI등재

    화자-청자 이원 상호작용에서 멀티모달 신호를 활용한 동적 공감 반응 생성 모델 = A Multimodal Dynamic Empathetic Response Generation Model for Speaker–Listener Dyadic Interactions

    한글로보기

    https://www.riss.kr/link?id=A110338422

    • 0

      상세조회
    • 0

      다운로드
    서지정보 열기
    • 내보내기
    • 내책장담기
    • 공유하기
      • URL 복사
    • 오류접수
    인용문이 복사되었습니다.

    부가정보

    다국어 초록 (Multilingual Abstract) kakao i 다국어 번역

    The rapid growth of online and remote interactions has increased demand for AI systems that can deliver emotionally supportive, empathetic responses, yet many conversational agents still fail to reflect the richness of human empathy in face-to-face, multimodal settings. This study proposes a multimodal empathetic response generation framework that explicitly models emotional dynamics in dyadic interactions to produce dynamic empathetic speaking videos. Given a speaker’s audio and video, the framework generates an empathizer’s responses across linguistic, acoustic, and visual modalities. The framework integrates three modules. First, a Video Conversation (ViCo)-based facial representation module encodes the speaker’s expression dynamics as 3D Morphable Model (3DMM) coefficients and predicts the empathizer’s facial expressions and head movements to enhance visual empathy. Second, an AnyGPT- based module generates semantically coherent and emotionally appropriate empathetic utterances from acoustic features. Third, to mitigate temporal mismatch between generated speech and facial motion, a lip-synchronization module (Wav2Lip) aligns lip movements with audio and produces natural conversational videos. Quantitative and qualitative evaluations show that the proposed framework generates more empathetic, dynamic, and context-consistent responses, highlighting the importance of multimodal integration for emotionally expressive, human-centered AI.
    번역하기

    The rapid growth of online and remote interactions has increased demand for AI systems that can deliver emotionally supportive, empathetic responses, yet many conversational agents still fail to reflect the richness of human empathy in face-to-face, m...

    The rapid growth of online and remote interactions has increased demand for AI systems that can deliver emotionally supportive, empathetic responses, yet many conversational agents still fail to reflect the richness of human empathy in face-to-face, multimodal settings. This study proposes a multimodal empathetic response generation framework that explicitly models emotional dynamics in dyadic interactions to produce dynamic empathetic speaking videos. Given a speaker’s audio and video, the framework generates an empathizer’s responses across linguistic, acoustic, and visual modalities. The framework integrates three modules. First, a Video Conversation (ViCo)-based facial representation module encodes the speaker’s expression dynamics as 3D Morphable Model (3DMM) coefficients and predicts the empathizer’s facial expressions and head movements to enhance visual empathy. Second, an AnyGPT- based module generates semantically coherent and emotionally appropriate empathetic utterances from acoustic features. Third, to mitigate temporal mismatch between generated speech and facial motion, a lip-synchronization module (Wav2Lip) aligns lip movements with audio and produces natural conversational videos. Quantitative and qualitative evaluations show that the proposed framework generates more empathetic, dynamic, and context-consistent responses, highlighting the importance of multimodal integration for emotionally expressive, human-centered AI.

    더보기

    분석정보

    View

    상세정보조회

    0

    Usage

    원문다운로드

    0

    대출신청

    0

    복사신청

    0

    EDDS신청

    0

    동일 주제 내 활용도 TOP

    더보기

    주제

    연도별 연구동향

    연도별 활용동향

    연관논문

    연구자 네트워크맵

    공동연구자 (7)

    유사연구자 (20) 활용도상위20명

    해외이동버튼