When Elephant Meets Duck : A Disaggregated HTAP Architecture for Cost-Effective and High-Performant Analytics on Data Lakehouse = 데이터레이크하우스에서 비용효율적 고성능 분산 HTAP 아키텍처 연구

다국어 초록 (Multilingual Abstract)

This thesis presents Elephant Meets Duck, a disaggregated HTAP architecture that mitigates OLTP/OLAP interference while improving cost efficiency in a lakehouse setting. The proposed stack places mutable operational state in PostgreSQL (hot store), migrates finalized historical records into Apache Iceberg on object storage (cold store), and executes analytical queries through DuckDB as a federated, vectorized compute head. The key idea is mutability-based tiering, where data is moved to the cold tier only after it becomes effectively immutable (e.g., update-once finalized), rather than continuously replicating the entire transactional dataset into an OLAP replica.
A central contribution is a correctness contract based on atomic publication and monotonic watermarks: watermarks advance only after a successful Iceberg commit, and hybrid queries route records across tiers using disjoint predicates, ensuring no duplicates and no gaps despite lagging physical cleanup. We evaluate the system using py-tpcc with CHbenchmark-style OLAP injection, comparing a PostgreSQL-only baseline against the Hybrid mode that routes OLAP to DuckDB over Iceberg. The results show that Hybrid consistently improves TPS under analytical injection (e.g., ~23.2% at p=1% and ~14.8% at p=10% for W=1, C=1), demonstrating that disaggregated federation with lifecycle-aware tiering can effectively reduce OLTP/OLAP contention in mixed workloads.

국문 초록 (Abstract)

본 논문은 HTAP 환경에서 OLTP/OLAP 성능 간섭과 복제 기반 설계의 비용 증폭(스토리지·동기화 오버헤드)을 완화하기 위해 데이터레이크하우스 위에서 동작하는 분산형 HTAP 아키텍처 Elephant Meets Duck을 제안한다. 제안 시스템은 가변(Hot) 데이터는 PostgreSQL에 유지하고, 더 이상 변경되지 않는 불변(Cold) 데이터는 Apache Iceberg 테이블로 오브젝트 스토리지에 저장하며, DuckDB를 연합(federation) 분석 엔진으로 사용해 두 계층을 단일 SQL로 통합 조회한다. 또한 Iceberg 스냅샷 커밋을 활용한 원자적 발행과 워터마크 기반 라우팅을 통해 조회 결과의 중복과 누락을 방지하는 정확성 계약을 제시하고, py-tpcc(TPC-C)기반의 CHbenchmark 벤치마크(OLAP 주입) 실험에서 PostgreSQL 단독(Naive) 대비 Hybrid 엔진이 TPS를 일관되게 개선함을 확인하여 혼합 워크로드에서의 간섭 완화 효과를 입증하였다.

목차 (Table of Contents)

• Chapter 1. Introduction 1
• Chapter 2. Background 7
• Chapter 3. TPC-C Data Characteristics and Quantitative Analysis of Log-Like Tables 10
• Chapter 4. Related Work and Industry Trends 16
• Chapter 5. System Design 22
• Chapter 6. Experiments 28
• Chapter 7. Conclusion 33
• Bibliography 37
• Abstract in Korean 38