NPMI와 TF-IDF를 고려한 자동 불용어 생성 기법이 의미론적 일관성에 미치는 영향: 태권도 연구를 중심으로 = The Influence of NPMI and TF-IDF-Based Automatic Stopword Generation on Semantic Consistency

다국어 초록 (Multilingual Abstract)

PURPOSE This study optimized stopword removal to enhance topic modeling performance. We propose an objective method combining normalized pointwise mutual information (NPMI) with median-based term frequency–inverse document frequency (TF–IDF) to automatically generate stopwords. METHODS Using text data from 443 research papers on “Taekwondo sparring,” we selected stopword candidates based on NPMI and identified 30 words with the lowest TF–IDF scores. We examined the impact of removing 1–30 stopwords on u_mass coherence scores. RESULTS The NPMI–TF–IDF method significantly improved coherence (R2 = .456; p < .001).
However, excessive removal led to diminishing returns, with the optimal coherence score (−11.442) achieved at 200 stopwords. In contrast, manually selected stopwords yielded a lower coherence score (−16.001). The findings indicate that integrating TF– IDF with NPMI effectively preserves meaningful words and outperforms PMI2 and PMI3 approaches. CONCLUSIONS Manual stopword selection can reduce reproducibility.
Optimizing stopword removal based on domain-specific characteristics is essential.
Future research should validate this method across diverse fields to establish a more generalizable standard.