텍스트 마이닝을 이용한 웹 포럼 불량글 탐지 모델 = The Spam Detection Model for Web Forums using Text Mining Techniques

다국어 초록 (Multilingual Abstract)

The spam in the discussion web forum causes user inconvenience and lowers the value of the web forum as the open source of user opinion. The importance of postings is evaluated in terms of the number of involved authors, so the spam distorts the analysis result by adding the unnecessary data in the opinion analysis. We propose the automatic detection model of spam postings in the web forum. We extract text features of posting contents using text mining techniques from the perspective of linguistics and then perform supervised learning to recognize spam from normal postings. Significant features are derived through the learning process and the automatic detection model is built based on those features. To build the automatic detection model of normal postings and spam, four evaluators are asked to recognize the spam posting in prior. We adopted the Naive Bayesian, Support Vector Machine (SVM), decision tree, which are known to perform well in data and text mining tasks. We can extract the text features to recognize the spam and detect automatically the newly posted spam. We apply the proposed model to the YahooFinace-Walmart forum, which is the world largest Walmart-related web forum.

참고문헌 (Reference)

1 Hayati P., "Toward spam 2.0: An evaluation of Web 2.0 anti-spam methods Industrial Informatics" 875-880, 2009

2 Buckland M., "The relationship between Recall and Precision" 45 (45): 12-19, 1999

3 Gruhl D., "The predictive power of online chatter" KDD 78-87, 2005

4 Vapnik VN., "The nature of statistical learning theory" Springer-Verlag 1995

5 Gillin P., "The New Influencers, A Marketer’s Guide to the New Social Media" Quill Driver Books\Word Dancer Press 2007

6 Robert F., "Syntax. Critical Concepts in Linguistics" Routledge 2006

7 Dunning T., "Statistical Identification of Language" New Mexico State University 94-273, 1994

8 Lin Y., "Splog detection using self-similarity analysis on blogtemporal dynamics" 2007

9 Jindal N., "Opinion Spam and Analysis" WSDM’08 2008

10 Lewis D., "Naive (Bayes) at forty: The independence assumption in information retrieval" Machine Learning 4-15, 1998

11 Morinaga S., "Mining product reputations on the Web" 341 : 2002

12 Zinman A., "Is Britney Spears spam" 2007

13 Quinlan JR., "Induction of decision trees. In Machine Learning"

14 Benevenuto F., "Identifying Video Spammers in Online Social Networks" AIRWeb 2008

15 Gwet K., "Handbook of Inter-Rater Reliability (Second Edition)" ISBN 2010

16 Sampson S., "Gathering customer feedback via the Internet: instruments and prospects" 98 (98): 71-, 1998

17 Glance N., "Deriving Marketing Intelligence from Online Discussion" KDD 2005

18 Han S., "Collaborative blog spam filtering using adaptive percolation search" WWW 2006

19 Mishne G., "Blocking Blog Spam with Language Model Disagreement" AIRWeb 2005

20 Wanas N., "Automatic Scoring of Online Discussion Posts" WICOW 2008

21 Paul K., "Analyzing Grammar: An Introduction" Cambridge University Press 35-, 2005

22 Wenger A., "Analysis of travel bloggers' characteristics and their communication about Austria as a tourism destination" 14 (14): 2008

23 Liu Y, "ARSA: A Sentiment-Aware Model for Predicting Sales Performance Using Blogs" SIGIR 2007

24 Niu Y., "A Quantitative Study of Forum Spamming Using Context-based Analysis" 2007