The Effects of Content and Distribution of Recommended Items on User Satisfaction: Focus on YouTube

다국어 초록 (Multilingual Abstract)

The performance of recommender systems (RS) has been measured mainly in terms of accuracy. However, there are other aspects of performance that are difficult to understand in terms of accuracy, such as coverage, serendipity, and satisfaction with recommended results. Moreover, particularly with RSs that suggest multiple items at a time, such as YouTube, user satisfaction with recommended results may vary not only depending on their accuracy, but also on their configuration, content, and design displayed to the user. This is true when classifying an RS as a single RS with one recommended result and as a multiple RS with diverse results. No empirical analysis has been conducted on the influence of the content and distribution of recommendation items on user satisfaction. In this study, we propose a research model representing the content and distribution of recommended items and how they affect user satisfaction with the RS. We focus on RSs that recommend multiple items. We performed an empirical analysis involving 149 YouTube users. The results suggest that user satisfaction with recommended results is significantly affected according to the HHI (Herfindahl-Hirschman Index). In addition, satisfaction significantly increased when the recommended item on the top of the list was the same category in terms of content that users were currently watching. Particularly when the purpose of using RS is hedonic, not utilitarian, the results showed greater satisfaction when the number of views of the recommended items was evenly distributed. However, other characteristics of selected content, such as view count and playback time, had relatively less impact on satisfaction with recommended items. To the best of our knowledge, this study is the first to show that the category concentration of items impacts user satisfaction on websites recommending diverse items in different categories using a content-based filtering system, such as YouTube. In addition, our use of the HHI index, which has been extensively used in economics research, to show the distributional characteristics of recommended items, is also unique. The HHI for categories of recommended items was useful in explaining user satisfaction.

참고문헌 (Reference)

1 유석종, "평가 신뢰도를 고려한 협업필터링의 유사도 모델" 한국정보기술학회 14 (14): 141-146, 2016

2 Lee, D, "When do recommender systems work the best?: The moderating effects of product attributes and consumer reviews on recommender performance" International World Wide Web Conferences Steering Committee 85-97, 2016

3 Zhang, S., "What is the optimal power generation mix of China? An empirical analysis using portfolio theory" 229 : 522-536, 2018

4 Cooper, J., "Video games and aggression in children 1" 16 (16): 726-744, 1986

5 Anand, D., "Utilizing various sparsity measures for enhancing accuracy of collaborative recommender systems based on local and global similarities" 38 (38): 5101-5109, 2011

6 Nguyen, T. T., "User personality and user satisfaction with recommender systems" 20 (20): 1173-1189, 2018

7 Zhang, W., "Travel attractions recommendation with travel spatial-temporal knowledge graphs" Springer 213-226, 2018

8 Mehrotra, R., "Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems" ACM 2243-2251, 2018

9 Yoo, W. S., "The role of interactivity in e-tailing : Creating value and increasing satisfaction" 17 (17): 89-96, 2010

10 Anderson, C., "The long tail: Why the future of business is selling less of more" Hachette Books 2006

11 Ferwerda, B, "The influence of users' personality traits on satisfaction and attractiveness of diversified recommendation lists" 43-47, 2016

12 Choi, J., "The influence of social presence on customer intention to reuse online recommender systems : The roles of personalization and product type" 16 (16): 129-154, 2011

13 De, P., "Technology usage and online sales : An empirical study" 56 (56): 1930-1945, 2010

14 Onuma, K., "TANGENT: A novel, ‘Surprise me’, recommendation algorithm" ACM 657-666, 2009

15 Thorat, P. B., "Survey on collaborative filtering, content-based filtering and hybrid recommendation system" 110 (110): 31-36, 2015

16 Meseguer-Martinez, A., "Satisfaction with online teaching videos : A quantitative approach" 54 (54): 62-67, 2017

17 Bobadilla, J., "Recommender systems survey" 46 : 109-132, 2013

18 Shani, G., "Recommender systems handbook" Springer 257-297, 2011

19 Lü, L., "Recommender systems" 519 (519): 1-49, 2012

20 Lu, J., "Recommender system application developments : A survey" 74 : 12-32, 2015

21 Vargas, S, "Rank and relevance in novelty and diversity metrics for recommender systems" ACM 109-116, 2011

22 Liang, T. P., "Personalized content recommendation and user satisfaction : Theoretical synthesis and empirical findings" 23 (23): 45-70, 2006

23 Adamopoulos, P., "On unexpectedness in recommender systems: Or how to better expect the unexpected" 5 (5): 54-, 2015

24 Katz, M. L., "Network externalities, competition, and compatibility" 75 (75): 424-440, 1985

25 Zeng, A., "Modeling mutual feedback between users and recommender systems" 7 : 07020-, 2015

26 Murakami, T., "Metrics for evaluating the serendipity of recommendation lists" Springer 40-46, 2007

27 Kaminskas, M, "Measuring surprise in recommender systems" 2014

28 Yao, Y. Y., "Measuring retrieval effectiveness based on user preference of documents" 46 (46): 133-145, 1995

29 Adomavicius, G, "Maximizing aggregate recommendation diversity: A graph-theoretic approach" 3-10, 2011

30 Gatian, A. W., "Is user satisfaction a valid measure of system effectiveness?" 26 (26): 119-131, 1994

31 Kotkov, D, "Investigating serendipity in recommender systems based on real user feedback" ACM 1341-1350, 2018

32 Ogara, S. O., "Investigating factors affecting social presence and user satisfaction with mobile instant messaging" 36 : 453-459, 2014

33 Sridharan, S., "Introducing serendipity in recommender systems through collaborative methods" University of Rhode Island 2014

34 Iaquinta, L, "Introducing serendipity in a content-based recommender system" IEEE 168-173, 2008

35 Lin, K. Y., "Intention to continue using Facebook fan pages from the perspective of social capital theory" 14 (14): 565-570, 2011

36 Ziegler, C. N, "Improving recommendation lists through topic diversification" ACM 22-32, 2005

37 Tang, T. Y., "I should not recommend it to you even if you will like it: The ethics of recommender systems" 22 (22): 111-138, 2016

38 Lee, J. K., "How is strategic brand alliance communicated to consumers? A content assessment of print ads" 1 (1): 5-40, 2012

39 Zhou, R., "How YouTube videos are discovered and its impact on video views" 75 (75): 6035-6058, 2016

40 Lu, C., "Herfindahl–Hirschman Index based performance analysis on the convergence development" 20 (20): 121-129, 2017

41 Hirschman, E. C., "Hedonic consumption : emerging concepts, methods and propositions" 46 (46): 92-101, 1982

42 Wang, E. S. T., "Forming relationship commitments to online communities : The role of social motivations" 28 (28): 570-575, 2012

43 Moghavvemi, S, "Facebook and YouTube addiction: The usage pattern of Malaysian students" IEEE 1-6, 2017

44 Huang, L. T., "Exploring utilitarian and hedonic antecedents for adopting information from a recommendation agent and unplanned purchase behaviour" 22 (22): 139-165, 2016

45 Zhou, T., "Examining mobile instant messaging user loyalty from the perspectives of network externalities and flow experience" 27 (27): 883-889, 2011

46 Herlocker, J. L., "Evaluating collaborative filtering recommender systems" 22 (22): 5-53, 2004

47 Jalili, M., "Evaluating collaborative filtering recommender algorithms : a survey" 6 : 74003-74024, 2018

48 Li, C., "Enhancing the efficiency of massive online learning by integrating intelligent analysis into MOOCs with an application to education of sustainability" 10 (10): 468-, 2018

49 Lin, C. P., "Elucidating individual intention to use interactive information technologies : The role of network externalities" 13 (13): 85-108, 2008

50 Li, W., "Effect of directors' social network centrality on corporate charitable donation" 47 (47): 1-9, 2019

51 Strahilevitz, M., "Donations to charity as purchase incentives : How well they work may depend on what you are trying to sell" 24 (24): 434-446, 1998

52 Qiao, Z., "Do foreign institutional investors enhance firm innovation in China?" 1-4, 2018

53 Kaminskas, M., "Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems" 7 (7): 2-, 2017

54 Kunaver, M., "Diversity in recommender systems : A survey" 123 : 154-162, 2017

55 Covington, P, "Deep neural networks for Youtube recommendations" ACM 191-198, 2016

56 Vargas, S., "Coverage, redundancy and size-awareness in genre diversity for recommender systems" ACM 209-216, 2014

57 Abbas, M, "Context-aware Youtube recommender system" IEEE 161-164, 2017

58 Kumar, A, "Comparison of various metrics used in collaborative filtering for recommendation system" IEEE 150-154, 2015

59 Nakatsuji, M, "Classical music for rock fans?: novel recommendations for expanding user interests" ACM 949-958, 2010

60 Salter, J., "CinemaScreen recommender agent : Combining collaborative and content-based filtering" 21 (21): 35-41, 2006

61 Iaquinta, L., "Can a recommender system induce serendipitous encounters?" IntechOpen 2010

62 Wallner, G., "Beyond the individual : Understanding social structures of an online player matchmaking website" 30 : 100284-, 2019

63 Ge, M, "Beyond accuracy: Evaluating recommender systems by coverage and serendipity" ACM 257-260, 2010

64 McNee, S. M, "Being accurate is not enough: how accuracy metrics have hurt recommender systems" ACM 1097-1101, 2006

65 Bremus, F., "Banking market structure and macroeconomic stability: Are low income countries special?" 20 (20): 73-100, 2015

66 Zhang, Y. C, "Auralist: Introducing serendipity into music recommendation" ACM 13-22, 2012

67 Baier, D., "Acceptance of recommendations to buy in online retailing" 17 (17): 173-180, 2010

68 Kotkov, D., "A survey of serendipity in recommender systems" 111 : 180-192, 2016

69 Qin, S, "A recommender system for youtube based on its network of reviewers" IEEE 323-328, 2010

70 Hong, S. E, "A comparative study of video recommender systems in big data era" IEEE 125-127, 2016