Students' interaction with AI drawing system : Focusing on students' attitude toward AI and drawing skills = 학습자와 인공지능 드로잉 시스템 간의 상호작용: 학습자의 인공지능에 대한 태도와 드로잉 능력에 따른 차이를 중심으로

다국어 초록 (Multilingual Abstract)

Recent advances and applications of AI have increased the opportunities for students to interact with AI in learning activities, including creative activities such as art. While human-AI interaction has been investigated in various fields, the underlying processes of how students actually interact with AI on a task given have been scarcely investigated. Particularly, differences in the student-AI interaction (SAI) depending on students' characteristics have not been addressed in detail yet. Furthermore, there is a lack of convincing evidence on the improvement of students' task performance through SAI. It is, thereby, missing out on the holistic understanding of the nature, effects, and areas of improvement in interaction by heterogeneous agents on learning.
To bridge these crucial gaps in our understanding, the present dissertation attempted to provide a deeper understanding of SAI on a learning task, particularly on a design task, and its empirical evidence. To be specific, the study aimed (1) to investigate differences in the SAI process amongst students with differing attitudes toward AI and drawing skills, (2) to examine students' improvement of the task performance through SAI, and (3) to explore students' diverse perceptions of SAI.
To achieve the aims of the study, the study conducted a within-subject design experiment participated by twenty Korean undergraduate students whereby an individual student was given a public advertisement design task to complete with AutoDraw, a machine learning-based drawing system. Taken both assessments of students' level of drawing skill and attitude toward AI, participants were classified into 4 groups consisting of 5 students: A group with (1) the positive attitude toward AI and the high level of drawing skills (PAHD); (2) the positive attitude toward AI but the low level of drawing skills (PALD); (3) the negative attitude toward AI but the high level of drawing skills (NAHD); (4) the negative attitude toward AI and the low level of drawing skills (NALD).
The study first investigated the group differences in the SAI process. The transcribed think-aloud obtained from a within-subject design was segmented into semantic units and analyzed using coding schemes. The study then conducted the Lag-Sequential Analysis to statistically significant linear patterns of each group and then chronologically incorporated them into the SAI duration via coded activity alignment series to distinguish the overall SAI process of each group. The study findings revealed the distinctive differences in SAI processes among groups. For instance, PAHD more effectively coordinated SAI than others whereas NAHD did not fully use AI even when SAI is helpful in the task operation. On the other hand, both PALD and NALD presented much reliance on AI regardless of their attitude toward AI while exhibiting opposite emotional responses to AutoDraw.
Following the results of the SAI process, the participants' final outcomes of public advertisement were evaluated by 8 experts. The study conducted nonparametric statistics of the Wilcoxon signed-rank test to determine if the design task performance in SAI is statistically significant compared to that of on their own (self-drawing). The Wilcoxon signed-rank test discovered that the students' improvement of the task performance through SAI were prevalent in every group of students in general. However, the scale and areas of such improvements varied; for instance, the overall task performance of NAHD drastically plummeted when the rest of the groups improved through the SAI. Concerning the evaluation criteria beholding specific categories (creativity, expressivity, and public utility), four groups differed in the task performance improvements. The improvement of the task performance on creativity was discerned in respective to students' attitudes toward AI; Both PAHD and PALD performance scores of the expressivity surpassed ones on their own. Concerning the expressivity enhancements, levels of the drawing skills determined the positive effects of the SAI; both PALD and NALD's performance were found to be improved in terms of public utility.
Last but not least, the improvement of the task performance on public utility was found in PAHD and NALD.
Along with these, this study conducted individual interviews with students to explore students' perceptions of SAI after the experiment. The study found that students considered AI as an independent learning mate, a detail-oriented and patient tutor, and an effective tool to complete the task. Students highlighted the benefits of SAI from two different sides: (1) affective domain which includes emotional comfort, positive attitudes toward the design task, joy of performing the task, and improved self-efficacy; and (2) task performance, which is driven from promoting their creativity, power of expression, task execution speed, and cost reduction. In addition, students discussed both the student-related and the AI-related obstacles to SAI.
Taking all findings together, the study presented implications for designing educational AI interacting with students for developers and instructional strategies for enhancing SAI. For instance, (1) the AI system interacting with students should better be designed with a holistic view of the SAI process that requires strong pedagogical support throughout a series of task activities; (2) the AI system should be developed to be more explainable to facilitate shared mental models between students and AI; (3) the AI's automation should better be approached with the view of providing adaptive scaffolding. Meanwhile, the study suggests teachers should (1) support students to thoroughly understand AI's strengths and weaknesses to perform a task given, situations that students consider to be challenging, and the critical importance of working with AI; (2) teachers need to support students to self-manage and self-regulate their own learning by encouraging autonomous learning as well as to co-regulate the task with AI; and (3) teachers should encourage students to talk through the data suggested by AI and create meaning behind data, as well as to active reflection-in-action during the SAI process to improve task performance in SAI.
This study contributes to the existing body of AI in education research on SAI by providing empirical evidence on the underlying process of SAI for a design task, its effects on the improvement of the task performance, and students' perception of SAI. The study would also serve as a lens of how students with differing attitudes toward AI and domain skills interact with AI and provide a foundation for a model of the SAI process on a design task, as well as a meaningful predictor of SAI performance. In addition, this dissertation proposed a coding scheme for analyzing SAI in performing a public advertisement design task, which can be an alternative tool for concentrating on how students communicate and interact with other AI systems and collaborative pedagogical agents.

국문 초록 (Abstract)

최근 교육현장에서는 미술 등 창의적 활동을 포함한 다양한 학습활동에서 학생과 AI 간의 상호작용의 기회가 증가하고 있다. 인간과 인공지능의 공생, 상호작용 시대를 예측하며 대비하는 시대적 변화 흐름과 함께 다양한 분야에서 인간과 AI 간의 상호작용이 활발히 논의되고 있는 것에 반해, 학습맥락에서 학생-AI 간 상호작용 (Student-AI Interaction, SAI)에 대한 연구는 상대적으로 부족하다. 특히 학습자 특성에 따른 SAI 과정과 학습과제 수행에 미치는 영향의 차이를 탐색∙규명하는 연구는 거의 전무하다. 결과적으로 학습맥락에서의 이종 에이전트 (heterogeneous agents)가 어떻게 상호작용을 하고, 무엇이 상호작용에 영향을 미치며, 상호작용의 실제적 효과성은 어떠한지 등에 대한 다층적이고 통합적인 이해가 부족하다. 이에 이 연구는 학습자의 인공지능에 대한 태도와 학습과제 수행 능력 수준 차이에 따라 (1) SAI 과정, (2) SAI로 인한 학습자의 과제수행능력 향상도, (3) SAI에 대한 학습자의 인식을 탐구하여 학습자의 학습을 지원하는 교육용 AI 설계 및 AI활용수업에 대한 시사점을 제시하고자 한다.
이를 위해 이 연구는 20명의 한국 대학생을 대상으로 머신러닝 기반 드로잉 시스템인 오토드로우 (AutoDraw)와 디자인과제(공익광고 그리기)를 수행하는 피험자 내 설계실험(a within-subject design experiment)을 실시하였다. 연구참여자는 AI에 대한 태도와 드로잉 실력에 따라 총 4 그룹: (1) AI에 대한 긍정적 태도와 드로잉 실력이 높은 학생 그룹 (Students with positive attitude toward AI and the high level of drawing skills, PAHD), (2) AI에 대한 긍정적 태도를 가졌으나 드로잉 실력이 낮은 학생 그룹 (students with the positive attitude toward AI but the low level of drawing skills, PAND), (3) AI에 대한 부정적 태도를 가졌으나 드로잉 실력이 높은 학생 그룹 (students with the negative attitude toward AI but the high level of drawing skills, NAHD), (3) AI에 대한 부정적 태도와 드로잉 실력이 낮은 학생 그룹 (students with the negative attitude toward AI and the low level of drawing skills, NALD)으로 분류하였다.
먼저 이 연구는 SAI 과정에서의 그룹 간 차이를 조사했다. 이를 위해 연구참여자가 드로잉 활동 과정 동안 머릿속에 나타나는 모든 인지적 과정을 구두로 설명한 사고구술(think-aloud) 프로토콜을 연구자가 개발한 코딩스킴을 사용하여 분석하였다. 이후, 순차분석(Lag-Sequential Analysis)을 통해 각 그룹의 통계적으로 유의한 선형 패턴을 파악하고 SAI 활동 시리즈의 시간적 정보를 통합하여 각 그룹의 전체 SAI 과정을 파악하였다. 이를 통해 이 연구는 SAI 과정에서 각 그룹 간 뚜렷한 차이를 확인할 수 있었다. 예를 들어, PALD와 NALD 그룹은 SAI 과정에서 AI에 대한 상반된 감정을 보였으나 전반적으로 AI에 대한 의존도가 높았다. NAHD는 SAI 활동에서 AI와의 소극적이고 제한된 상호작용을 하였다. 반면, PAHD는 SAI 과정에서 학습자-AI간의 적극적인 조정활동을 수행하였다.
두 번째로, 이 연구는 SAI로 인한 학습자의 과제수행능력 향상도가 각 그룹별로 어떠한 차이를 보이는지 파악하기 위해 연구참여자들의 최종 드로잉 결과물에 대한 전문가 평가를 실시하였다. 총 8인의 전문가 평가 결과는 윌콕슨 부호순위 검정 (Wilcoxon Signed Rank Test)의 비모수 통계를 수행하여 분석하였으며, 그 결과 전반적으로 모든 그룹에서 SAI로 인한 과제수행능력 향상도가 확인되었다. 그러나 향상된 평가영역(내용의 창의성, 표현력의 표현성, 효과의 공익성)과 규모는 그룹별로 차이가 있었다. 예를 들어, 모든 그룹이 전반적인 평가영역에서 SAI를 통해 과제 수행력이 향상된 것에 반해, NAHD 그룹은 모든 영역에서 과제 수행력이 하락하였다. 또한, 세부 평가 영역에서도 그룹 간 차이가 발견되었다. 창의성 영역에서는 AI에 대한 태도가 긍정적인 PAHD와 PALD그룹이 점수 향상을 보였고, 표현력에서는 드로잉 실력이 낮은 PALD와 NALD의 점수가 향상되었다. 반면, 공익성 영역에서는 PAHD와 NALD의 점수가 향상되었다.
마지막으로 이 연구는 SAI에 대한 학습자들의 인식을 조사하기 위해 실험 후 연구참여자들과 개별 인터뷰를 실시하였다. 이를 통해 SAI 상황에서 학습자가 기대하는 AI의 역할은 학습동료, 세심하고 인내심 있는 튜터, 그리고 학습과제 수행을 지원하는 효과적인 학습도구인 것으로 파악되었다. 또한, 학생들은 학습과제 수행에 있어 SAI의 장점을 정의적 영역 지원과 학습과제 수행력 지원 측면에서 논의하였으며, SAI의 제한점은 AI와 관련된 제한점과 학습자와 관련된 제한점으로 나누어 논의하였다.
연구 결과를 바탕으로, 이 연구는 교육용 AI 개발과 AI수업에 대한 시사점을 제시하였다. 예를 들어, 교육용 AI는 SAI의 과정이 특정 과제 하나를 수행하는 행위가 아닌, 일련의 다양한 학습활동과 그 과정 속에서 유의미한 교육적 지원이 필요함을 고려해야 한다. 또한, 학습자와 AI 간의 공유된 멘탈모델을 지원하기 위해 AI는 보다 설명가능 해야 한다. 아울러, 학습맥락에서 AI의 자동화를 적응적 스캡폴딩의 관점에서 접근해야 한다. 반면, 교육적으로 유의미한 SAI를 지원하기 위해 교사는 SAI를 촉진하는 학습과제 및 활동 설계를 하고, 학습자가 AI의 강점 및 약점, AI와의 상호작용에서 고려해야 하는 점 등을 명확히 이해하도록 한다. 또한, 교사는 학습자가 SAI 과정에서 행동 중 성찰 (reflection-in-action) 및 데이터 스토리텔링 (data-storytelling)을 하도록 지원해야 한다. 나아가 교사는 학습자의 자기주도학습 및 학습자-AI간의 협력적 자기조절을 지원해야 한다.
이상과 같은 연구 결과는 교육분야의 학생-AI 상호작용에 대한 연구에 기여한다. 특히, AI에 대한 태도와 과제수행 능력에 따른 SAI 과정 모델을 제시하였으며, SAI로 인한 학습자의 과제수행능력 향상도에 대한 실제적∙경험적 증거를 제공하였다. 또한, 이 연구에서 개발 및 활용 한 SAI 분석 코딩스킴과 다양한 연구방법은 학생-AI 간의 상호작용에 대한 추후 연구에 참고 자료가 될 것이다.

목차 (Table of Contents)

• I. Introduction 1
• 1. Statement of the problem 1
• 2. Research aim and questions 7
• 3. Research significance 8
• 4. Conceptual Definitions 10
• 4.1. Artificial Intelligence (AI) drawing systems 10
• 4.2. Student-AI drawing system Interaction 10
• 4.3. Attitude toward AI 11
• 4.4. Domain-specific skills 11
• II. Literature Review 12
• 1. AI and education 12
• 1.1. Applications of AI in education 12
• 1.2. AI in art education 15
• 2. Understanding Student-AI Interaction 18
• 2.1. Theoretical perspectives for understanding student-AI interaction 18
• 2.1.1. Actor Network Theory (ANT) 18
• 2.1.2. Distributed cognition (DC) 22
• 2.1.3. Computers are Social Actors (CASA) 23
• 2.2. Student-AI interaction in learning 24
• 2.3. Drawing with AI 30
• 3. Characteristics of students that influence effective Student-AI interaction 33
• 3.1. Attitude toward AI 33
• 3.2. Level of domain-specific skills 36
• III. Research methodology 39
• 1. Research Design 39
• 2. Research tool 41
• 2.1. Design task 41
• 2.2. AI drawing system 42
• 3. Participants 43
• 4. Research procedure 46
• 5. Data Collection and Analysis 52
• 5.1. Students Think-aloud protocols (TAPs) 52
• 5.2. Expert evaluation on students' task outcomes 56
• 5.3. Interviews 59
• 6. Ethical consideration 60
• IV. Research findings 61
• 1. The SAI process 61
• 1.1. The differences in the SAI activity participation among groups 61
• 1.2. The differences in the SAI process among groups 67
• 1.2.1 PAHD 70
• 1.2.2 PALD 77
• 1.2.3 NAHD 83
• 1.2.4 NALD 90
• 2. The improvement of task performance through SAI 99
• 2.1. The differences in the improvement of students' overall task performance through SAI among groups 99
• 2.2. The differences in the improvement of the task performance across creativity, expressivity, and public utility through SAI among groups 100
• 3. Students' perception of SAI 104
• 3.1. The differences in the perception of the expected roles of AI 104
• 3.2. The differences in the perception of advantages of SAI 107
• 3.2.1 Enhancing the affective domain 107
• 3.2.2 Enhancing the task performance 110
• 3.3. The differences in the perception of barriers to SAI 113
• 3.3.1 AI-related barriers 113
• 3.3.2 Student-related barriers 120
• V. Discussion 124
• 1. The SAI process 124
• 2. The improvement of task performance through SAI 131
• 3. Students' perception of SAI 134
• VI. Conclusions 137
• 1. Study implications 137
• 2. Conclusions 141
• 3. Limitations of the study and recommendations for future research 143
• Reference 144
• Appendices 179

참고문헌 (Reference)

1. Human-computer integration, Farooq , U. , & Grudin , J, 23 ( 6 ) , 26-32, , 2016

2. The new spirit of capitalism ., Boltanski , L. , & Chiapello , E., 18 ( 3 ) , 161-188 ., , 2005

3. Transactive memory in the classroom, Jackson , M. , & Moreland , R. L., 40 ( 5 ) , 508-534 ., , 2009

4. CASA , WASA , and the dimensions of us, Karr-Wisniewski , P. , & Prietula , M., 26 ( 6 ) , 1761-1771 ., , 2010

5. Social robots for education : A review, Belpaeme , T. , Kennedy , J. , Ramachandran , A. , Scassellati , B. , & Tanaka , F., 3 ( 21 ) ., , 2018

6. The reflective practice of design teams, Valkenburg , R. , & Dorst , K., 19 ( 3 ) , 249-271 ., , 1998

7. Using thematic analysis in psychology ., Braun , V. , & Clarke , V., 3 ( 2 ) , 77-101 ., , 2006

8. Intelligent agents : Theory and practice, Wooldridge , M. , & Jennings , N. R., 10 ( 2 ) , 115-152 ., , 1995

9. Validity in quantitative content analysis, Rourke , L. , & Anderson , T., 52 ( 1 ) , 5-18, , 2004

10. Social interactions in HRI : the robot view, Breazeal , C., 34 ( 2 ) , 181-186 ., , 2004

11. An integrative model of organizational trust, Mayer , R.C. , Davis , J.H. , & Schoorman , F.D, 20 ( 3 ) , 709-734 ., , 1995

12. Stupid tutoring systems , intelligent humans, Baker , R. S., 26 ( 2 ) , 600-614 ., , 2016

13. Collective intelligence and group performance, Woolley , A. W. , Aggarwal , I. , & Malone , T. W., 24 ( 6 ) , 420-424 ., , 2015

14. The competence of learning companion agents ., Hietala , P. , & Niemirepo , T., 9 ( 3-4 ) , 178-192 ., , 1998

15. The evolution of research on digital education, Dillenbourg , P., 26 ( 2 ) , 544-560 ., , 2016

16. Why digital medicine depends on interoperability, Lehne , M. , Sass , J. , Essenwanger , A. , Schepers , J. , & Thun , S., 2 ( 1 ) , 1-5 ., , 2019

17. Recognizing , defining , and representing problems, Pretz , J. E. , Naples , A. J. , & Sternberg , R. J ., 30 ( 3 ), , 2003

18. The conceptual bases of study strategy inventories, Entwistle , N. , & McCune , V., 16 ( 4 ) , 325-345 ., , 2004

19. The mangle of practice : Time , agency and science, Pickering , A. , & Papineau , D., 377 ( 6549 ) , 491-491, , 1995

20. A survey on image data augmentation for deep learning ., Shorten , C. , & Khoshgoftaar , T. M., 6 ( 1 ) , 1-48 ., , 2019

21. Agent theories , architectures , and languages : a survey, Wooldridge , M. , & Jennings , N. R., pp . 1-39, , 1994

22. Machines and mindlessness : Social responses to computers, Nass , C. , & Moon , Y ., 56 , 81-103, , 2000

23. ICT and organisation change : Introduction to special issue, Barrett , M. , Grant , D. & Wailes , N., 42 ( 1 ) , 6-22 ., , 2006

24. Application of Artificial Intelligence in Modern Art Teaching, Kong , F., 15 ( 13 ) , 238-251 ., , 2020

25. Unlocking the potential of advanced manufacturing technologies, Boyer , K. K. , Leong , G. K. , Ward , P. T. , & Krajewski , L. J ., 15 ( 4 ) , 331-347 ., , 1997

26. User Acceptance of Information Technology: Towarda Unified View, Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D, 425-478, , 2003

27. Adaptive intelligent support to improve peer tutoring in algebra, Walker , E. , Rummel , N. , & Koedinger , K. R., 24 ( 1 ) , 33-61, , 2014

28. Themovement of mixed methods research and the role of educators ., Creswell , J. W. , & Garrett , A. L., 28 ( 3 ) , 321-333, , 2008

29. Design Ability . Teaching Design and Technology in Secondary Schools, Cross , N., 11 , 124 ., , 2002

30. Establishing and maintaining long-term human-computer relationships ., Bickmore , T. W. , & Picard , R. W., 12 ( 2 ) , 293-327 ., , 2005

31. Positive emotions trigger upward spirals to ward emotional well-being, Fredrickson , B. L. , & Joiner , T., 13 ( 2 ) , 172-175 ., , 2002

32. Scaffolding of small groups ' metacognitive activities with an avatar, Molenaar , I. , Chiu , M. M. , Sleegers , P. , & Van Boxtel , C., 6 ( 4 ) , 601-624 ., , 2011

33. Beyond Web 2.0 : Mapping the technology landscapes of young learners ., Clark , W. , Logan , K. , Luckin , R. , Mee , A. , & Oliver , M., 25 ( 1 ) , 56-69 ., , 2009

34. Modeling key drivers of e-learning satisfaction among student teachers, Teo , T. , & Wong , S. L., 48 ( 1 ) , 71-95, , 2013

35. Teaming with a synthetic teammate : Insights into human-autonomy teaming, McNeese , N. J. , Demir , M. , Cooke , N. J. , & Myers , C., 60 ( 2 ) , 262-273 ., , 2018

36. Attribution theory , achievement motivation , and the educational process, Weiner , B, 42 ( 2 ) , 203-215, , 1972

37. Domain-specific knowledge and why teaching generic skills does not work ., Tricot , A. , & Sweller , J ., 26 ( 2 ) , 265-283 ., , 2014

38. On teams , teamwork , and team performance : Discoveries and developments, Salas , E. , Cooke , N. J. , & Rosen , M . A ., 50 ( 3 ) , 540-547 ., , 2008

39. Big data for education : Data mining , data analytics , and web dashboards, West , D. M., 4 ( 1 ), , 2012

40. Example-tracing tutors : Intelligent tutor development for non-programmers, Aleven , V. , McLaren , B. M. , Sewall , J. , Van Velsen , M. , Popescu , O. , Demi , S. , Ringenberg , M. , & Koedinger , K. R., 26 ( 1 ) , 224-269 ., , 2016

41. Understanding collaborative learning processes in new learning environments, Hmelo-Silver , C. E. , Chernobilsky , E. , & Jordan , R., 36 ( 5-6 ) , 409- 430 ., , 2008

42. Autonomy , authenticity , authorship and intention in computer generated art, McCormack , J. , Gifford , T. , & Hutchings , P., pp . 35-50, , 2019

43. Effects of anticipatory perceptual simulation on practiced human-robot tasks, Hoffman , G. , & Breazeal , C., 28 ( 4 ) , 403-423 ., , 2010

44. Knowledge in the head and on the web : Using topic expertise to aid search ., Duggan , G. B. , & Payne , S. J ., pp . 39-48 ), , 2008

45. Reflections on evidence-based practice and rational clinical decision making, Ylvisaker , M. , Coelho , C. , Kennedy , M. , Sohlberg , M. M. , Turkstra , L. , Avery , J. , & Yorkston , K., 10 ( 3 ) , 25-33 ., , 2002

46. Toward a framework for levels of robot autonomy in human-robot interaction ., Beer , J. M. , Fisk , A. D. , & Rogers , W. A ., 3 ( 2 ) , 74 ., , 2014

47. Almost human : Anthropomorphism increases trust resilience in cognitive agents, De Visser , E. J. , Monfort , S. S. , McKendrick , R. , Smith , M. A. , McKnight , P. E. , Krueger , F. , & Parasuraman , R., 22 ( 3 ) , 331 ., , 2016

48. Autonomous tools and design : a triple-loop approach to human-machine learning, Seidel , S. , Berente , N. , Lindberg , A. , Lyytinen , K. , & Nickerson , J. V., 62 ( 1 ) , 50-57 ., , 2018

49. Event-based categorical sequential analyses of the medical interview : a review, Mazzi , M. A. , Del Piccolo , L. , & Zimmermann , C., 12 ( 2 ) , 81-85 ., , 2003

50. Technological Pedagogical Content Knowledge : A Framework for Teacher Knowledge, Mishra , P. , & Koehler , M. J, 108 ( 6 ) , 1017-1054 ., , 2006

51. User Acceptance of Computer Technology : A Comparison of Two Theoretical Models, Davis F. D. Bagozzi R. P. & Warshaw P. R., 35 ( 8 ) , 982-1003 ., , 1989

52. A study on the digital textbook development strategies and development direction, Jeong , E. S. , Song , Y. H. , & Chae , J . B ., pp . 230-235 ), , 2008

53. Emotional responses to computers : Experiences in unfairness , anger , and spite, Ferdig , R. E. , & Mishra , P., 13 ( 2 ) , 143-161 ., , 2004

54. Why not robot teachers : artificial intelligence for addressing teacher shortage, Edwards , B. I. , & Cheok , A. D., 32 ( 4 ) , 345-360 ., , 2018

55. Activity theory as a framework for designing constructivist learning environments, Jonassen , D. H. , & Rohrer-Murphy , L., 47 ( 1 ) , 61-79 ., , 1999

56. Actors , observers , and the attribution process : Toward a reconceptualization ., Monson , T. C. , & Snyder , M., 13 ( 1 ) , 89-111 ., , 1977

57. Initial validation of the general attitudes towards Artificial Intelligence Scale, Schepman , A. , & Rodway , P., 1 , 100014, , 2020

58. Evaluation in the wild : A distributed cognition perspective on teacher assessment, Halverson , R. R. , & Clifford , M. A ., 42 ( 4 ) , 578-619 ., , 2006

59. Orchestration tools to support the teacher during student collaboration : a review, van Leeuwen , A. , & Rummel , N., 47 ( 2 ) , 143-158, , 2019

60. The machines are coming : Future directions in instructional communication research, Edwards , A. , & Edwards , C., 66 ( 4 ) , 487-488, , 2017

61. The mind in the machine : Anthropomorphism increases trust in an autonomous vehicle, Waytz , A. , Heafner , J. , & Epley , N., 52 , 113-117 ., , 2014

62. University students ’ approaches to learning : rethinking the place of technology, Goodyear , P. , & Ellis , R. A, 29 ( 2 ) , 141-152 ., , 2008

63. Welcoming our robot overlords : Initial expectations about interaction with a robot, Spence , P. R. , Westerman , D. , Edwards , C. , & Edwards , A ., 31 , 272-280 ., , 2014

64. Scaffolding self-directed learning with personalized learning goal recommendations ., Ley , T. , Kump , B. , & Gerdenitsch , C., pp . 75-86 ), , 2010

65. SmartPaint : a co-creative drawing system based on generative adversarial networks ., Sun , L. , Chen , P. , Xiang , W. , Chen , P. , Gao , W. Y. , & Zhang , K. J ., 20 ( 12 ) , 1644-1656 ., , 2019

66. Measuring the effects and effectiveness of interactive advertising : A research agenda, Pavlou , P. A. , & Stewart , D. W., 1 ( 1 ) , 61-77 ., , 2000

67. Understanding the role of artificial intelligence in personalized engagement marketing, Kumar , V. , Rajan , B. , Venkatesan , R. , & Lecinski , J ., 61 ( 4 ) , 135-155 ., , 2019

68. Design and evaluation of teacher assistance tools for exploratory learning environments, Mavrikis , M. , Gutierrez-Santos , S. , & Poulovassilis , A ., pp . 168-172 ), , 2016

69. Social supports from teachers and peers as predictors of academic and social motivation, Wentzel , K. R. , Battle , A. , Russell , S. L. , & Looney , L. B, 35 ( 3 ) , 193-202 ., , 2010

70. Development of Deep Learning-based Art Learning Support Tool : Using Generative Modeling, Lee , U. G. , Kang , S. H. , Lee , J. C. , Choi , S. Y. , Coir , U , & Lim , C. I ., 26 ( 1 ) , 207-236 ., , 2020

71. Student Perception of Adaptive Collaborative Learning Support through Learning Analytics, Cho , Y. H. , Kim , K. H. , & Han , J. Y ., 25 ( 1 ) , 25-57 ., , 2019

72. Distributed cognition : toward a new foundation for human-computer interaction research ., Hollan , J. , Hutchins , E. , & Kirsh , D., 7 ( 2 ) , 174-196 ., , 2000

73. Factors influencing teachers ’ intention to use technology : Model development and test, Teo , T., 57 ( 4 ) , 2432-2440, , 2011

74. A coding scheme for analysing problem-solving processes of first-year engineering students, Grigg , S. J. , & Benson , L. C., 39 ( 6 ) , 617-635 ., , 2014

75. The nature of language learning experiences beyond the classroom and its learning outcomes, Inozu , J. , Sahinkarakas , S. , & Yumru , H., 8 ( 1 ) , 14-21 ., , 2010

76. The role of beliefs and attitudes in learning statistics : Towards an assessment framework, Gal , I. , & Ginsburg , L., 2 ( 2 ), , 1994

77. Human issues influencing the successful implementation of advanced manufacturing technology, Chung , C. A, 13 ( 3-4 ) , 283-299, , 1996

78. Measuring trust in human robot interactions : Development of the trust perception scale-HRI, Schaefer , K. E., pp . 191-218, , 2016

79. The use of single-subject research to identify evidence-based practice in special education, Horner , R. D. , Carr , E. G. , Halle , J. , McGee , G. , Odom , S. , & Wolery , M., 71 , 165-80 ., , 2005

80. A Theoretical Extension of the Technology Acceptance Model : Four Longitudinal Field Studies, Venkatesh V. & Davis F. D., 46 ( 2 ) , 186-204, , 2000

81. Initial interaction expectations with robots : Testing the human-to-human interaction script, Edwards , C. , Edwards , A. , Spence , P. R. , & Westerman , D., 67 ( 2 ) , 227-238 ., , 2016

82. Teaching the teacher : tutoring SimStudent leads to more effective cognitive tutor authoring, Matsuda , N. , Cohen , W. W. , & Koedinger , K. R., 25 ( 1 ) , 1-34 ., , 2015

83. Team-based cellular manufacturing : A review and survey to identify important social factors, Fraser , K. , Harris , H. & Luong , L., 18 ( 6 ) , 714-730 ., , 2007

84. Communication and Artificial Intelligence : Opportunities and Challenges for the 21st Century, Gunkel , D. J, 1 , 1 ( 1 ) , 1-25 ., , 2012

85. ECHOES : An intelligent serious game for fostering social communication in children with autism, Bernardini , S. , Porayska-Pomsta , K. , & Smith , T. J, 264 , 41-60, , 2014

86. Socially shared regulation : Exploring perspectives of social in self-regulated learning theory, Hadwin , A. , & Oshige , M., 113 ( 2 ) , 240-264 ., , 2011

87. An educational psychology success story : Social interdependence theory and cooperative learning, Johnson , D. W. , & Johnson , R. T., 38 ( 5 ) , 365-379 ., , 2009

88. Developing and applying TMS-based collaborative learning model for facilitating learning transfer, Lee , J, 37 ( 6 ) , 993-1003, , 2017

89. Understanding the applicability of sequential data analysis techniques for analysing usability data, Cuomo , D. L., 13 ( 1-2 ) , 171-182, , 1994

90. Quantitative description of early mother-infant interaction using information theoretical statistics, Palthe , T. , Hopkins , B. , & Vos , J. E., 112 ( 1-2 ) , 117-148, , 1990

91. I , teacher : using artificial intelligence ( AI ) and social robots in communication and instruction, Edwards , C. , Edwards , A. , Spence , P. R. , & Lin , X, 67 ( 4 ) , 473-480, , 2018

92. A study of users ’ reactions to a mixed online discussion model : A lag sequential analysis approach, Wu , S. Y. , Chen , S. Y. , & Hou , H. T., 31 ( 3 ) , 180-192 ., , 2015

93. Fostering social agency in multimedia learning : Examining the impact of an animated agent ’ s voice, Atkinson , R. K. , Mayer , R. E. , & Merrill , M. M., 30 ( 1 ) , 117-139 ., , 2005

94. MTFeedback : providing notifications to enhance teacher awareness of small group work in the classroom, Martinez-Maldonado , R. , Clayphan , A. , Yacef , K. , & Kay , J ., 8 ( 2 ) , 187-200 ., , 2014

95. Effects of users ' domain knowledge on user experience- focused on the UI design of mobile applications, Choi , J. M., 49 , 253-262, , 2015

96. Not all trust is created equal : Dispositional and history-based trust in human-automation interactions, Merritt , S. M. , & Ilgen , D. R., 50 ( 2 ) , 194-210 ., , 2008

97. Understanding individual problem-solving style : A key to learning and applying creative problem solving, Treffinger , D. J. , Selby , E. C. , & Isaksen , S. G., 18 ( 4 ) , 390-401 ., , 2008

98. Exploring the outlook of art education using artificial intelligence in university liberal arts education, Ahn , J. Y. , Park , T. J. , & Hong , S. J ., 34 ( 3 ) , 107-132 ., , 2020

99. Personalized adaptive learning : an emerging pedagogical approach enabled by a smart learning environment, Peng , H. , Ma , S. , & Spector , J. M., 6 ( 1 ) , 1-14 ., , 2019

100. An exploratory study on student-intelligent robot teacher relationship recognized by middle school students, Lee , S. S. , & Kim , J ., 18 ( 4 ) , 37-44 ., , 2020

101. A tool for evaluating advertising concepts : Desirable characteristics as viewed by creative practitioners ., Stuhlfaut , M. W. , & Yoo , C. Y ., 19 ( 2 ) , 81-97 ., , 2013

102. Distinguishing automatic and controlled components of attitudes from direct and indirect measurement methods, Ranganath , K. A. , Smith , C. T. , & Nosek , B . A ., 44 ( 2 ) , 386-396 ., , 2008

103. Information seeking in full-text end-user-oriented search systems : The roles of domain and search expertise, Marchionini , G., 15 ( 1 ) , 35-69 ., , 1993

104. The contribution of perceived classroom learning environment and motivation to student engagement in science, Tas , Y, 31 ( 4 ) , 557-577, , 2016

105. All users of information retrieval systems are not created equal : An exploration into individual differences, Borgman , C. L., 25 ( 3 ) , 237-251 ., , 1989

106. Collaborative learning practices : teacher and student perceived obstacles to effective student collaboration, Le , H. , Janssen , J. , & Wubbels , T., 48 ( 1 ) , 103-122, , 2018

107. Impact of using an educational robot-based learning system on students ’ motivation in elementary education, Chin , K. Y. , Hong , Z. W. , & Chen , Y. L., 7 ( 4 ) , 333-345 ., , 2014

108. Hello AI : Uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making, Cai , C. J. , Winter , S. , Steiner , D. , Wilcox , L. , & Terry , M., Vol . 3 , No . CSCW , pp . 1-24, , 2019

109. Exploring self-regulation and group-regulation in collaborative problem solving with cloud computing technology, Cho , Y. H. , Seol , B. , Lee , H , Kang , D. , & Cho , A. R., 23 ( 3 ) , 345-371 ., , 2017

110. Teachers , computer tutors , and teaching : The artificially intelligent tutor as an agent for classroom change, Schofield , J. W. , Eurich-Fulcer , R. , & Britt , C. L., 31 ( 3 ) , 579-607 ., , 1994

111. Case Analysis and Characteristics of the Convergence between Artificial Intelligence and Art Creation Activities, Choi , H. S. & Shon , Y. M., 28 ( 3 ) , 289-299 ., , 2017

112. Corpus-assisted creative writing : Introducing intermediate Italian learners to a corpus as a reference resource, Kennedy , C. , & Miceli , T., 14 ( 1 ) , 28-44 ., , 2010

113. Out-of-class communication and student perceptions of instructor humor orientation and socio-communicative style, Aylor , B. , & Oppliger , P., 52 ( 2 ) , 122-134 ., , 2003

114. Beyond AI : Multi-Intelligence ( MI ) Combining natural and artificial intelligences in hybrid beings and systems, Fox , S., 5 ( 3 ) , 38 ., , 2017

115. Computers that care : Investigating the effects of orientation of emotion exhibited by an embodied computer agent, Brave , S. , Nass , C. , & Hutchinson , K., 62 , 161-178 ., , 2005

116. Investigating American and Chinese Subjects ’ explicit and implicit perceptions of AI-Generated artistic work ., Wu , Y. , Mou , Y. , Li , Z. , & Xu , K., 104 , 106186 ., , 2020

117. SMILI : A framework for interfaces to learning data in open learner models , learning analytics and related fields, Bull , S. , & Kay , J, 26 ( 1 ) , 293-331 ., , 2016

118. A comparative study on the expected roles and appearance of social robots according to students ' personality traits, Kim , J. , & Lee , S. S., 26 ( 3 ) , 71-91 ., , 2020

119. Emotional attachment , performance , and viability in teams collaborating with embodied physical action ( EPA ) robots, You , S. , & Robert , L.P., 19 ( 5 ) , 377-407, , 2017

120. Domain Knowledge , Search Behaviour , and Search Effectiveness of Engineering and Science Students : An Exploratory Study, Zhang , X. , Anghelescu , H. G. , & Yuan , X ., 10 ( 2 ) , 217, , 2005

121. Exploratory versus explanatory visual learning analytics : Driving teachers ’ attention through educational data storytelling, Echeverria , V. , Martinez-Maldonado , R. , Shum , S. B. , Chiluiza , K. , Granda , R. , & Conati , C., 5 ( 3 ) , 72-97 ., , 2018

122. Design of experiments and response surface methodology to tune machine learning hyperparameters , with a random forest case-study, Lujan-Moreno , G. A. , Howard , P. R. , Rojas , O. G. , & Montgomery , D. C., 109 , 195-205 ., , 2018

123. From Alexa to Siri and the GDPR : the gendering of virtual personal assistants and the role of data protection impact assessments, Loideain , N. N. , & Adams , R., 36 , 105366, , 2020

124. Personalised and self regulated learning in the Web 2.0 era : International exemplars of innovative pedagogy using social software, McLoughlin , C. , & Lee , M. J ., 26 ( 1 ), , 2010

125. Robotic assisted design workflows : a study of key human factors influencing team fluency in human-robot collaborative design processes, Nahmad Vazquez , A. , & Jabi , W., 62 ( 5 ) , 409-423 ., , 2019

126. The longitudinal effect of intergenerational gap in acculturation on conflict and mental health in Southeast Asian American adolescents, Ying , Y. W. , & Han , M., 77 ( 1 ) , 61-66, , 2007

127. The effect of functional roles on perceived group efficiency during computer-supported collaborative learning : a matter of triangulation, Strijbos , J. W. , Martens , R. L. , Jochems , W. M. , & Broers , N. J ., 23 ( 1 ) , 353-380 ., , 2007

128. Applying multilevel modelling to content analysis data : Methodological issues in the study of role assignment in asynchronous discussion groups, De Wever , B. , Van Keer , H. , Schellens , T. , & Valcke , M., 17 ( 4 ) , 436-447 ., , 2007

129. Learning performance and behavioral patterns of online collaborative learning : Impact of cognitive load and affordances of different multimedia, Wang , C. , Fang , T. , & Gu , Y, 143 , 103683, , 2020

130. A predictive model for the successful integration of concurrent engineering with people and organizational factors : based on data of 25 companies, Duffy , V. , Danek , A. , & Salvendy , G., 5 ( 4 ) , 429-445 ., , 1995

131. Sharing Spaces with Robots in a Home Scenario-Anthropomorphic Attributions and their Effect on Proxemic Expectations and Evaluations in a Live HRI Trial, Syrdal , D. S. , Dautenhahn , K. , Walters , M. L. , & Koay , K. L., pp . 116-123 ), , 2008

132. Evaluating the impact of instructional support using data mining and process mining : A micro-level analysis of the effectiveness of metacognitive prompts, Sonnenberg , C. , & Bannert , M., 8 ( 2 ) , 51-83, , 2016

133. The ASSISTments ecosystem : Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching, Heffernan , N. T. , & Heffernan , C. L., 24 ( 4 ) , 470-497, , 2014

134. A design of artificial intelligence engine to understand abtract art based on expert system : an artificial intelligence engine to understand Mondrian 's abstract art, Lee , E. & Lee , K. H., 13 ( 2 ) , 103-118, , 2014