특수교육 연구에서 영가설 유의성 검증의 문제점과 대안 고찰 = Null Hypothesis Significance Testing in Special Education Research: Problems and Solutions

국문 초록 (Abstract)

연구목적: 본 논문에서는 특수교육에서 주된 양적 연구방법으로 사용되어 온 영가설 유의성 검증(NHST)방법의 문제점을 살펴보고 그 대안을 탐색하는 연구목적을 설정하였다. 연구방법: 본 논문에서 NHST의 개념 이해 및 관련 문제점을 설명하고 NHST의 대안이 될 수 있는 방안을 모색하기 위하여 NHST의 이슈를 다루고 있는 최신 논문과 함께 전통적으로 인용이 많이 되었던 논문을 참조하였다. 연구결과: 본 논문에서 먼저 NHST의 개념을 제공하였고, NHST의 문제점을 개념 자체의 문제점과 해석 및 활용의 문제점으로 구분하여 제공하였다. 결론: NHST의 문제점을 해결할 수 있는 방안으로 효과크기, 신뢰구간, 임상적 유의성 평가, 베이지안 통계 방법을 제안하였다. 향후 특수교육 연구자들이 본 논문에서 제안한 방법으로 NHST의 문제점을 극복하길 기대해 본다.

다국어 초록 (Multilingual Abstract)

Purpose: The main purpose of this study was to examine main problems on Null Hypothesis Significance Testing (NHST) used as main research method in special education research and to explore alternative methods on NHST. Method: Important and highly cited studies concerning misuses and limitations of NHST and alternative methods on NHST were reviewed in this study. Results: This study summarized not only some concepts of NHST, but also some limitations of NHST including misconceptions and misinterpretations. Conclusion: This study presents four alternative methods – effect size, confidence interval, metrics for assessing practical importance, and Bayesian statistics. It is recommend for researchers in special education to use these methods for overcoming these limitations of NHST.

목차 (Table of Contents)

• Ⅰ. 연구의 필요성
• Ⅱ. NHST의 기본 개념
• Ⅲ. NHST의 문제점
• Ⅳ. 특수교육 양적 연구에서 NHST의 대안 방법
• Ⅳ. 결 론
• 참고문헌
• Abstract

참고문헌 (Reference)

1 전윤희, "학습장애 또는 학습부진 학생들의 수학문장제 문제해결력 중재효과에 대한 메타분석" 한국특수교육학회 47 (47): 139-163, 2013

2 최승오, "특수체육 연구에서 통계적 유의성 검증에 대한 재검토 및 제언" 한국특수체육학회 14 (14): 35-54, 2006

3 이은옥, "통계적 유의성 검정상의 오류와 임상적 유의성의 중요성" 13 (13): 1-6, 1999

4 백진희, "유아특수교육 대상자를 위한 긍정적 행동지원 국내연구동향 분석" 단국대학교 특수교육대학원 2012

5 Donald Sharpe, "Why the resistance to statistical innovations? Bridging the communication gap." American Psychological Association (APA) 18 (18): 572-582, 2013

6 Steven N. Goodman, "Why is getting rid of p-values so hard? Musings on science and statistics" Informa UK Limited 73 (73): 26-30, 2019

7 John P. A. Ioannidis, "Why Most Published Research Findings Are False" Public Library of Science (PLoS) 2 (2): e124-, 2005

8 Denes Szucs, "When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment" Frontiers Media SA 11 : 390-, 2017

9 Doris Luft Baker, "Validity of CBM measures of oral reading fluency and reading comprehension on high-stakes reading assessments in Grades 7 and 8" Springer Science and Business Media LLC 28 (28): 57-104, 2015

10 Steven N. Goodman, "Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy" American College of Physicians 130 (130): 995-, 1999

11 Gelman, A., "The statistical crisis in science" 102 : 460-465, 2014

12 Peter R. Freeman, "The role ofP-values in analysing trial results" Wiley 12 (12): 1443-1452, 1993

13 Jacob Cohen, "The earth is round (p < .05)." American Psychological Association (APA) 49 (49): 997-1003, 1994

14 Nix, T. W., "The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing" 5 (5): 3-14, 1998

15 Richard J. Smith, "The continuing misuse of null hypothesis significance testing in biological anthropology" Wiley 166 (166): 236-245, 2018

16 Keuth, H., "The Philosophy of Karl Popper" Cambridge University Press 2005

17 E. L. Lehmann, "The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two?" Informa UK Limited 88 (88): 1242-1249, 1993

18 Megan L. Head, "The Extent and Consequences of P-Hacking in Science" Public Library of Science (PLoS) 13 (13): e1002106-, 2015

19 Andrew Gelman, "The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant" Informa UK Limited 60 (60): 328-331, 2006

20 Ronald Carver, "The Case Against Statistical Significance Testing" Harvard Education Publishing Group 48 (48): 378-399, 1978

21 Ronald L. Wasserstein, "The ASA statement on p-values: context, process, and purpose" Informa UK Limited 70 (70): 129-133, 2016

22 Sander Greenland, "Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations" Springer Science and Business Media LLC 31 (31): 337-350, 2016

23 Frank L. Schmidt, "Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers." American Psychological Association (APA) 1 (1): 115-129, 1996

24 Blakeley B. McShane, "Statistical Significance and the Dichotomization of Evidence" Informa UK Limited 112 (112): 885-895, 2017

25 Fidler, F., "Should psychology abandon p-values and teach CIs instead? Evidence-based reforms in statistics education" International Association for Statistical Education 2006

26 Valentin Amrhein, "Scientists rise up against statistical significance" Springer Science and Business Media LLC 567 (567): 305-307, 2019

27 Amrhein, V., "Remove, rather than redefine, statistical significance" 2 (2): 4-4, 2018

28 Daniel J. Benjamin, "Redefine statistical significance" Springer Science and Business Media LLC 2 (2): 6-10, 2018

29 Nicholas A. Gage, "Publication Bias in Special Education Meta-Analyses" SAGE Publications 83 (83): 428-445, 2017

30 Heene, M., "Psychological science under scrutiny: Recent challenges and proposed solutions" John Wiley 34-52, 2017

31 JEFFREY A. GLINER, "Problems With Null Hypothesis Significance Testing" Elsevier BV 40 (40): 250-252, 2001

32 Bertie Vidgen, "P-Values: Misunderstood and Misused" Frontiers Media SA 4 : 2016

33 Burnham, K. P., "P values are only an index to evidence: 20th-vs. 21st-century statistical science" 95 (95): 627-630, 2014

34 Hubbard, R., "P values are not error probabilities (Report No. 03-26)"

35 Daniel H. Robinson, "On the Past and Future of Null Hypothesis Significance Testing" JSTOR 66 (66): 263-271, 2002

36 Michael Perdices, "Null hypothesis significance testing, p-values, effects sizes and confidence intervals" Cambridge University Press (CUP) 19 (19): 70-80, 2018

37 Travers, J. C., "Null hypothesis significance testing and p values" 32 (32): 208-215, 2017

38 David R. Anderson, "Null Hypothesis Testing: Problems, Prevalence, and an Alternative" JSTOR 64 (64): 912-923, 2000

39 Paul L. Morgan, "Null Hypothesis Significance Testing: Philosophical and Practical Considerations of a Statistical Controversy" Informa UK Limited 11 (11): 209-221, 2003

40 Bryan G. Cook, "Null Effects and Publication Bias in Special Education Research" SAGE Publications 42 (42): 149-158, 2017

41 Dragicevic, P., "Modern statistical methods for HCI" Springer 291-330, 2016

42 Anderson, D. R., "Model based inference in the life sciences: A primer on evidence" Springer 2008

43 R. Rosenthal, "Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews" Annual Reviews 52 (52): 59-82, 2001

44 Long, J. D., "Longitudinal data analysis for the behavioral sciences using R" Sage 2012

45 Alexander Etz, "Introduction to Bayesian Inference for Psychology" Springer Science and Business Media LLC 25 (25): 5-34, 2018

46 Murtaugh, P. A., "In defense of P values" 95 (95): 611-617, 2014

47 Bryan G. Cook, "Group-Difference Effect Sizes: Gauging the Practical Importance of Findings from Group-Experimental Research" Wiley 33 (33): 56-63, 2018

48 Jose D. Perezgonzalez, "Fisher, Neyman-Pearson or NHST? A tutorial for teaching data testing" Frontiers Media SA 6 : 1-11, 2015

49 Joseph P. Simmons, "False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant" SAGE Publications 22 (22): 1359-1366, 2011

50 Bryan G. Cook, "Evidence‐Based Reviews: How Evidence‐Based Practices are Systematically Identified" Wiley 35 (35): 6-13, 2020

51 Kruschke, J. K., "Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan" Elsevier 2015

52 Theodore J. Christ, "Curriculum-Based Measurement of Reading: An Evaluation of Frequentist and Bayesian Methods to Model Progress Monitoring Data" SAGE Publications 36 (36): 55-73, 2018

53 Gigerenzer, G., "Cognition as intuitive statistics" Psychology Press 1987

54 Eric-Jan Wagenmakers, "Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications" Springer Science and Business Media LLC 25 (25): 35-57, 2018

55 Riko Kelter, "Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP" Springer Science and Business Media LLC 20 (20): 142-, 2020

56 Alexander Etz, "Bayesian Inference and Testing Any Hypothesis You Can Specify" SAGE Publications 1 (1): 281-295, 2018

57 Blakeley B. McShane, "Abandon Statistical Significance" Informa UK Limited 73 (73): 235-245, 2019

58 E.J. Masicampo, "A peculiar prevalence of p-values just below .05" SAGE Publications 65 (65): 2271-2279, 2012

59 Hanauer, M., "A decision-making framework to support response to intervention approaches using mixed methods Bayesian analyses" 25 (25): 68-76, 2018

60 Bryan G. Cook, "A Call for Examining Replication and Bias in Special Education Research" SAGE Publications 35 (35): 233-246, 2014