Innovative Strategies for Peer Review

Edward Barroga 대한의학회 2020 Journal of Korean medical science Vol.35 No.20

Peer review is a crucial part of research and publishing. However, it remains imperfect and suffers from bias, lack of transparency, and professional jealousy. It is also overburdened by an increasing quantity of complex papers against the stagnant pool of reviewers, causing delays in peer review. Additionally, many medical, nursing, and healthcare educators, peer reviewers, and authors may not be completely familiar with the current changes in peer review. Moreover, reviewer education and training have unfortunately remained lacking. This is especially crucial since current initiatives to improve the review process are now influenced by factors other than academic needs. Thus, increasing attention has recently focused on ways of streamlining the peer review process and implementing alternative peer-review methods using new technologies and open access models. This article aims to give an overview of the innovative strategies for peer review and to consider perspectives that may be helpful in introducing changes to peer review. Critical assessments of peer review innovations and incentives based on past and present experiences are indispensable. A theoretical appraisal must be balanced by a realistic appraisal of the ethical roles of all stakeholders in enhancing the peer review process. As the peer review system is far from being perfect, identifying and developing core competencies among reviewers, continuing education of researchers, reviewer education and training, and professional engagement of the scientific community in various disciplines may help bridge gaps in an imperfect but indispensable peer review system.

해부실습에서 동료평가에 대한 의과대학생들의 인식과 필요성

조정준,강성우,이승준,정수정,이재호 대한체질인류학회 2019 해부·생물인류학 (Anat Biol Anthropol) Vol.32 No.3

Peer review in dissection is a good evaluation method that allows students not only to recognize professional behavior of peers, but also to look back on themselves. The purpose of this study is to analyze the recognition and feelings of students about peer review in dissection and to use peer review more effectively. In this study, it was to examine the reflection on grades of peer review, the improvement on attitude of dissection by peer review, and the reliability of peer review, also, additionally necessary subjects, the method of peer review, the emotions of students during the peer review. As a result of the questionnaire survey, the necessity about reflection of peer review (48, 67.6%), the improvement on attitude of dissection (41, 57.7%) and the reliability of peer review (30, 44.1%) were positive. Moreover, many students asked additionally necessary subjects not to be required except for anatomy (42, 67.7%). In the questionnaire on the method of peer review, to select students who have been good and bad in the practice is best (34 students, 57.6%). In the questionnaire for emotions of students during the peer review, “bad” was the most common (25, 39.7%), and the average value was about 2.7 in the 5 scale. There is a limitation to utilize peer review as an evaluation method of anatomy education. Therefore, much more effort is needed to improve the utilization of peer review and the professor should support the emotions of students during the peer review. 해부실습에서 동료평가는 학생들이 자신과 동료들의 행동을 되돌아보게 하는 좋은 평가 방법이다. 본 연구의 목적은 해부실습에서 동료평가에 대한 학생들의 인식과 감정을 분석함으로써 향후 동료평가를 보다 효 과적으로 사용하고자 한다. 본 연구에서는 2018학년도 K대학교 의과대학에 재학 중인 본과 1학년 학생 72명 을 대상으로 동료평가의 성적 반영, 실습 태도의 개선 효과, 평가 신뢰도와 동료평가 시행이 추가적으로 필요 한 과목, 동료평가 방법, 동료평가 시 학생들의 감정에 대한 총 6개 문항을 설문하였다. 설문 조사 결과, 동료평 가의 성적 반영의 필요성 (48명, 67.6%), 실습 태도의 개선 효과 (41명, 57.7%)와 평가의 신뢰도 (30명, 44.1%) 는 긍정적인 반응을 보였다. 그리고 해부학 외 동료평가가 필요한 과목은 없다 (42명, 67.7%)는 의견이 많았다. 평가방법에 대한 설문에서는 해부실습에 성실히 참여한 학생과 성실히 참여하지 않은 학생 모두를 평가하자 는 의견이 가장 많았다 (34명, 57.6%). 그리고 평가 시 감정에 대한 설문에서는 나쁨 (25명, 39.7%)이 가장 많았 으며, 평균값은 5점 척도에서 약 2.7로써 나쁨과 보통 사이였다. 아직 해부학 교육의 평가 방법으로 동료평가를 활용하는 데에는 많은 한계가 있기 때문에 동료평가의 활용도를 높이기 위해서는 평가 시 학생들의 감정을 잘 조절할 수 있도록 많은 노력이 필요하다.

Ten Tips for Performing Your First Peer Review: The Next Step for the Aspiring Academic Plastic Surgeon

Martin Frendø,Andreas Frithioff,Steven Arild Wuyts Andersen 대한성형외과학회 2022 Archives of Plastic Surgery Vol.49 No.4

Performing the first peer review of a plastic surgical research article can be an overwhelming task. However, it is an essential scholarly skill and peer review is used in a multitude of settings: evaluation of journal articles, conference abstracts, and research proposals. Furthermore, peer reviewing provides more than just the opportunity to read and help improve other’s work: peer reviewing can improve your own scientific writing. A structured approach is possible and recommended. In these ten tips, we provide guidance on how to successfully conduct the first peer reviews. The ten tips on peer reviewing concern: 1) Appropriateness: are you qualified and prepared to perform the peer review? 2) Familiarization with the journal and its reviewing guidelines; 3) Gathering first impressions of the paper followed by specific tips for reviewing; 4) the abstract and introduction; 5) Materials, methods, and results (including statistical considerations); and 6) discussion, conclusion, and references. Tip 7 concerns writing and structuring the review; Tips 7 and 8 describe how to provide constructive criticism and understanding the limits of your expertise. Finally, Tip 10 details why—and how—you become a peer reviewer. Peer review can be done by any plastic surgeon, not just those interested in an academic career. These ten tips provide useful insights for both the aspiring and the experienced peer reviewer. In conclusion, a systematic approach to peer reviewing is possible and recommended, and can help you getting started to provide quality peer reviews that contribute to moving the field of plastic surgery forward.

Data journals: types of peer review, review criteria, and editorial committee members’ positions

서선경,김지현 한국과학학술지편집인협의회 2020 Science Editing Vol.7 No.2

Purpose: This study analyzed the peer review systems, criteria, and editorial committee structures of data journals, aiming to determine the current state of data peer review and to offer suggestions. Methods: We analyzed peer review systems and criteria for peer review in nine data journals indexed by Web of Science, as well as the positions of the editorial committee members of the journals. Each data journal’s website was initially surveyed, and the editors-in-chief were queried via email about any information not found on the websites. The peer review criteria of the journals were analyzed in terms of data quality, metadata quality, and general quality. Results: Seven of the nine data journals adopted single-blind and open review peer review methods. The remaining two implemented modified models, such as interactive and community review. In the peer review criteria, there was a shared emphasis on the appropriateness of data production methodology and detailed descriptions. The editorial committees of the journals tended to have subject editors or subject advisory boards, while a few journals included positions with the responsibility of evaluating the technical quality of data. Conclusion: Creating a community of subject experts and securing various editorial positions for peer review are necessary for data journals to achieve data quality assurance and to promote reuse. New practices will emerge in terms of data peer review models, criteria, and editorial positions, and further research needs to be conducted.

Korean College Students’ Peer Review and Revision Process and Their Attitudinal Change toward Peer Review

권은숙,김신혜 언어과학회 2018 언어과학연구 Vol.0 No.84

The current study aims to investigate the impact of Korean EFL college students’ peer review and revision on their essay writing, and to describe how students’ attitudes towards peer review change over time. Twelve Korean college students participated in this five-month peer review and revision process for two essays and their peer review comments and revisions were collected for analysis. The findings show that the participants’ peer review focused on grammar and vocabulary rather than meaning-level changes such as content and organization. Most of review comments were reflected in the revisions but some were not reflected due to the participants’ lack of writing skills and perceptions of effectiveness of peer review comments. The participants became more aware of their own writing and motivated to write as they involved in the peer review process. The results suggest that training in peer review and building trust among peers are essential to effective peer review.

The changing face of peer review

Irene Hames 한국과학학술지편집인협의회 2014 Science Editing Vol.1 No.1

It is a time of great innovation in peer review. Traditional models are being adapted and completely new ones introduced. Independent peer-review services are also starting to be offered by organizations outside the traditional journal ecosphere. In both new and established systems, the importance of increasing openness, transparency, and interaction between peer-review participants is being recognized, and these are being introduced to varying degrees. Concern with the ‘wastage’ of review effort in traditional peer review, where manuscripts often go from journal to journal, being reviewed afresh at each, before being accepted for publication, is also being addressed. Reviews are being transferred (‘cascaded’) and shared between some journals. The separation of the two basic functions of peer review—critical review and selection—as originally introduced by the journal PLOS ONE has been a major innovation, leading to the publication of sound work irrespective of its perceived novelty, interest, or importance. Post-publication review is also becoming more important and is another growth area. The concept of ‘portable’ reviews has been introduced, where authors can take reviews with them—either after they have obtained them from a peer-review provider in return for a fee or had their manuscript reviewed and declined at some journals—and include them with submissions to journals. The dynamics of publication are changing alongside, with journals able to ‘bid’ for papers that have been reviewed by independent organizations and make publishing offers to the authors. A number of innovations and ‘alternative’ peer-review models are described. They all, however, face many of the same issues as traditional peer review, and the same basic principles of good and ethical practice apply.

Korean College Students` Peer Review and Revision Process and Their Attitudinal Change toward Peer Review

Eunsook Kwon(권은숙),Shinhye Kim(김신혜) 언어과학회 2018 언어과학연구 Vol.0 No.84

The current study aims to investigate the impact of Korean EFL college students` peer review and revision on their essay writing, and to describe how students` attitudes towards peer review change over time. Twelve Korean college students participated in this five-month peer review and revision process for two essays and their peer review comments and revisions were collected for analysis. The finding show that the participants` peer review focused on grammar and vocabulary rather than meaning-level changes such as content and organization. Most of review comments were reflected in the revisions but some were not reflected due to the participants` lack of writing skills and perceptions of effectiveness of per review comments. The participants become more aware of their own writing and motivated to write as they involved in the peer review process. The results suggest that training in peer review and building trust among peers are essential to effective peer review.

국내 연구자의 국제 학술지 동료 심사 활동에 관한 연구: Publons를 중심으로

조재인 한국비블리아학회 2022 한국비블리아학회지 Vol.33 No.1

As a new academic publication model is attempted to improve the transparency, efficiency, and speed of scientific knowledge production and distribution, the open peer review platform for verification and openness of peer review history is also activated. Publons is a global platform for tracking, validating, disclosing, and recognizing the peer-reviewed histories of more than 3 million researchers worldwide. This study analyzed the review activities of 579 researchers from domestic universities who are actively reviewing international journals through Publons. As a result of the analysis, first, researchers from domestic universities who actively review international academic journals were found to be in the fields of medicine and electrical and electronics, and in most fields, assistant professors or higher with high WOS indexed research papers are participating. Second, there was a long-tail phenomenon in which a small number of reviewers with extremely high number of review papers existed in all academic fields, and there was no significant difference in the number of review papers and review report length depending on the nationality, academic status, and age of the reviewers. Lastly, although there was a weak correlation between the amount of papers reviewed by reviewers and the number of published papers, it was found that researchers with an extremely large number of reviews do not necessarily produce as many research papers. 과학지식 생산 유통의 투명성, 효율성, 신속성을 제고하기 위하여 새로운 학술 출판 모델이 시도되면서, 동료 심사 이력의 검증과 개방을 위한 피어 리뷰 플렛폼도 활성화되고 있다. Publons는 3백만명 이상의 전 세계 연구자들의 동료 심사 이력을 추적해 검증하고 공개할 뿐 아니라 업적으로 인정하기 위한 글로벌 플렛폼이다. 본 연구는 Publons를 활용해 국제 학술지 심사 위원으로 활동하는 국내 대학 소속 연구자 579명을 선별해 동료 심사 활동을 분석하였다. 분석 결과 첫 번째, 국제 학술지에서 활발한 심사 활동을 하는 국내 연구자는 공학, 의약학 분야에 많으며, 대부분의 분야에서 높은 국제 학술지 출판 실적을 보유한 조교수급 이상의 교수들인 것으로 나타났다. 두 번째, 모든 학문 분야에서 극단적으로 높은 심사 논문수를 보이는 소수의 심사자가 존재하였으며, 심사자의 신분, 나이, 국적에 따라서 심사 논문수와 심사 보고서 길이에 유의미한 차이는 존재하지 않았다. 마지막으로 심사자의 심사 논문수와 SCI 급 출판 논문수 사이에는 약한 상관성(r = 0.311)이 존재했지만, 많은 연구실적을 보유한 연구자가 그만큼 많은 국제 학술지 심사 활동을 한다고 말하기 어려운 것으로 분석되었다.

Negotiations and Criticisms in L2 Peer Review Sessions

이지혜 현대영어교육학회 2018 현대영어교육 Vol.19 No.4

The purpose of this study is to describe the types of negotiations and the use of criticizing strategies manifested during peer review interactions. Four low-level learners and four native speakers(NSs) of English participated in peer review sessions. The L2 learners, novice peer reviewers, were compared to the NSs who were familiar with peer review tasks from their English writing classes from high school through college. Data were collected through interactive peer review activities. The participants were asked to exchange the first draft of their essays and give suggestions following peer review guidelines. The peer review interactions were audiotaped and analyzed for types of negotiations and criticizing strategies. Discourse analysis revealed that whereas the NSs employed negotiation types to discuss opinions and provide constructive suggestions, the L2 learners mainly discussed the procedures of the peer review task and showed difficulties in comprehending the essay drafts. In realizing criticisms, compared to the NSs, the L2 learners used a relatively narrow range of criticism strategies and mitigating devices. The study analyzes the learner behavior and suggests pedagogical implications for enhancing L2 peer reviews.

The Effects of Peer Review Interactions on Korean College Students’ Writing

권은숙 대한영어영문학회 2019 영어영문학연구 Vol.45 No.1

The present study investigated the types of peer reviews and revisions by Korean EFL college students on their writing, and described students’ perception towards peer review. It comprised of a sample of twelve college students joined in this peer review and revision process through surveys and in-depth interviews. The results demonstrate that the participants’ peer review paid more attention to addition and positive praise, and their revisions focused on corrections, additions, and no revision. Based on the holistic and analytic scoring results, it indicates that the peer review activity did not reach the expected increase in writing quality. However, participants increased in C-test scores and vocabulary, and employed many writing strategies from their peers. After the peer review process, the participants recognized that the peer review interaction helped them improve their learning: how to share ideas, organize their writing, compare different thoughts of peers, read each other’s writings, and find out their own strengths and weaknesses. The results showed that the participants became more positive towards writing in English because they became less anxious in writing as they repeatedly engaged in the peer review process.