Continuous Improvement System on Outcome Planning and Assessment: Case Study of UFE

Munkhzaya Batbaatar(Munkhzaya Batbaatar ),Saranchimeg Nasanjargal(Saranchimeg Nasanjargal ) 적정기술학회 2023 적정기술학회지(Journal of Appropriate Technology) Vol.9 No.3

Purpose of Research: The move to outcome-based education necessitates a process of continuous improvement for academic programs including outcome-based planning, implementation, and assessment. UFE has been demonstrating a commitment to academic excellence through emphasizing outcome-based education since 2009. To do so, the university initiated and developed undergraduate program policy and regulations that indicate learning outcomes at each level including the course and programs. In other words, it shows how course learning outcomes support the programs and we constantly try to measure and evaluate them accordingly. Our learning outcomes model is based on a three-level hierarchically structured definition of learning outcomes that consistently apply to both the entire undergraduate program as well as to each individual course and faculty use this model to design, monitor, and revise both the entire curriculum as well as each individual course on an ongoing basis. As part of this development, the program learning outcomes are being integrated into the information system to link each course's learning outcomes. The new system will make it possible to perform a detailed data analysis to directly assess the program learning outcomes. The overall process guides the university in planning, improving, implementing, and monitoring based on stakeholder satisfaction and performance analysis. This research focuses on quantitative and qualitative analysis of program learning outcomes based on student performance and stakeholder assessment to determine educational achievement, and which can be used in the decision-making about developing or updating programs. 1. During the planning process, we developed a curriculum matrix mapping program learning outcomes onto each of the individual courses and its tasks that would be automated by our information system. So, based on this data, to ensure that all outcomes are covered in at least one course, and preferably more than one. 2. Reviewing the assessment results of determined learning outcomes and its performance. Additionally, Student self assessment is used for this investigation. 3. As well as stakeholders’ assessments are designed and used as significant indicators of educational achievement. So, the gap analysis was employed to examine differences between student performance and the stakeholders’ assessments. 4. To review external stakeholders’ assessment which learning outcomes are preferred or not and their expectations are met. Methods of Research: The research employs a mix of quantitative and qualitative data collection methods to comprehensively assess program education objectives (PEO) and program learning objectives (PLO) at UFE. The following research methods are used in this research. • Quantitative Analysis: Quantitative data from assessments and student performance is statistically analyzed to measure the achievement of program learning outcomes. This includes descriptive statistics, inferential statistics, and potentially regression analysis to identify significant factors influencing outcomes. • Qualitative Analysis: Qualitative data, including stakeholder assessments and student self-assessment narratives, is analyzed using thematic analysis or content analysis to identify recurring themes, patterns, and insights. • Gap Analysis: The gap analysis assesses the differences between student performance and stakeholder assessments, high lighting areas where perceptions may differ. • External Stakeholder Assessment Review: The preferences and expectations of external stakeholders are analyzed to determine which learning outcomes are preferred and if they align with UFE's educational objectives. • By employing these research methods, this study aims to contribute valuable insights into the continuous improvement system of UFE and provide evidence-based recommendations for enhancing the outcome planning and assessment processes within the institution.

Course-embedded assessment in Korean nursing baccalaureate education

Hyun Kyoung Kim 한국의학교육학회 2016 Korean journal of medical education Vol.28 No.3

Purpose: This study aimed to evaluate the program outcome in nursing baccalaureate in Korea. The analysis based on course embedded assessment. The concrete objectives were establishment of program outcome assessment, confirmation of student competency through weighting of program outcomes, and using the results of the evaluation in the circular feedback process in a nursing school in Korea. Methods: This study was conducted with a nursing education curriculum in a Korean nursing school. Data were collected through 28 students’ program outcome measurement from January 2013 to December 2014. Data were analyzed using a pairwise comparison method and analytic hierarchy process. Results: There were 1 to 3 direct and indirect assessment tools and for each program outcomes and each tool had measurable rubrics. There were 1 to 3 direct assessment tools for each program outcome, and each tool had measurable rubrics. This model derived rank of program outcomes from "care integration" to "global perception" through weight calculation. All direct assessment results were over 70%. The indirect assessment results were over the cutoff except for program outcomes 4 and 7. Conclusion: Each step of course embedded assessment was adaptive in nursing program outcome measure. The achievement of learning outcome provided reasonable tools for faculty and students.

정신분열병에서의 결과 평가

김철권 釜山大學校 醫科大學 精神科學敎室 1998 釜山精神醫學 Vol.7 No.-

Outcomes assessment imply an objective measure of an occurrence between two points in time. In clinical trials, the outcomes assessment generally represent the change as a result of a specific intervention. Outcomes can be assessed at different levels, such as efficacy, effectiveness, and quality assessment. Integrating outcomes assessment into clinical practice will serve several functions in addition to improving patient care. First, it can facilitate the identification and promotion of more effective therapies and help to identify ineffective treatment Second, the development and adoption of treatment guidelines can be linked to specific outcomes. Third, outcomes research may affect educational and training curricula of clinicians. Fourth, outcomes studies can provide clinicians with better predictive information about the etiology, psychopathology, and treatment. Fifth, it help to establish the effective mental health management and delivery system. The domain of outcomes assessment in schizophrenia depend on the perspective and interest of different individuals and groups. Traditionally, the clinician has a great emphasis on the symptomatic assessment. Functional assessmentis a central consideration in the view of patients. Family have much interest in the domain of the burden assessment, Satisfaction with psychiatric service is a major concern of not only service provider such as hospital and insurance company but also service consumer such as patient and family.

성과중심교육에서 학생평가

임선주,Im, Sun Ju 연세대학교 의과대학 2013 의학교육논단 Vol.15 No.1

Assessment plays a vital role in outcome-based education (OBE). This article describes the characteristics of assessment and appropriate assessment instruments for measuring learning outcomes in OBE. Assessment in OBE needs to be formative, continuous, and frequent. Miller's pyramid is useful for selecting the appropriate assessment instruments to reflect a specific outcome; different methods can be applied to evaluate one outcome. Outcomes and competency mean that student must 'do'; therefore, performance tests are emphasized. Qualitative methods as well as quantitative methods are used to evaluate the outcomes of areas such as professionalism or ethics. An absolute criterion-based standard is usually applied to decide whether students pass or fail, but the decision should be based on gathering value judgments and reaching consensus. Active participation of faculty members and students in assessment is crucial.

간호학 학사학위 프로그램 학습성과 평가체계 개발 사례 : 의사소통 능력 향상을 중심으로

김복남,김순옥 한국간호과학회 간호행정학회 2014 간호행정학회지 Vol.20 No.2

연구목적 : 간호학 학사학위 프로그램에 대한 간호교육인증의 핵심은 성과중심(outcomes based) 교육을 바탕으로 지속적 교육과정 개선 구조를 통하여 졸업생의 능력 및 자질을 향상시키는 것이다. 이러한 간호교육인증 기준의 학습성과 달성을 위하여 간호교육 평가와 교육목표, 과정을 이해하는 것이 필요하다. 연구방법 : 본 연구는 프로그램 학습성과 평가방법을 중점적으로 다루었다. 먼저, 프로그램 학습성과 평가방법의 일반적인 문제점을 논한 다음, 프로그램 학습성과 평가방법 개선의 중점사항을 기술하였다. 다음에는 프로그램 학습성과 평가체계와 평가방법을 구체적인 예를 들어 설명하였으며, 채점기준을 활용한 학습성과 평가방법의 장점을 제시하였다. 연구결과 : 본 연구는 한국간호평가원에서 제시하는 학습성과 12개중 3번째 학습성과인 의사소통 적용 관련 에 대한 평가도구와 채점기준을 실제 예를 들어 구체적으로 제시하였다. 결론 : 본 연구결과에서 제시된 평가도구는 향후 이러한 평가체계를 활용한 성과기반 교육과정의 품질개선을 통해 간호교육의 질을 향상시키는데 기여할 것으로 사료된다. Purpose: The core of accreditation for nursing education is to improve the ability and quality of graduates through a CQI system based on outcomes. For program outcomes of nursing education accreditation criterion, it is necessary to understand the relationship among education assessment, objective and curriculum. Methods: The focus of this study was on program outcomes and the assessment. First measures for improving the general problems of program outcomes associated with the assessment system were stated. Detailed measure for practicing program outcomes, the assessment system and the assessment tool using Rubrics were discussed. Results: Detailed measure were established for practicing the program outcomes, assessment system and assessment tool, which makes use of Rubrics, program outcome No 3 of the Korean Accreditation Board of Nursing Education. Conclusion: Therefore it is expected that the assessment system established in this study will allow universities to conduct program outcomes and assessment.

노인등록연구 사례를 통한 임상평가지표 선정 과정 및 검증된 설문도구 승인 경험의 공유

조나현,전형선,하원배,이정한,고미미,김영은,정지연,임정태 대한한의학회 2024 대한한의학회지 Vol.45 No.1

Objectives: Underpinned by the context of a Korean traditional medicine cohort study on healthy aging, this research primarily aims to guide the selection of Clinical Outcome Assessments (COAs) for elderly healthy aging patient registry research, offering insights into the selection process; and secondly, to streamline the resource-intensive process of obtaining permissions for validated COAs, benefiting future traditional Korean medicine clinical researchers. Methods : In this study, we identified outcomes through a review of previous studies, followed by a process involving expert consultations to select the final outcomes. Subsequently, for the selected outcomes that were Clinical Outcome Assessments (COAs) developed tools, we searched in commercial databases to confirm the availability of Korean versions and the necessity of obtaining permissions. Finally, we obtained permissions for their utilization and, when needed, acquired the original instrument questionnaire through payment. Results: Through a literature review of existing observational studies, a total of 57 outcomes were selected, with 19 of them identified as COA instruments. Upon verifying usage permissions for these 19 instruments, it was found that 17 required author-specific permissions, and among these, 2 needed a purchase as they were commercially available. Conclusion: This study provides a detailed overview of outcome selection and permission acquisition for elderly patient registry research. It underscores the importance of Clinical Outcome Assessment (COA) tools and the rigorous approval process, aiming to enhance research reliability. Continuous verification of COA information is essential, and future research should explore Core Outcome Set (COS) development through consensus-building approaches like Delphi studies.

A Team Project Based Assessment Method for Engineering Design Course

Jongwan Kim 보안공학연구지원센터 2016 International Journal of Multimedia and Ubiquitous Vol.11 No.8

Implementing prototypes and engineering design education that requires creativity among engineering students has become a universal trend. Engineering design education is a subject taught in a setting where students gather in a team, decide on a single topic, and cooperate with one another to perform a project. Team project based evaluation is a main part of engineering design education. Many instructors agree that harmonizing creative evaluation and outcome assessment is difficult. To resolve this issue, we have surveyed various creativity assessments and engineering design education methodologies, and have modelled existing evaluating elements into a creative process and outcome assessment framework for each team project assessment. We have evaluated students in three levels (fair, good, and excellent) for each assessment element. Based on the survey results from students who took the course, we concluded that the new assessment model is an innovative method of assessment that enhanced both the students’ and the instructor’s satisfaction of the course.

프로그램 학습성과 평가방법 연구

김복기(Bokki Kim),박진영(Jinyoung Park) 한국공학교육학회 2008 공학교육연구 Vol.11 No.4

공학교육인증의 핵심은 “성과중심(outcomes based)” 교육을 바탕으로 지속적 자율 개선 구조를 통하여 졸업생의 능력 및 자질을 향상시키는 것이다. 공학 교육 인증기준 중 학습성과의 경우 교육평가, 교육 목표, 교육과정과의 관계 속에서 올바르게 이해될 필요가 있다. 본 논문에서는 프로그램 학습성과 평가방법을 중점적으로 다루었다. 먼저 프로그램 학습 성과 평가방법의 일반적인 문제점을 개선할 수 있는 방안을 기술하였다. 또한 프로그램 학습성과 평가체 계와 루브릭을 활용한 평가도구의 구체적 실행 방법을 논하였다. 따라서 본 연구를 통하여 수립된 평가 도구를 통하여 대학현장에서 학습성과 평가가 실제 적으로 운영될 수 있기를 기대한다. The core of accreditation for engineering education is to improve the ability and quality of graduates through CQI system based on outcomes. For the program outcomes of the engineering education accreditation criterion, it is necessary to understand the relationship among education assessment, objective and curriculum. This study has focused on the program outcomes and the assessment. First the measure for improving the general problems of the program outcomes associated with assessment tool have been stated. And the detailed measure for practicing the program outcome, the assessment system and the assessment tool, which makes use of Rubrics, have been discussed. Therefore it is expected that the program outcome and the assessment may be conducted at universities with the assessment tool established in this study.

캡스톤디자인 교과목 기반의 프로그램 학습성과 평가 연구

이희원(Hee Won Lee),김성환(Sung Hwan Kim),박 근(Keun Park),김정엽(Jung Yup Kim) 한국공학교육학회 2010 공학교육연구 Vol.13 No.6

본 논문은 서울과학기술대학교(구 서울산업대학교) 기계설계자동화공학부에서 16년간의 캡스톤디자인 운영경험과 개선을 바탕으로 캡스톤디자인 교과목 기반의 프로그램 학습성과의 평가 사례에 대해 서술하였다. 우선, 캡스톤디자인 교과목과 관련된 프로그램 학습성과를 소개하였고, 평가 방법 및 항목에 대해서 설명하였다. 이를 바탕으로 하여 2008년도의 학습성과 평가 결과를 도출하였고, 이를 분석한 후 개선 전략을 수립하였다. 전략으로서 학생들에게 심사 기준과 모범사례를 온라인상에 미리 공개하였고, 그 결과 2009년도에는 전반적으로 약 3.3% 개선된 학습성과 평가 결과를 얻을 수 있었다. 마지막으로 코스 임베 디드 평가 개념 측면에서 캡스톤디자인 교과목을 이용한 프로그램 학습성과 평가의 타당성, 효율성, 내실성, 그리고 지속가능 성에 대해서도 논의하였다. This paper presents a case study on the assessment of program outcomes using the capstone design course based on 16-year management experience and improvements of the school of mechanical design and automation engineering in the seoul national university of science and technology. First, the program outcomes related with the capstone design course are introduced, and the assessment methods and items are described. Based on them, the assessment results in 2008 were obtained and a strategy to improve the results were built after analyzation. As for the strategy, we revealed the assessment items and good examples in an internet web site in advance. As a result, we were able to obtain about 3.3% improved assessment results in 2009. Finally, the validity, efficiency, internal stability, and sustainability of the assessment of the program outcomes using the capstone design course are addressed in terms of a notion of the course-embedded assessment.

공학교육의 프로그램 학습성과 평가를 위한 코스 임베디드 평가의 적용 가능성 고찰

한지영 한국공학교육학회 2009 공학교육연구 Vol.12 No.3

The purpose of this study was to suggest possibility of adaptation of course-embedded assessment for program outcome in engineering education through literature review on concept, benefits, adaptation process, case study in higher education of course-embedded assessment.The model of course-embedded assessment was given with consideration for each step, that is, set up of program outcomes, check of related courses, selection of assessment tools, set up of performance criteria, implementation of course-embedded assessment, and analysis of results. Some prerequisites were given to set up course-embedded assessment successfully in engineering education field of Korea. 이 연구는 ‘코스 임베디드 평가’의 개념, 효과, 적용방법 및 실제 적용된 사례를 살펴봄으로써 공학교육 분야에의 적용 가능성을 제안하기 위하여 수행되었다. 이를 위해 코스 임베디드 평가를 주요어(key word)로 하여 이의 개념, 효과, 적용 방법 및 적용 사례를 중심으로 한 ‘문헌연구’를 수행하였다. 본 연구를 통해 코스 임베디드 평가의 모형과 모형의 각 단계별 유의사항을 제시하였다. 제시된 코스 임베디드 평가의 모형 단계는 다음과 같다. 즉, 프로그램 학습성과의 설정, 관련 코스의 확인, 평가도구의 선정, 평가 준거의 설정, 평가 시행 및 평가 결과의 분석이다. 또한 우리나라에서 코스 임베디드 평가가 성공적으로 정착하기 위해 선행되어야 할 조건들을 함께 제시하였다.