정부업무평가와 국가연구개발사업평가 활용의 영향요인 비교 연구

다국어 초록 (Multilingual Abstract)

Most of the empirical research papers on policy evaluation utilization have tried to explain from the standpoint of frequency of influencing factors. Some papers dealt with the influencing factors of policy evaluation utilization by using regression analysis techniques. However, both frequency and regression analysis techniques have their own limits in explaining policy evaluation utilization. To Understand policy evaluation utilization, it is required to figure out which influencing factors are more important in a particular setting and how the causal relationships between them are created. Because policy evaluation utilization can be well explained by analyzing the complexities of the correlation between cause and effect.
Especially, several research papers published in Korea have some limits in examining the influencing factors of policy evaluation utilization, and figuring out which influencing factors are more relevant to domestic situations than others. Furthermore, most of them do not fully explain which influencing factors are more important than others in a particular setting, and how the causal relationships between those influencing factors are created. Therefore, the objective of this dissertation is to analyze the influencing factors of policy evaluation utilization and the causal relationships between them in a comparative method with investigating public service evaluation and National R&D Programs evaluation, which are the two main policy evaluation subjects in Korea.

2. Method
In this dissertation I used content analysis by sorting out the influencing factors of policy evaluation utilization enumerated in previous research papers. Sixteen influencing factors were selected as important ones in terms of utilization context, evaluation institution, interaction, evaluation results, and acceptance as a result of rigorous content analysis. The analysis of influencing factors in the dissertation was conducted by surveying public service evaluation and National R&D Programs evaluation carried out in 2006 and 2007 in Korea. In total, sixty-eight national institutions were subject to the survey. Of those, thirty were accounted for public service evaluation; thirty-eight were surveyed for National R&D Programs evaluation. 1,360 copies of the survey questionnaire were distributed, and 1,161 copies were collected. Among them, 1,124 copies were used for analysis in this dissertation. I have engaged in exploratory factor analysis, reliability test, and confirmatory factor analysis, respectively, in order to verify the validity and reliability of those collected materials for research purposes. And the explaining capability of influencing factors was examined with regression analysis, and the causal relationships between influencing factors was analyzed through Structural Equation Modeling(SEM).

3. Findings
The following are the results of regression analysis of influencing factors.
In the case of public service evaluation, acceptance, interaction, evaluation institution, and organizational context are more important in order in terms of the explaining ability of influencing factors. Expression of evaluation results factor is excluded from regression analysis. Meanwhile, in the case of National R&D Programs evaluation, acceptance, evaluation results, interaction, and Organizational Context rank from highest to lowest in terms of the explaining ability of influencing factors. The factor of evaluation institution is excluded from regression analysis.

The analytic results of the causal relationships between influencing factors are as follows.
① Regardless of evaluation types, it has been found that acceptance, evaluation institution, and interaction rank high in terms of total effect on utilization. ② Organizational Context, interaction, and acceptance have direct effect on policy evaluation utilization in the case of public service evaluation. On the other hand, National R&D Programs evaluation has only one influencing factor of such effect: acceptance. ③ Public service evaluation and National R&D Programs evaluation have three influencing factors of indirect effect on policy evaluation utilization: evaluation institution, interaction, and expression of evaluation results. Organizational Context is excluded from indirect effect.

4. Implications
1) Theoretical Perspectives
First, in this dissertation, I have found that there are diverse utilization types and relevant influence factors are different. In addition, the causal relationships between influence factors are different, depending on evaluation types.
Second, I have also found that policy evaluation utilization is a complex process in which various perspectives such as organizational interest, communications, and rationals integrate with one another dynamically.
Third, the dissertation shows empirically that the assumptions behind previous research papers can apply in a particular case in a different manner. ① The basic assumptions behind previous researches cannot apply where users are not satisfied with the evaluation institution. ② In the case where objectivity and fairness of an evaluation are stressed, interaction between evaluators and users is quite limited. Also, in an organizational atmosphere where such interaction is perceived negatively, interaction as an influence factor does not have direct effect on policy evaluation utilization. ③ In the case where the contents of evaluation results are highly complex and in detail, policy evaluation utilization does not take place in the first place, even if organizational norms dictating utilization of evaluation results exist. In this case, policy evaluation utilization depends largely on the degree of users' acceptance.
Fourth, the dissertation is the first study that has performed an empirical analysis of imposed use both at home and abroad. Most of the previous research papers did not present imposed use as one of the influencing factors. Therefore, it is needed to approach imposed use as one of the influencing factors from a different standpoint.

2) Policy Implications
First, Organizational Context(utilization institution) needs to be improved. It should be pointed out that we must correct our misconception that policy evaluation utilization takes place almost automatically, if institutional norms concerning policy evaluation utilization are established. It is necessary to consider that possible users' selectiveness will play a role in utilization encouragement from an institutional standpoint.
Second, improvement of evaluation institution is required. In this dissertation, I have found that many users are not satisfied with the evaluation institution. Such improvement necessitates the employment of evaluators with reputation and policy expertise in a relevant policy area. And, it is needed to precisely establish evaluation methods in a manner that enhances the credibility of evaluators and is more acceptable to users. It is also desirable that the evaluators make an evaluation plan after consultation with the users.
Third, a more active interaction is needed. An ideal interaction between evaluators and users does not take place by means of the submit of evaluation materials and the existence of an opportunity of challenging the evaluation results. Thus, a highly effective interaction requires beyond reaching a consensus on evaluation procedures and results. Also, users' negative conception on interaction should be changed in a positive direction.
Fourth, expression methods of evaluation results should be improved. It is necessary that evaluators make an effort to find an effective way of explanation which helps users understand the results of policy evaluation. So the evaluators should know who the users are and how intelligent they are prior to engaging in policy evaluation. Moreover, the evaluators should submit their evaluation papers before the budget and programs plans of the administrative agency concerned are established, in order to facilitate policy evaluation utilization.
Fifth, it is needed to find out a way in which policy evaluation can enhance users' acceptance. Most importantly, the evaluators should try to present evaluation results in a manner that can satisfy users' information needs. Also, the users should be more open-minded to the evaluation results presented by the evaluators.

목차 (Table of Contents)

• 제1장 서론 1
• 제1절 연구 목적 1
• 제2절 연구 범위 4
• 제3절 연구 방법 6
• 제2장 이론적ㆍ제도적 배경 10
• 제1절 정책평가 활용연구의 등장과 연구 경향 10
• 1.정책평가 활용연구의 등장과 중요성 10
• 2.정책평가 활용연구의 경향 13
• 제2절 정책평가 활용의 의의와 유형 31
• 1.정책평가 활용의 의의 31
• 2.정책평가 활용의 유형 45
• 3.정책평가 목적과 정책평가 활용 55
• 제3절 정책평가 활용의 영향요인 58
• 1.선행연구에서 제시된 정책평가 활용의 영향요인 검토 58
• 2.선행연구의 한계를 극복하기 위한 접근방법 69
• 3.내용분석에 의한 정책평가 활용의 영향요인 도출 72
• 제4절 정부업무평가와 국가연구개발사업평가 88
• 1.정부업무평가의 특성과 활용 88
• 2.국가연구개발사업평가의 특성과 활용 92
• 3.정부업무평가와 국가연구개발사업평가의 비교 95
• 제3장 연구 설계 98
• 제1절 연구의 분석 틀 98
• 1.분석 틀의 설정 98
• 2.분석요소의 정의 103
• 3.연구가설의 설정 112
• 제2절 자료 수집 및 분석 방법 121
• 1.자료의 수집 121
• 2.자료의 분석 방법 127
• 3.설문의 구성 128
• 제4장 조사결과 분석 및 논의 130
• 제1절 응답자의 특성 및 예비 분석 130
• 1.응답자의 개인적 특성 130
• 2.예비 분석 132
• 3.수정된 분석 틀 및 연구가설 141
• 제2절 정책평가 활용에 대한 응답 분포 및 차이 분석 143
• 1.정책평가 활용의 영향요인과 활용유형에 대한 응답 분포 143
• 2.평가유형에 따른 영향요인과 활용유형의 차이 분석 150
• 3.응답자 소속에 따른 영향요인과 활용유형의 차이 분석 153
• 제3절 정책평가 활용에 대한 영향요인의 영향력 분석 160
• 1.영향요인과 활용유형의 상관관계 분석 160
• 2.정책평가 활용에 대한 영향요인의 다중회귀분석 163
• 3.다중회귀분석 결과에 의한 영향요인 비교 170
• 제4절 정책평가 활용의 영향요인들 간 인과관계 분석 174
• 1.연구모형의 적합도 분석 174
• 2.평가유형에 따른 영향요인들 간 인과관계 분석 178
• 3.인과관계분석 결과에 의한 영향요인 비교 186
• 4.연구가설의 검증 200
• 제5절 분석 결과의 종합 및 정책적 시사점 205
• 1.분석 결과의 종합 205
• 2.정책적 시사점 210
• 제5장 결 론 220
• 제1절 연구 결과의 요약 220
• 제2절 연구의 함의 222
• 1.이론적 함의 222
• 2.정책적 함의 227
• 제3절 연구의 한계와 향후 연구 과제 228
• 참고문헌 229
• [부록 1] 표 1 1차 내용분석 결과 246
• [부록 2] 표 2 2차 내용분석 결과:활용 맥락-① 조직맥락 252
• [부록 3] 설문지 1 260
• [부록 4] 표 3 연구대상 전체에 대한 활용유형의 평균 및 표준편차 268
• ABSTRACT 275