신경망 학습의 일반화 성능향상을 위한 인자들의 결합효과 = The Joint Effect of factors in Generalization Performance of Neural Network Learning Procedure

다국어 초록 (Multilingual Abstract)

The goal of this paper is to study the joint effect of factors of neural network learning procedure. There are many factors, which may affect the generalization ability and learning speed of neural networks, such as the initial values of weights, the learning rates, and the regularization coefficients. We will apply a constructive training algorithm for neural network, then patterns are trained incrementally by considering them one by one. First, we will investigate the effect of these factors on generalization performance and learning speed. Based on these factors’ effect, we will propose a joint method that simultaneously considers these three factors, and dynamically tune the learning rate and regularization coefficient. Then we will present the results of some experimental comparison among these kinds of methods in several simulated nonlinear data. Finally, we will draw conclusions and make plan for future work.

국문 초록 (Abstract)

본 연구에서는 신경망 학습의 일반화 성능과 학습속도를 개선시키기 위한 인자들의 결합 효과를 살펴본다. 신경망 학습에서 중요한 평가 척도로서 여기서 고려하는 인자들에는 초기 가중값의 범위와 학습률 그리고 계수조정 등이 있다. 특히 초기 가중값과 학습률을 고정시킨 후 새롭게 조정된 계수들을 단계적으로 변화시키는 새로운 인자 결합방법을 이용한다. 이를 통하여 신경망 학습량과 학습속도를 비교해 보고, 계수조정을 통한 개선된 학습 영향을 살펴본다. 그리고 비선형의 단순한 예제를 이용한 실증분석을 통하여 신경망 모형의 일반화 성능과 학습 속도 개선을 위한 각 인자들의 개별 효과와 결합 효과를 살펴보고 그 개선 방안을 논의한다.

참고문헌 (Reference)

1 M.H. Hassoun, "‘Fundamentals of Artificial Neural Networks’" MIT Press 1995.

2 D.R. Wilson, "The Need for Small Learning Rates on Large Problem" 115-119, 2001.

3 Y. Wu, "The Effect of Initial Weight Learning Rate and Regularization on Generalization Performance and Efficiency" 1191-1194, 2002.

4 V. Cherkassky,R. Shepherd, "Regularization effect of weight initialization in back propagation networks" 2258-2261, 1998.

5 S. Bo, "Optimal Weight Decay in Perceptron" 551-556, 1996.

6 R.A. Jacobs, "Increased Rates of convergence Through Learning Rate Adaptation" 1 (1): 295 -307, 1988

7 A. Atiya, "How Initial Conditions affect Generalization Performance in Large Networks" 8 (8): 448 -451, 1997

8 D.E. Rumelhart, "Generalization by Weight-Elimination with Application to Forecasting in Advances in Neural Information Processing Systems" Morgan Kaufmann 875-882, 1991.

9 T. Tollenaere, "Fast Adaptive back-propagation with Good Scaling Properties" 3 (3): 561-573, 1990.

10 Y. Zhang,D. Liu, "A New Learning Algorithm for Feedforwad Neural Networks" 39-44, 2001.