그래프 합성곱-신경망 구조 탐색 : 그래프 합성곱 신경망을 이용한 신경망 구조 탐색

최수연,박종열 국제문화기술진흥원 2023 The Journal of the Convergence on Culture Technolo Vol.9 No.1

This paper proposes the design of a neural network structure search model using graph convolutional neural networks. Deep learning has a problem of not being able to verify whether the designed model has a structure with optimized performance due to the nature of learning as a black box. The neural network structure search model is composed of a recurrent neural network that creates a model and a convolutional neural network that is the generated network. Conventional neural network structure search models use recurrent neural networks, but in this paper, we propose GC-NAS, which uses graph convolutional neural networks instead of recurrent neural networks to create convolutional neural network models. The proposed GC-NAS uses the Layer Extraction Block to explore depth, and the Hyper Parameter Prediction Block to explore spatial and temporal information (hyper parameters) based on depth information in parallel. Therefore, since the depth information is reflected, the search area is wider, and the purpose of the search area of the model is clear by conducting a parallel search with depth information, so it is judged to be superior in theoretical structure compared to GC-NAS. GC-NAS is expected to solve the problem of the high-dimensional time axis and the range of spatial search of recurrent neural networks in the existing neural network structure search model through the graph convolutional neural network block and graph generation algorithm. In addition, we hope that the GC-NAS proposed in this paper will serve as an opportunity for active research on the application of graph convolutional neural networks to neural network structure search. 본 논문은 그래프 합성곱 신경망을 이용한 신경망 구조 탐색 모델 설계를 제안한다. 딥 러닝은 블랙박스로 학습이 진행되는 특성으로 인해 설계한 모델이 최적화된 성능을 가지는 구조인지 검증하지 못하는 문제점이 존재한다. 신경망 구조 탐색 모델은 모델을 생성하는 순환 신경망과 생성된 네트워크인 합성곱 신경망으로 구성되어있다. 통상의 신경망 구조 탐색 모델은 순환신경망 계열을 사용하지만 우리는 본 논문에서 순환신경망 대신 그래프 합성곱 신경망을 사용하여 합성곱 신경망 모델을 생성하는 GC-NAS를 제안한다. 제안하는 GC-NAS는 Layer Extraction Block을 이용하여 Depth를 탐색하며 Hyper Parameter Prediction Block을 이용하여 Depth 정보를 기반으로 한 spatial, temporal 정보(hyper parameter)를 병렬적으로 탐색합니다. 따라서 Depth 정보를 반영하기 때문에 탐색 영역이 더 넓으며 Depth 정보와 병렬적 탐색을 진행함으로 모델의 탐색 영역의 목적성이 분명하기 때문에 GC-NAS대비 이론적 구조에 있어서 우위에 있다고 판단된다. GC-NAS는 그래프 합성곱 신경망 블록 및 그래프 생성 알고리즘을 통하여 기존 신경망 구조 탐색 모델에서 순환 신경망이 가지는 고차원 시간 축의 문제와 공간적 탐색의 범위 문제를 해결할 것으로 기대한다. 또한 우리는 본 논문이 제안하는 GC-NAS를 통하여 신경망 구조 탐색에 그래프 합성곱 신경망을 적용하는 연구가 활발히 이루어질 수 있는 계기가 될 수 있기를 기대한다.

Application of Neural Network Model to Vehicle Emissions

김대현,이정 서울시립대학교 도시과학연구원 2010 도시과학국제저널 Vol.14 No.3

The issue of air quality is now a major concern around the world and the vehicle emissions model is very important. Most of the current vehicle emission models are multiple regression techniques. In this study, a neural network-based model has been proposed to achieve better estimation accuracy. The estimation performance of two models, the proposed neural network-based model and a general regression model, has been compared using mean absolute error (MAE). A comparative study between two models to estimate vehicle emissions, the proposed neural network-based model and a general regression model, has been conducted to assess the estimation performance of the proposed model in terms of mean absolute percentage error. Experimental results in this study revealed that the neural network model performed better as it was able to decrease the error for emission estimation comparing with the multiple regression models. More importantly, in this study a lookup table (LUT) method has been proposed to overcome the black-box problem, which is a disadvantage of the neural network models. It could be useful for any other researches to estimate emissions without developing and training the neural network model which can be a time-consuming task. The issue of air quality is now a major concern around the world and the vehicle emissions model is very important. Most of the current vehicle emission models are multiple regression techniques. In this study, a neural network-based model has been proposed to achieve better estimation accuracy. The estimation performance of two models, the proposed neural network-based model and a general regression model, has been compared using mean absolute error (MAE). A comparative study between two models to estimate vehicle emissions, the proposed neural network-based model and a general regression model, has been conducted to assess the estimation performance of the proposed model in terms of mean absolute percentage error. Experimental results in this study revealed that the neural network model performed better as it was able to decrease the error for emission estimation comparing with the multiple regression models. More importantly, in this study a lookup table (LUT) method has been proposed to overcome the black-box problem, which is a disadvantage of the neural network models. It could be useful for any other researches to estimate emissions without developing and training the neural network model which can be a time-consuming task.

딥러닝 신경망을 이용한 신용카드 부도위험 예측의 효용성 분석

윤종문 한국금융학회 2019 금융연구 Vol.33 No.1

This study aims to discuss the usefulness of the deep learning neural network and the possibility of the deep learning neural network analysis in judging credit information by using credit card default data. Deep learning neural network analysis in the financial sector excluding the current stock price prediction model is under limited research. It is mainly used for upgrading models of the credit rating (Kvamme et al., 2016, 2018; Tran, 2016; Luo, 2017) and the delinquency rate (Sirignano et al., 2018). In the credit card market, it is focused on credit card issuance and fraud detection model (Ramanathan, 2014, Niimi, 2015). As mentioned earlier, there has not been much analysis of deep learning neural network using financial market data. This is because the study of deep learning neural networks is actively carried out mainly in the field of computer science such as image, speech recognition, natural language processing. Additionally, Researchers in the financial sector have difficulty learning deep learning algorithms and setting up a computer runtime environment. It is also difficult to apply the algorithm to financial data due to lower dimension than the image. Nowadays, financial companies have been interested in machine learning and are increasing their recruitment, but it is still in the stage of verifying the possibility of deep learning neural network. Therefore, This study examines the possibility of improving the accuracy of credit card default risk prediction by using a deep learning neural network algorithm. To do this, we use existing machine learning algorithms (Logistic, SVM, Random Forest, Lasso, etc.) as a comparison index for performance check of deep learning neural network analysis. Firstly, the deep learning neural network is constructed with two hidden layers and five neurons, and derives the prediction accuracy according to the activation function and the initial value setting method. There are Sigmoid, ReLU, tanh and Maxout as active functions, and random value, Xavier, RBM, He’s as initialization methods. Based on this, we compare the accuracy of existing machine learning algorithms. As a result, the deep learning neural network analysis showed performance improvement between 0.6% and 6.6%p compared to the existing machine learning algorithms (Logistic, SVM, Random Forest, Lasso, etc.). Among these results, the active function and the initial value setting method with the highest prediction accuracy are ReLU (rectified linear units) and Xavier initialization. However, there is no significant improvement in performance with increasing number of hidden layers and neurons up to 10 and 25, respectively. Also, the dropout and CNN (convolution neural network) models, which showed high performance in the field of image identification, showed no significant difference in prediction accuracy. Nevertheless, it could be interpreted that the increase of hidden layers can improve the accuracy of estimation because the highest accuracy (0.8161) and the AUC (0.7726) are observed for 10 hidden nodes and 15 neurons. However, we can’t say that accuracy increases linearly by the number of hidden layers and neurons. These limitation could be due to the quantitative and qualitative limitations of the credit card data used here. We did not use recurrent neural network (RNN) and long-short term memory (LSTM) models since the personal default data for credit card used in this study is cross-sectional data. These method are for Time-Series data. Therefore, it is expected that it will be able to obtain better results in identification problems (credit rating, delinquency rate, interest rate calculation) of present various financial markets if these deep learning neural network methodologies are applied through big data including time series data. This study can be turned into a question of how deep learning analysis can lower the default risk and delinquency rate by using financial data from a practical point of ... 본 연구는 국내․외 금융시장에서 아직 활성화되지 못한 딥러닝 신경망(deep learning neural network) 알고리즘을 이용해 신용카드 부도위험 예측의 정확도 향상 가능성에 대해서 점검한다. 이를 위해 기존 머신러닝 알고리즘(Logistic, SVM, Random Forest, Lasso 등)을 딥러닝 신경망 분석의 성능 점검을 위한 비교 지표로 활용한다. 우선, 딥러닝 신경망은 두 개의 은닉층(hidden layers)과 다섯 개의 뉴런(neuron)으로 구축하고, 활성함수(activation function)와 초기값(initial value) 설정방법에 따른 예측정확도를 도출한다. 그 결과 딥러닝 신경망 분석이 기존 머신러닝 알고리즘 보다 최소 0.6%p에서 최대 6.6%p 성능이 향상된 것으로 나타났다. 이 중 가장 높은 예측 정확도를 보인 활성함수와 초기값 설정방식은 ReLU(rectified linear units)와 Xavier(2010)이고 이를 기준으로 은닉층과 뉴런의 수를 각각 최대 10개와 25개까지 늘려 분석한 결과에서도 유사한 결과가 나타났다. 다만, 기존 연구에서와 같이 은닉층과 뉴런의 수의 증가에 따른 뚜렷한 성능의 향상은 나타나지 않았다. 또한, 이미지 식별 분야에서 높은 성능을 보였던 Dropout과 CNN(convolution neural network) 모델도 예측 정확도에서 큰 차이를 보이지 않았다. 이는 여기에서 사용된 신용카드 데이터가 다수 픽셀(pixel)로 이루어진 이미지 데이터와 비교해 양적․질적 한계가 있기 때문으로 판단된다. 한편, 본 연구에서 사용된 개인의 신용카드 부도 데이터는 횡단면 자료이기 때문에 시계열 데이터에서 높은 성능을 나타내는 RNN(recurrent neural network) 및 LSTM(Long- Short Term Memory) 등의 딥러닝 신경망 알고리즘을 사용하지는 않았다. 따라서 추후 시계열 자료가 포함된 빅데이터를 통해 이들 딥러닝 신경망 방법론을 적용한다면, 현재의 다양한 금융시장의 식별문제(신용등급, 연체율, 금리산정)에 있어 보다 향상된 결과를 도출할 수 있을 것으로 기대된다.

Geometry Characteristics Prediction of Single Track Cladding Deposited by High Power Diode Laser Based on Genetic Algorithm and Neural Network

Huaming Liu,Xunpeng Qin,Song Huang,Lei Jin,Yongliang Wang,Kaiyun Lei 한국정밀공학회 2018 International Journal of Precision Engineering and Vol.19 No.7

This paper aims to establish a correlation between the process parameters and geometrical characteristics of the sectional profile of the single track cladding deposited by high power diode laser with rectangle beam spot. By applying the genetic algorithm and back propagation neural network, a nonlinear model for predicting the geometry features of the single track cladding is developed. A full factorial design method is used to conduct the experiments, and the experimental results are chosen randomly as training dataset and testing dataset for the neural network. Three main input variables such as laser power, scanning speed, and powder thickness were considered. The performance of the genetic algorithm and back propagation artificial neural network was compared to that of the standard back propagation neural network. To improve the accuracy of the neural network, one-hidden-layer and double-hidden-layer neural network with different architectures were performed. Further, one-output and multi-output neural network are also trained and tested. The results indicate that, by using genetic algorithm, the prediction accuracy of the neural network is significantly improved. Meanwhile, the double-hidden-neural network has higher prediction accuracy than the one-hidden-layer-neural network, while the one-output-neural network has higher prediction accuracy than the multi-output-neural network.

Deep Neural Network와 Convolutional Neural Network 모델을 이용한 산사태 취약성 매핑

공성현,백원경,정형섭,Gong, Sung-Hyun,Baek, Won-Kyung,Jung, Hyung-Sup 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.6

Landslides are one of the most prevalent natural disasters, threating both humans and property. Also landslides can cause damage at the national level, so effective prediction and prevention are essential. Research to produce a landslide susceptibility map with high accuracy is steadily being conducted, and various models have been applied to landslide susceptibility analysis. Pixel-based machine learning models such as frequency ratio models, logistic regression models, ensembles models, and Artificial Neural Networks have been mainly applied. Recent studies have shown that the kernel-based convolutional neural network (CNN) technique is effective and that the spatial characteristics of input data have a significant effect on the accuracy of landslide susceptibility mapping. For this reason, the purpose of this study is to analyze landslide vulnerability using a pixel-based deep neural network model and a patch-based convolutional neural network model. The research area was set up in Gangwon-do, including Inje, Gangneung, and Pyeongchang, where landslides occurred frequently and damaged. Landslide-related factors include slope, curvature, stream power index (SPI), topographic wetness index (TWI), topographic position index (TPI), timber diameter, timber age, lithology, land use, soil depth, soil parent material, lineament density, fault density, normalized difference vegetation index (NDVI) and normalized difference water index (NDWI) were used. Landslide-related factors were built into a spatial database through data preprocessing, and landslide susceptibility map was predicted using deep neural network (DNN) and CNN models. The model and landslide susceptibility map were verified through average precision (AP) and root mean square errors (RMSE), and as a result of the verification, the patch-based CNN model showed 3.4% improved performance compared to the pixel-based DNN model. The results of this study can be used to predict landslides and are expected to serve as a scientific basis for establishing land use policies and landslide management policies.

감정평가에 기반한 환경과 행동패턴 학습을 위한 궤환 모듈라 네트워크

김성주(Seong-Joo Kim),최우경(Woo-Kyung Choi),김용민(Yong-Min Kim),전홍태(Hong-Tae Jeon) 한국지능시스템학회 2004 한국지능시스템학회논문지 Vol.14 No.1

감정은 지능을 지닌 존재의 이성판단에 영향을 준다. 그래서 주변 환경정보에 의해 평가된 기본적이고 보편적인 감정을 로봇에 가미하면 더욱 인간과 가까운 지능 로봇이 될 것이다. 그러나 인간의 감정을 학습하기 위해서는 다양한 감각정보의 학습과 패턴 분류가 선행되어야 하고 이를 위해서 적합한 네트워크 구조가 요구된다. 신경망은 시스템의 특징을 추출하는데 매우 우수한 능력을 발휘하고 있다. 그러나 일시적 혼선현상과 지역 최소치에 수렴하는 단점이 있다. 그래서 복잡한 문제를 단순한 여러 개의 부분적인 문제로 나누어 해결하는 모듈라 설계방법이 관심의 대상이 되고 있다. 본 논문에서는 수많은 감정평가와 학습 데이터 패턴들을 학습하기 위해서 재결합과 재구성에 탁월한 성능을 지닌 Jacobs와 Jordan이 제안한 모듈라 네트워크와 상황의 재 표현이 가능하고 예측값과 모델링에 적합한 특징을 지닌 궤환 신경망을 결합하였다. 구성된 구조는 기존의 모듈라 네트워크의 학습결과와 비교 검토하였다. Rational sense is affected by emotion. If we add the factor of estimated emotion by environment information into robots, we may get more intelligent and human-friendly robots. However, various sensory information and pattern classification are prescribed for robots to learn emotion so that the networks are suitable for the necessity of robots. Neural network has superior ability to extract character of system but neural network has defect of temporal cross talk and local minimum convergence. To solve the defects, many kinds of modular neural networks have been proposed because they divide a complex problem into simple several sub-problems. The modular neural network, introduced by Jacobs and Jordan, shows an excellent ability of re-composition and re-combination of complex work. On the other hand, the recurrent network acquires state representations and representations of state make the recurrent neural network suitable for diverse applications such as nonlinear prediction and modeling. In this paper, we applied recurrent network for the expert network in the modular neural network structure to learn data pattern based on emotional assessment. To show the performance of the proposed network, simulation of learning the environment and behavior pattern is proceeded with the real time implementation. The given problem is very complex and has too many cases to learn. The result will show the performance and good ability of the proposed network and will be compared with the result of other method, general modular neural network.

Computer Network Fault Diagnosis Based On Neural Network

Wang Qian 보안공학연구지원센터 2015 International Journal of Future Generation Communi Vol.8 No.5

Computer network is one of the world's most important infrastructures in twenty-first Century, network fault diagnosis has become the focus of attention. With the development of artificial intelligence, using the neural network technology into the network fault diagnosis area can play an important role to the advantages of neural network in fault diagnosis. In this paper, the method is widely used, which is combined the self organizing feature map (SOM) neural network and multilayer feedforward neural network (BP): The result of the training samples using SOM neural network clustering algorithm is added to the original training samples and set a certain weight, through iterative update to the weight, in order to improve the convergence the speed of BP neural network. Using computer network fault diagnosis as a practical example for the computer simulation and analysis developes a set of computer network diagnosis system can provide reference and assistance for the work of theory research and application.

Application of Data Fusion Technology Based on Weight Improved Particle Swarm Optimization Neural Network Algorithm in Wireless Sensor Networks

Xiajun Ding,Hongbo Bi,Xiaodan Jiang,Lu zhang 보안공학연구지원센터 2016 International Journal of Future Generation Communi Vol.9 No.3

With the development of sensor technology, network technology, embedded control technology and wireless communication technology, the application of wireless sensor networks (WSN) has become more and more widely. Wireless sensor networks have been named the most influential and important technology of the world in twenty-first Century. In wireless sensor networks, data fusion is an important research branch. In this paper, a data prediction model of wireless sensor network based on weight improved particle swarm optimization neural network algorithm is proposed. In view of the deficiency of the traditional BP neural network model, this paper combines with the characteristics of the data prediction model, and the BP neural network model is improved and integrated. After that, we train the neural network's sample set, and add the momentum item to correct the weight, so that the neural network can be predicted more quickly and accurately. The main idea of this paper is to predict the future data based on the historical data which are collected by sensor nodes, so as to achieve the purpose of reducing the amount of data transmission in the network and saving the energy of nodes. Finally, the experimental results show that the improved particle swarm optimization algorithm based on weight improved particle swarm optimization neural network algorithm has higher accuracy than the multiple regression method and the grey prediction method. In addition, the method can be used to effectively save energy in wireless sensor data transmission.

최적 EN를 사용한 MNN에 의한 Mobile Robot제어

최우경(Woo-Kyung Choi),김성주(Seong-Joo Kim),서재용(Jae-Yong Seo),전홍태(Hong-Tae Jeon) 한국지능시스템학회 2003 한국지능시스템학회논문지 Vol.13 No.2

이동로봇(Mobile Robot)의 자율주행 기능에는 추종, 접근, 충돌회피, 경고 등의 여러 기능이 있다. 이 기능들을 하나의 Neural Network로 구성하고 학습하는 것은 쉬운 일이 아니다. 이동로봇의 자율주행 기능들을 각각의 Module로 구성하고 상황에 맞게 학습된 Module의 출력 값으로 이동로봇을 제어하면 단일 신경망의 단점을 보안할 수 있을 것이다. 이동로봇은 인간의 감각을 대신할 수 있는 다중 초음파 센서와 USB 카메라를 장착하고 있으며, 이곳에서 측정된 환경정보 데이터들은 Modular Neural Network(MNN)을 통해 학습을 한다. Expert Network(EN)의 활성화 함수를 최적결합으로 MNN을 구성하였고, 그 구조는 학습시간과 오차를 개선할 수 있을 것으로 본다. Gating Network(GN)는 MNN의 출력값인 이동로봇의 진행 방향과 속도를 스위칭 함으로써 제어하는 역할을 한다. 본 논문에서는 Modular Neural Network(MNN) 내의 Expert Network(EN)을 최적설계 하였고, 제안한 MNN의 검증을 위해 실시간으로 반복하여 이동로봇에 구현하였다. 그 실험의 결과값은 로봇을 상황에 맞게 운행, 제어하였고, 만족할 만한 성과를 얻을 수 있었다. Skills in tracing of the MR divide into following, approaching, avoiding and warning and so on. It is difficult to have all these skills learned as neural network. To make this up for, skills consisted of each module, and Mobile Robot was controlled by the output of module adequate for the situation. A mobile Robot was equipped multi-ultrasonic sensor and a USB Camera, which can be in place of human sense, and the measured environment information data is learned through Modular Neural Network. MNN consisted of optimal combination of activation function in the Expert Network and its structure seemed to improve learning time and errors. The Gating Network(GN) used to control output values of the MNN by switching for angle and speed of the robot. In the paper, EN of Modular Neural network was designed optimal combination. Traveling with a real MR was performed repeatedly to verity the usefulness of the MNN which was proposed in this paper. The robot was properly controlled and driven by the result value and the experimental is rewarded with good fruits.

포스터 발표 : Neural Network을 이용한 Hepatitis C Virus Nonstructural 5A Region의 단백서열과 인터페론 치료 반응과의 관련성 분석

김진욱,이동호,김나영,황진혁 대한간학회 2003 Clinical and Molecular Hepatology(대한간학회지) Vol.9 No.3(S)

배경/목적: C형 간염바이러스(HCV)는 유전형에 따라 인터페론에 대한 감수성의 차이가 있으며, HCV group Ib는 인터페론 치료에 잘 반응하지 않음이 알려져있다. HCV nonstructural 5A protein (NS5A) 부위의 아미노산 변이가 인터페론에 대한 치료반응과 관련이 있다는 보고가 있어 interferon- sensitivity-determining region(ISDR)이라고 알려졌으나 이후의 보고들은 상반되는 결과를 보였는데, 기존의 분석에서는 단백서열 변이의 개수만 고려되었고 변이의 위치가 고려되지 않았다. Artificial neural network은 복잡한 패턴을 인식하는 문제 해결에 이용되는 model로서, 많은 양의 자료를 이용하여 network을 training시킴으로써 새로운 자료를 판별하거나 예측하는 데에 이용되는 분석기법이다. 연자 등은 인터페론과 ISDR의 변이에 관한 기존의 보고들을 neural network 기법으로 meta-analysis하여 치료반응과의 관련성을 분석하였다. 대상과 방법: PubMed에서 HCV genotype Ib ISDR의 단백서열과 인터페론 치료에 대한 반응 결과를 확인할 수 있었던 25편의 논문에 등재된 단백서열 1159 건을 분석하였다. 통계분석은 단백변이가 없는 wild type, 변이의 수가 1-3개인 intermediate type, 4개 이상인 mutant type으로 구분하여 Chi-square test로 유의성을 검정하였으며, neural network 분석은 supervised training으로는 multilayer perception(MLP), radial basis function(RBF), linear network을, unsupervised training으로는 Kohonen network(SOFM)를 사용하였다. 1999년 이후에 보고된 658건의 서열을 이용하여 training : selection : test를 2:1:1로 시행한 후 1999년 이전에 보고된 501건의 서열을 이용하여 neural network의 판별력을 확인하였다. 결과: 아미노산 변이의 개수를 3개 이하와 4개 이상을 기준으로 나누었을 때 인터페론 치료의 반응여부는 82%의 정확도를 보였으며, 변이의 수와 인터페론 치료반응도 간에는 p<0.0001의 유의한 상관성이 관찰되었다. 인터페론 반응에 대한 민감도와 특이도는 각각 42.8%, 94.7% 이었으며, 아미노산 변이의 수가 1-3개 사이인 경우가 49.5%로서 이 경우 위음성율은 22.7%이었다. Neural network중 RBF network이 가장 우수한 성적을 보였으며 이 모델로 예측한 인터페론 반응도의 정확도(correct classificatioin)는 73.8% 이었다. 아미노산 변이가 1-3개인 군에 국한하여 neural network training을 시행한 결과, 이용된 모든 network에서 인터페론 치료 반응여부를 재현성있게 판별할 수 없었다. 결론: HCV NS 5A region의 변이는 통계적으로 인터페론 치료의 반응과 유의한 상관관계가 있으나, 약 반수에서는 변이의 수가 1-3개로서 예측이 어려워 임상적 효용이 적다고 판단되었다. 이 경우 neural network을 이용한 패턴 분석 결과 치료 반응과 관련된 특정 패턴을 찾을 수 없었으며, 따라서 단백 치환의 위치를 고려하더라도 Intermediate group에서는 인터페론 치료의 반응을 예견하는 데에 ISDR의 분석이 도움이 되지 않는 것으로 판단된다.