딥러닝 프레임워크의 비교

정여진(Yeojin Chung),안성만(SungMahn Ahn),양지헌(Jiheon Yang),이재준(Jaejoon Lee) 한국지능정보시스템학회 2017 지능정보연구 Vol.23 No.2

The deep learning framework is software designed to help develop deep learning models. Some of its important functions include “automatic differentiation” and “utilization of GPU”. The list of popular deep learning framework includes Caffe (BVLC) and Theano (University of Montreal). And recently, Microsofts deep learning framework, Microsoft Cognitive Toolkit, was released as open-source license, following Google’s Tensorflow a year earlier. The early deep learning frameworks have been developed mainly for research at universities. Beginning with the inception of Tensorflow, however, it seems that companies such as Microsoft and Facebook have started to join the competition of framework development. Given the trend, Google and other companies are expected to continue investing in the deep learning framework to bring forward the initiative in the artificial intelligence business. From this point of view, we think it is a good time to compare some of deep learning frameworks. So we compare three deep learning frameworks which can be used as a Python library. Those are Googles Tensorflow, Microsoft’s CNTK, and Theano which is sort of a predecessor of the preceding two. The most common and important function of deep learning frameworks is the ability to perform automatic differentiation. Basically all the mathematical expressions of deep learning models can be represented as computational graphs, which consist of nodes and edges. Partial derivatives on each edge of a computational graph can then be obtained. With the partial derivatives, we can let software compute differentiation of any node with respect to any variable by utilizing chain rule of Calculus. First of all, the convenience of coding is in the order of CNTK, Tensorflow, and Theano. The criterion is simply based on the lengths of the codes and the learning curve and the ease of coding are not the main concern. According to the criteria, Theano was the most difficult to implement with, and CNTK and Tensorflow were somewhat easier. With Tensorflow, we need to define weight variables and biases explicitly. The reason that CNTK and Tensorflow are easier to implement with is that those frameworks provide us with more abstraction than Theano. We, however, need to mention that low-level coding is not always bad. It gives us flexibility of coding. With the low-level coding such as in Theano, we can implement and test any new deep learning models or any new search methods that we can think of. The assessment of the execution speed of each framework is that there is not meaningful difference. According to the experiment, execution speeds of Theano and Tensorflow are very similar, although the experiment was limited to a CNN model. In the case of CNTK, the experimental environment was not maintained as the same. The code written in CNTK has to be run in PC environment without GPU where codes execute as much as 50 times slower than with GPU. But we concluded that the difference of execution speed was within the range of variation caused by the different hardware setup. In this study, we compared three types of deep learning framework: Theano, Tensorflow, and CNTK. According to Wikipedia, there are 12 available deep learning frameworks. And 15 different attributes differentiate each framework. Some of the important attributes would include interface language (Python, C ++, Java, etc.) and the availability of libraries on various deep learning models such as CNN, RNN, DBN, and etc. And if a user implements a large scale deep learning model, it will also be important to support multiple GPU or multiple servers. Also, if you are learning the deep learning model, it would also be important if there are enough examples and references.

준지도학습 방식 속성 기반 감성분석을 이용한 다중 기준 추천시스템

박동준(Park, Dong Jun),정여진(Chung, YeoJin) 한국지능정보시스템학회 2022 한국지능정보시스템학회 학술대회논문집 Vol.2022 No.6

보건의료 빅데이터에서의 자연어처리기법 적용방안 연구: 단어임베딩 방법을 중심으로

김한상 ( Hansang Kim ),정여진 ( Yeojin Chung ) 한국보건행정학회 2020 보건행정학회지 Vol.30 No.1

While healthcare data sets include extensive information about patients, many researchers have limitations in analyzing them due to their intrinsic characteristics such as heterogeneity, longitudinal irregularity, and noise. In particular, since the majority of medical history information is recorded in text codes, the use of such information has been limited due to the high dimensionality of explanatory variables. To address this problem, recent studies applied word embedding techniques, originally developed for natural language processing, and derived positive results in terms of dimensional reduction and accuracy of the prediction model. This paper reviews the deep learning-based natural language processing techniques (word embedding) and summarizes research cases that have used those techniques in the health care field. Then we finally propose a research framework for applying deep learning-based natural language process in the analysis of domestic health insurance data.

개별 기업에 대한 인터넷 검색량과 주가변동성의 관계

전새미(Saemi Jeon),정여진(Yeojin Chung),이동엽(Dongyoup Lee) 한국지능정보시스템학회 2016 지능정보연구 Vol.22 No.2

최근 인터넷의 보편화와 정보통신 기술의 발달로 인해 인터넷을 통한 정보검색이 일상화 됨에 따라 주식에 관한 정보역시 검색엔진, 소셜네트워크서비스, 인터넷 커뮤니티 등을 통해 획득하는 경우가 잦아졌다. 특정 단어에 대한 키워드 검색량은 사용자의 관심도를 반영하기 때문에 다양한 연구에서 개별 기업에 대한 인터넷 검색량은 투자자의 관심도에 대한 척도로서의 사용가능성을 각광받았다. 특정 주식에 대한 투자자의 관심이 증가할 때 일시적으로 주가가 상승하였다가 회복하는 반전현상은 여러 연구를 통해 검증되어 왔지만 그 동안 투자자의 관심도는 주로 주식거래량, 광고 비용 등을 사용해 간접적으로 측정되었다. 본 연구에서는 국내 코스닥 시장에 상장된 기업에 대한 인터넷 검색량을 투자자의 관심의 척도로 사용하여 투자자의 관심에 근거한 주가변동성의 변화를 전체 시장 측면과 산업별 측면에서 관찰한다. 또한 투자자 관심이 야기한 가격압박에 의한 주가 반전현상의 존재를 코스닥 시장에서 검증하고 산업 간의 반전정도의 차이를 비교한다. 실증분석 결과 비정상적인 인터넷 검색량 증가는 주가변동성의 유의적인 증가를 가져왔고 이러한 현상은 IT S/W, 건설, 유통 산업군에서 특히 강하게 나타났다. 비정상적인 인터넷 검색량의 증가 이후 2주 간 주가변동성이 증가하였고 3~4주 후에는 오히려 변동성이 감소하는 것을 확인하였다. 이러한 주가 반전현상 역시 IT S/W, 건설, 유통 산업군에서 보다 극단적으로 발생하는 것으로 나타난다.

커널 밀도 추정치를 이용한 유한혼합모형의 초기화 방법과 모형기반 군집분석에의 응용

조현주(Hyun-ju Cho),정여진(Yeojin Chung),김영민(Youngmin Kim) 한국데이터정보과학회 2018 한국데이터정보과학회지 Vol.29 No.2

유한 혼합 모형은 확률 모형에 기반한 군집분석을 위해 다양하게 활용되고 있다. EM 알고리즘은 유한혼합모형과 같이 숨겨진 구조의 모형의 최대우도추정량을 계산하는데 널리 쓰인다. 하지만 EM 알고리즘은 초기 값에 따라 그 성능이 좌우되어 우도함수의 최대값이 아닌 고정점으로 수렴하거나 수렴 속도가 느려질 가능성이 있다. 본 연구에서는 커널 밀도 추정치의 정점의 위치를 사용하여 유한 혼합모형의 모수 추정을 위한 EM 알고리즘을 초기화 하는 방법을 제안한다. 기존의 무작위 초기화에 기반한 방법들과 비교하여 모의실험을 통해 본 연구에서 제안하는 초기화 방법이 우도함수의 최대값을 찾는데 더 우월함을 확인한다. 또한 추정된 유한확률모형을 기반으로 한 군집분석을 기업부도 데이터에 적용하여 보다 나은 군집 결과를 가져옴을 보여준다. The finite mixture model is widely used for model-based cluster analysis. The EM algorithm finds the maximum likelihood estimates of the finite mixture model. Since the performance of the EM algorithm is largely influenced by its initial value, the choice of the initial value has been regarded as an important factor for EM. This study proposes a new initialization method for the EM algorithm using the kernel density estimator. The location of modes of the kernel density estimate is calculated by the MEM algorithm and set as an initial value for component means for Gaussian mixture model. Simulation study and application on corporate default data show that the proposed method gives parameter estimates higher than the exising methods. In addition, we apply the model-based clustering based on the estimated mixture model and compare the performance of clustering.

한국어 음소 단위 LSTM 언어모델을 이용한 문장 생성

안성만(SungMahn Ahn),정여진(Yeojin Chung),이재준(Jaejoon Lee),양지헌(Jiheon Yang) 한국지능정보시스템학회 2017 지능정보연구 Vol.23 No.2

Language models were originally developed for speech recognition and language processing. Using a set of example sentences, a language model predicts the next word or character based on sequential input data. N-gram models have been widely used but this model cannot model the correlation between the input units efficiently since it is a probabilistic model which are based on the frequency of each unit in the training set. Recently, as the deep learning algorithm has been developed, a recurrent neural network (RNN) model and a long short-term memory (LSTM) model have been widely used for the neural language model (Ahn, 2016; Kim et al., 2016; Lee et al., 2016). These models can reflect dependency between the objects that are entered sequentially into the model (Gers and Schmidhuber, 2001; Mikolov et al., 2010; Sundermeyer et al., 2012). In order to learning the neural language model, texts need to be decomposed into words or morphemes. Since, however, a training set of sentences includes a huge number of words or morphemes in general, the size of dictionary is very large and so it increases model complexity. In addition, word-level or morpheme-level models are able to generate vocabularies only which are contained in the training set. Furthermore, with highly morphological languages such as Turkish, Hungarian, Russian, Finnish or Korean, morpheme analyzers have more chance to cause errors in decomposition process (Lankinen et al., 2016). Therefore, this paper proposes a phoneme-level language model for Korean language based on LSTM models. A phoneme such as a vowel or a consonant is the smallest unit that comprises Korean texts. We construct the language model using three or four LSTM layers. Each model was trained using Stochastic Gradient Algorithm and more advanced optimization algorithms such as Adagrad, RMSprop, Adadelta, Adam, Adamax, and Nadam. Simulation study was done with Old Testament texts using a deep learning package Keras based the Theano. After pre-processing the texts, the dataset included 74 of unique characters including vowels, consonants, and punctuation marks. Then we constructed an input vector with 20 consecutive characters and an output with a following 21st character. Finally, total 1,023,411 sets of input-output vectors were included in the dataset and we divided them into training, validation, testsets with proportion 70:15:15. All the simulation were conducted on a system equipped with an Intel Xeon CPU (16 cores) and a NVIDIA GeForce GTX 1080 GPU. We compared the loss function evaluated for the validation set, the perplexity evaluated for the test set, and the time to be taken for training each model. As a result, all the optimization algorithms but the stochastic gradient algorithm showed similar validation loss and perplexity, which are clearly superior to those of the stochastic gradient algorithm. The stochastic gradient algorithm took the longest time to be trained for both 3- and 4-LSTM models. On average, the 4-LSTM layer model took 69% longer training time than the 3-LSTM layer model. However, the validation loss and perplexity were not improved significantly or became even worse for specific conditions. On the other hand, when comparing the automatically generated sentences, the 4-LSTM layer model tended to generate the sentences which are closer to the natural language than the 3-LSTM model. Although there were slight differences in the completeness of the generated sentences between the models, the sentence generation performance was quite satisfactory in any simulation conditions: they generated only legitimate Korean letters and the use of postposition and the conjugation of verbs were almost perfect in the sense of grammar. The results of this study are expected to be widely used for the processing of Korean language in the field of language processing and speech recognition, which are the basis of artificial intelligence systems.

Airbnb 숙소 유형에 따른 호스트의 자기소개 텍스트가 공유성과에 미치는 영향

심지환 ( Sim Ji Hwan ),김소영 ( Kim So Young ),정여진 ( Chung Yeojin ) 한국지식경영학회 2020 지식경영연구 Vol.21 No.4

최근 빠르게 성장하고 있는 숙박 공유경제 시장에서 품질에 대한 불확실성은 사용자의 만족도에 영향을 미치는 위험 요소지만, 이는 시설 제공자가 공개하는 정보를 통해 완화될 수 있다. 그 중 시설 제공자의 본인에 대한 자기소개는 사용자와의 정서적 교류를 통해 심리적 거리를 제거함으로써 공유 성과에 긍정적 영향을 미친다. 본 연구는 대표적인 숙박 공유경제 플랫폼인 Airbnb에서 호스트의 자기소개가 포함하는 정보의 종류에 따라 공유성과에 미치는 영향을 분석하고, Airbnb의 숙소 유형에 따라 차이를 분석하였다. 이를 위해 호스트가 공개하는 자기소개 텍스트를 문장별로 분리하고 비지도 학습기반의 딥러닝 방법인 Attention-Based Aspect Extraction 방법을 활용하여 각 문장이 포함하는 의미를 추출하였다. 추출된 의미를 토대로 자기소개 텍스트가 포함하는 의미가 공유성과에 미치는 영향과 숙소 유형에 따른 교호작용 효과를 분석하였다. 연구결과, 숙소 유형별로 호스트의 특정 성향이 공유성과에 긍정적인 영향을 미치는 것을 확인하였고, 이를 통해 숙소 유형에 따라 공유성과를 극대화하기 위한 마케팅 전략에 대한 실증적인 함의를 제공한다. In accommodation sharing economy, customers take a risk of uncertainty about product quality, which is an important factor affecting users' satisfaction. This risk can be lowered by the information disclosed by the facility provider. Self-presentation of the hosts can make a positive effect on listing performance by eliminating psychological distance through emotional interaction with users. This paper analyzed the self-presentation text provided by Airbnb hosts and found key aspects in the text. In order to extract the aspects from the text, host descriptions were separated into sentences and applied the Attention-Based Aspect Extraction method, an unsupervised neural attention model. Then, we investigated the relationship between aspects in the host description and the listing performance via linear regression models. In order to compare their impact between the three facility types(Entire home/apt, Private rooms, and Shared rooms), the interaction effects between the facility types and the aspect summaries were included in the model. We found that specific aspects had positive effects on the performance for each facility type, and provided implication on the marketing strategy to maximize the performance of the shared economy.