A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning

Duk-Sun Shim,Joseph Shim 제어·로봇·시스템학회 2023 International Journal of Control, Automation, and Vol.21 No.11

An optimization algorithm is essential for minimizing loss (or objective) functions in machine learning and deep learning. Optimization algorithms face several challenges, one among which is to determine an appropriate learning rate. Generally, a low learning rate leads to slow convergence whereas a large learning rate causes the loss function to fluctuate around the minimum. As a hyper-parameter, the learning rate is determined in advance before parameter training, which is time-consuming. This paper proposes a modified stochastic gradient descent (mSGD) algorithm that uses a random learning rate. Random numbers are generated for a learning rate at every iteration, and the one that gives the minimum value of the loss function is chosen. The proposed mSGD algorithm can reduce the time required for determining the learning rate. In fact, the k-point mSGD algorithm can be considered as a kind of steepest descent algorithm. In a real experiment using the MNIST dataset of hand-written digits, it is demonstrated that the convergence performance of mSGD algorithm is much better than that of the SGD algorithm and slightly better than that of the AdaGrad and Adam algorithms.

Self-Imitation Learning을 이용한 개선된 Deep Q-Network 알고리즘

선우영민(Yung-Min Sunwoo),이원창(Won-Chang Lee) 한국전기전자학회 2021 전기전자학회논문지 Vol.25 No.4

Self-Imitation Learning은 간단한 비활성 정책 actor-critic 알고리즘으로써 에이전트가 과거의 좋은 경험을 활용하여 최적의 정책을 찾을 수 있도록 해준다. 그리고 actor-critic 구조를 갖는 강화학습 알고리즘에 결합되어 다양한 환경들에서 알고리즘의 상당한 개선을 보여주었다. 하지만 Self-Imitation Learning이 강화학습에 큰 도움을 준다고 하더라도 그 적용 분야는 actor-critic architecture를 가지는 강화학습 알고리즘으로 제한되어 있다. 본 논문에서 Self-Imitation Learning의 알고리즘을 가치 기반 강화학습 알고리즘인 DQN에 적용하는 방법을 제안하고, Self-Imitation Learning이 적용된 DQN 알고리즘의 학습을 다양한 환경에서 진행한다. 아울러 그 결과를 기존의 결과와 비교함으로써 Self-Imitation Leaning이 DQN에도 적용될 수 있으며 DQN의 성능을 개선할 수 있음을 보인다. Self-Imitation Learning is a simple off-policy actor-critic algorithm that makes an agent find an optimal policy by using past good experiences. In case that Self-Imitation Learning is combined with reinforcement learning algorithms that have actor-critic architecture, it shows performance improvement in various game environments. However, its applications are limited to reinforcement learning algorithms that have actor-critic architecture. In this paper, we propose a method of applying Self-Imitation Learning to Deep Q-Network which is a value-based deep reinforcement learning algorithm and train it in various game environments. We also show that Self-Imitation Learning can be applied to Deep Q-Network to improve the performance of Deep Q-Network by comparing the proposed algorithm and ordinary Deep Q-Network training results.

Engaged Learning 수업 모델을 활용한 교양필수 교과목 연구-컴퓨팅적사고와 알고리즘 중심으로

노은희,조문기 한국교양교육학회 2024 교양교육연구 Vol.18 No.4

본 연구는 Engaged Learning 수업 모델을 활용한 S 대학의 교양 필수 ‘컴퓨팅적사고와 알고리즘’수업 사례를 다룬다. 연구 대상은 2023년 1학기 1학년 총 188명이다. 수업은 ‘문제 정의 - 아이디어도출 - 해결 방안 적용⋅확인’ 3단계의 수업 원리를 적용하여 2학점으로 설계하였다. 연구 방법은SPSS를 사용해 t-검증을 하였으며 분반별 분석을 수행하였다. 연구 결과, Engaged Learning 수업 모델을적용한 강의 내용의 수준, 수업 만족, 프로젝트 진행에서 분반별로 유의미한 차이를 보였다. 문예창작,영화예술, 법대 학생들로 구성된 분반이 다른 전공 계열 학생들보다 모두 낮은 만족도를 보였고, 소프트,AI 융합학과 학생들이 Engaged Learning 적용 강의 내용 수준에 대해 가장 만족하였고, 전자공학과IT융합 학생들의 Engaged Learning을 활용한 수업 방식과 Engaged Learning 수업 모델을 적용한 프로젝트 수업에서 만족도가 가장 높았다. Engaged Learning 활용 수업에 있어 학생 참여도는 긍정 84.1%,부정은 0.5%를 보였고, Engaged Learning 수업 모델을 활용한 수업 방식이 강의식 수업보다 수업내용을이해하는데 긍정 73.8%, 부정은 7.50%로 나타났다. 컴퓨팅적사고와 알고리즘 교과목 운영은 EngagedLearning 수업 모델을 적용하여 수업을 운영하는 것이 강의식 수업보다 효과가 있음을 보였다. 향후Engaged Learning 수업 모델은 다른 교과목 수업에서도 활용이 확대될 수 있을 것으로 기대된다. This study deals with a case study of a required Liberal Arts class called “Computational Thinking and Algorithms” atS University using the Engaged Learning class model. The study targets 188 first-year students in the first semester of 2023. The class was designed as a two-credit course by applying the three-stage teaching principle of problem definition - ideageneration - solution application - confirmation. The research method was a t-test using SPSS and quartile analysis. Theresearch results showed significant differences by quartile in the level of class content, class satisfaction, and project progressionapplying the Engaged Learning class model. The quartiles composed of students from the Literary and Creative Arts, FilmArts, and Law departments all showed lower satisfaction than the students from the other majors, while the students fromthe Software and AI integration department were the most satisfied with the level of lecture content applying Engaged Learning. However, the students from the Electronics and IT integration department were the least satisfied with the level of lecturecontent applying Engaged Learning as well as the project progression. The level of student participation in the EngagedLearning-based class was 84.1% positive and 0.5% negative, indicating that the Engaged Learning-based class model wasmore satisfactory to the students than the lecture-based class model. The percentage of affirmative and negative responsesto the Engaged Learning model was 73.8% and 7.50%, respectively, indicating that students were more likely to understandthe course content than in a lecture-style class. We further found that applying the Engaged Learning lesson model to themanagement of the Computational Thinking and Algorithms subject was more effective than lecture-style teaching. It isexpected that the Engaged Learning lesson model will be expanded to be used in other subject classes in the future.

Quasi-oppositional-based Rao algorithms for multi-objective design optimization of selected heat sinks

R. V. Rao,R. B. Pawar 한국CDE학회 2020 Journal of computational design and engineering Vol.7 No.6

In this paper, an endeavor is made to enhance the convergence speed of the recently proposed Rao algorithms. The new upgraded versions of Rao algorithms named as “quasi-oppositional-based Rao algorithms” are proposed in this paper. The quasi-oppositional-based learning is incorporated in the basic Rao algorithms to diversify the searching process of the algorithms. The performance of the proposed algorithms is tested on 51 unconstrained benchmark functions. Also, three multi-objective optimization case studies of different heat sinks such as a single-layered microchannel heat sink (SL-MCHS), a double-layered microchannel heat sink (DL-MCHS), and a plate-fin heat sink (PFHS) are attempted to investigate the effectiveness of the proposed algorithms in solving real-world complex engineering optimization problems. The results obtained using the proposed algorithms are compared with the results obtained using the well-known advanced optimization algorithms such as genetic algorithm (GA), artificial bee colony (ABC), differential evolution (DE), particle swarm optimization (PSO), teaching-learning-based algorithm (TLBO), Jaya algorithm, multi-objective genetic algorithm (MOGA), non-dominated sorting genetic algorithm (NSGA-II), real-coded GA (RCGA), direction-based GA, self-adaptive multi-population (SAMP) Rao algorithms, and basic Rao algorithms. The proposed quasi-oppositional-based Rao algorithms are found superior or competitive to the other optimization algorithms considered.

Squeak Etoys 활용 알고리즘 학습이 중학생의 문제해결력에 미치는 영향

정미연,이영준,이은경 대한공업교육학회 2008 대한공업교육학회지 Vol.33 No.2

Many former researchers demonstrated that algorithm learning has a positive outcome on students' problem-solving abilities. One of the methods for algorithm learning, the ‘programming learning’ method is highly effective. However, there are numerous constraints in schools for programming learning. This study attempts to overcome these issues. Squeak Etoys, one of the educational visual programming languages for easy and interesting learning, has been selected as a learning tool. We developed the algorithm-learning curriculum for middle school students. They were divided into a control group and an experimental group. The students learned on the basis of equal curriculum but, they used other learning tools through over a total 6 sessions. The result showed that Squeak Etoys based Algorithm learning has a positive effect on improving middle school learners' problem solving abilities, self-efficacies and logical thinking abilities. Although the students' logical thinking abilities in the experimental group are improved a lot more than the students' abilities in control group, the students' logical think abilities in the both groups are improved. Therefore, algorithm education in secondary schools are necessary. In conclusion, Squeak Etoys based Algorithm learning has a positive effect on problem solving ability and self efficacy. The developed curriculum can be applicable as a basis for study on algorithm learning and educational programming language. 알고리즘 학습이 학습자의 창의적 문제해결력, 논리적사고력과 같은 고차원적인 인지 능력 향상에 도움이 되지만, 알고리즘 학습을 처음 접하는 어린 학습자들의 경우 인지적 부담으로 작용할 수 있다. 따라서 프로그래밍을 통한 알고리즘 학습에서 학습자의 인지적 부담을 감소시켜주고 알고리즘 학습에 대한 흥미와 동기 유발을 위한 새로운 교수 설계가 필요하다. 본 연구에서는 중학생의 알고리즘 학습을 위한 프로그래밍 도구로 비주얼 기반 교육용 프로그래밍 언어인 Squeak Etoys를 선정하고 이를 활용한 알고리즘 학습 프로그램을 설계하였다. 설계한 학습 프로그램을 실제 중학교 학습자들에게 적용한 결과, Squeak Etoys 활용 알고리즘 학습이 프로그래밍을 처음 접하는 초보 학습자들의 문제해결성향과 자기효능감, 논리적사고력 향상에 유의미한 영향을 주었음을 확인하였다. 특히 논리적사고력의 경우, 실험집단과 통제집단모두 유의미한 향상을 보였다. 이는 알고리즘을 설계하고 표현하는 과정에 대한 학습경험 자체가 논리적사고력 신장에 효과적임을 의미한다. 따라서 초·중등교육에서의 알고리즘 교육은 의미 있는 학습내용이라고 볼 수 있다. 그러나 Squeak Etoys 활용 알고리즘 학습을 진행한 집단이 통제집단에 비해 논리적사고력 향상의 정도가 유의미하게 높은 것은 Squeak Etoys와 같은 비주얼 기반 프로그래밍 학습 환경이 알고리즘 학습에 대한 외생적 인지부하를 감소시켜주고 문제해결성향 및 자기효능감에 긍정적인 영향을 줌으로써 논리적사고력과 같은 고등정신능력 향상에 도움을 준 것으로 해석할 수 있다. 이러한 연구 결과는 향후 새로운 교육과정 시행될 경우, 알고리즘 학습을 위한 교육용 프로그래밍 언어의 선정 및 교수 학습 설계에 유용한 지침을 제공해 줄 수 있을 것이다.

인공지능 프로그램의 특허적격성에 관한 검토 - 머신러닝을 중심으로 -

황의철 서울대학교 기술과법센터 2020 Law & technology Vol.16 No.5

인공지능 및 머신러닝도 소프트웨어의 일종이므로 미국의 소프트웨어 발명의 특허적격 관련 판결들을 참고할만하다. Alice 판결 이후 미국 CAFC는 컴퓨터 관련 발명에 대한 특허 적격성을 부정하는 다수의 판결들을 내렸으나, 반대로 특허 적격성을 긍정한 판결들도 있다. 머신러닝의 특수성을 감안하여 특허적격을 검토하기 위해서, 학습 데이터, 학습 알고리즘 및 학습완료 모델로 나누어 살펴본다. 먼저 학습데이터 관련하여, 데이터 그 자체만으로는 특허적격을 인정받기 어렵지만 데이터 간의 구조적 상호관계가 정의되어 이로 인해 컴퓨터가 특정한 기능을 수행할 수 있다면 이는 컴퓨터 프로그램에 준하여 볼 수 있다. 학습 알고리즘에 관하여서는, 최근 미국의 하급심에서 그 발명이 특정 분야나 응용과 관계가 없고 단지 기계를 통한 데이터의 실행일 뿐이어서 일반적인 추상적 개념이라는 이유로 특허적격을 부정한 사례가 있다. 또한 학습완료 모델은 학습 알고리즘에 기반하여 구체적인 학습 데이터를 이용해 학습이 완료되어 파라미터가 조정된 결과이므로, 학습 알고리즘과 유사하게 특허적격성 문제를 판단 할 수 있다고 보인다. 다만, 학습이 완료된 경우는 특정한 기술분야에 인공지능을 구체적으로 접목시킨 경우가 많을 것으로 예상된다. 우리나라의 경우 아직 인공지능 발명의 특허적격이 문제된 사례가 없는 것으로 보이고, 특허청의 심사기준에도 인공지능 발명의 성립성에 구체적인 기준이 없다. 특허청의 심사기준을 구체적인 사례와 함께 명확히 하는 것이 필요해 보인다. Since artificial intelligence or machine learning program is also a type of software, cases regarding the software patent eligibility in the USA should be considered. After the Alice decision, the CAFC often denied the patent eligibility for computer-related inventions. However, there were also cases in which the patent eligibility for computer-related inventions is admitted. Data for learning, learning algorithm and learning-completed model will be discussed for the patent eligibility of the machine learning, respectively. With regard to the data for learning, the simple data itself cannot be patent-eligible generally. However, if the structural relationship between the data elements is defined so that specific functions can be performed by the computer, such data for learning can be regarded as equivalent to a computer program. As for the learning algorithms an U.S. district court decided that the invention is not patent-eligible since it is not based on a particular technical field or application, but merely the execution of data through a computer. In addition, since the learning-completed model is the result of the learning using specific learning data based on learning algorithms, it seems that the patent eligibility can be determined similarly to the learning algorithm. However, there may be tendency that the AI programs will be applied specifically to certain technology fields in the learning-completed models. In Korea, there seems to be no case for the patent eligibility of artificial intelligence inventions yet, and there is no specific standard for the patent eligibility of AI inventions in the examination guideline.

다층신경망 학습 알고리즘 변화에 따른 건물 냉방부하 예측 모델의 성능 비교 평가

성남철,홍구표 한국생태환경건축학회 2022 한국생태환경건축학회 논문집 Vol.22 No.4

Purpose: In this study, among the methods of applying machine learning when predicting the load of a building, the cooling load of a building was predicted using a neural network model. To investigate the appropriateness of the learning algorithm of the multi-layer neural network model, the main purpose is to compare the predictive performance according to the change in the learning algorithm. Method: Among the learning algorithms applicable to multilayer neural networks, a total of 16 algorithms were used to predict the cooling load and compare the prediction results. The input variables of the input layer of the neural network model are outdoor dry bulb temperature, outdoor humidity, and Seasonally Data. The training period is 70% and the test period was 30%. The number of layers in the hidden layer is 3, the number of neurons is 20, and Epochs is 100. CvRMSE and MBE are used as performance index of the prediction model. The maximum, minimum, average, and standard deviation of the 20 prediction results are calculated, and the prediction performance according to the change in the learning algorithm was compared. Result: As a result of analyzing the predictive performance for each learning algorithm, the predictive performance according to the learning algorithm was different. Considering the results and deviations of the two indicators of predictive performance comprehensively, the model using the Levenberg-Marquardt (LM) learning algorithm is judged to have the best predictive performance.

Fault Detection Performance of Machine Learning Algorithms for 3D Printing in Smart Manufacturing

Rita Rijayanti(리타 리자얀티),Kyohong Jin(진교홍),Mintae Hwang(황민태) 한국정보통신학회 2020 한국정보통신학회 여성 ICT 학술대회 논문집 Vol.2020 No.8

본 논문에서는 스마트 제조의 주요 요소 기술인 3D 프린팅을 위해 적용 가능한 기계학습 알고리즘들을 조사 분석하고서, 어떠한 알고리즘이 고장 감지 성능 면에서 우수한지를 객관적인 자료를 통해 비교 분석한 결과를 제시한다. 본 연구는 FDM(Fused Deposition Modeling)을 위한 3D 프린팅 공정 사례에 초점을 맞추고 있으며, 비교 대상 기계학습 알고리즘으로는 계층 구조, 신경망 그리고 K-means 알고리즘을 고려하였다. 인쇄성, 임계치 복잡성 감소, 조립 전 가속, 결함 검출 등의 성능 비교 파라미터를 적용해 상기 기계학습 알고리즘들을 비교해 본 결과 K-means 알고리즘이 88%의 정확도를 보여주고 있어 타 알고리즘에 비해 고장 감지 성능이 우수한 것으로 조사되었다. 이 결과는 다양한 스마트 제조 애플리케이션에 기계학습 알고리즘을 적용하기 위한 참고 자료로 활용될 수 있을 것이라 기대된다. In this paper, we investigate and analyze applicable machine learning algorithms for 3D printing, a key technology of smart manufacturing and select a superior algorithm for failure detection using comparative analysis. This study focuses on the case of the 3D printing process for Fused Deposition Modeling (FDM). It considers the hierarchical structure, neural network, and K-means algorithm as the machine learning algorithm to be compared. As a result of comparing the machine learning algorithms by applying performance comparison parameters such as printability, reduction in threshold complexity, acceleration before assembly, and defect detection, the K-means algorithm shows 88% accuracy, the failure detection performance is higher than other algorithms. This result is expected to be used as a reference for applying machine learning algorithms to various smart manufacturing applications.

Courses Recommendation Algorithm Based On Performance Prediction In E-Learning

Koffi, Dagou Dangui Augustin Sylvain Legrand,Ouattara, Nouho,Mambe, Digrais Moise,Oumtanaga, Souleymane,ADJE, Assohoun International Journal of Computer ScienceNetwork S 2021 International journal of computer science and netw Vol.21 No.2

The effectiveness of recommendation systems depends on the performance of the algorithms with which these systems are designed. The quality of the algorithms themselves depends on the quality of the strategies with which they were designed. These strategies differ from author to author. Thus, designing a good recommendation system means implementing the good strategies. It's in this context that several research works have been proposed on various strategies applied to algorithms to meet the needs of recommendations. Researchers are trying indefinitely to address this objective of seeking the qualities of recommendation algorithms. In this paper, we propose a new algorithm for recommending learning items. Learner performance predictions and collaborative recommendation methods are used as strategies for this algorithm. The proposed performance prediction model is based on convolutional neural networks (CNN). The results of the performance predictions are used by the proposed recommendation algorithm. The results of the predictions obtained show the efficiency of Deep Learning compared to the k-nearest neighbor (k-NN) algorithm. The proposed recommendation algorithm improves the recommendations of the learners' learning items. This algorithm also has the particularity of dissuading learning items in the learner's profile that are deemed inadequate for his or her training.

자가학습 가격 책정 알고리즘과 암묵적 담합에 관한 시뮬레이션 분석

양훈식,신위뢰 한국산업조직학회 2019 産業組織硏究 Vol.27 No.3

We study collusive outcomes when firms employ self-learning pricing algorithms. Two firms are in a repeated prisoner’s dilemma environment, and they choose their action using Q-learning algorithm. Simulations show that they can reach a significant degree of collusive behavior without explicit agreements. We document how the anti-competitive behavior varies with parameter values of Q-learning algorithm, and discuss the role of parameters on the collusive outcomes. We also consider asymmetric cases where one firm acts differently: one uses Tit-for-tat strategy, a different parameter value for Q-learning algorithm, and different memory capacity. 본 연구는 자가학습(self-learning) 알고리즘을 이용한 가격 책정이 기업 간의 암묵적 담합을 발생시킬 수 있음을 보인다. 즉, 한 시장에서 경쟁하는 두 기업이 각자의 이윤을 극대화하기 위해 알고리즘을 이용하여 가격을 설정할 때, 사전적인 합의 없이도 다기간의 가격 책정을 통한 상호 작용과 학습을 통해 경쟁을 줄이고 일정 수준의 담합을 달성할 수 있음을 시뮬레이션 분석을 통해 살펴본다. 이를 위해 본 연구에서는 반복적인 죄수의 딜레마(prisoner’s dilemma) 게임을 응용한 환경 하에서, Q-learning 알고리즘을 이용하여 매 기 가격을 선택하는 두 기업을 설정하고, 일정 기간이 지난 후 두 기업 사이에 높은 수준의 담합이 지속적으로 나타남을 확인한다. 이는 두 기업이 대칭적인 경우뿐만 아니라 비대칭 적인 경우, 즉 두기업이 사용하는 알고리즘의 학습 속도나 기억 능력에 차이가 있거나, 하나의 기업이 다른기업의 선택을 모방하는 팃포탯(Tit for Tat) 전략을 사용하는 환경에서도 나타난다. 이러한결과는 담합과 관련한 현재의 규제가 알고리즘을 이용한 새로운 유형의 암묵적 담합에는 유효하지 않을 가능성을 제시한다.