Pendulum 게임에서 Q-learning과 SARSA 강화학습 알고리즘 성능 비교

이유빈(You-bin Lee),이성준(Sung-Joon Lee),온병원(Byung-Won On) 한국정보기술학회 2021 Proceedings of KIIT Conference Vol.2021 No.6

강화학습은 특정 환경 안에서 에이전트가 앞으로 누적될 보상을 최대화할 수 있도록 일련의 행동으로 정의되는 정책을 찾는 방법이다. 필수 구성 요소는 환경, 상태, 에이전트, 행동, 보상으로 정의된다. 본 논문은 Pendulum 게임 환경에서 Q-learning과 SARSA 2가지의 강화학습 알고리즘을 적용하여 성능 비교 및 분석을 진행한다. 실험 환경은 OpenAI Gym을 이용하여 구축하였다. 실험을 통해 Off-Policy 방식의 Q-learning이 On-Policy 방식의 SARSA 보다 높은 보상 값을 가지며 게임을 안정적으로 유지하는 것을 확인할 수 있었다. Reinforcement learning is a method for finding policies defined by a set of actions so that agents can maximize the rewards to be accumulated in the future in a given environment. Prerequisites are defined as environment, status, agent, action, reward. This paper applies two reinforcement learning algorithms, Q-learning and SARSA, to compare and analyze performance in a Pendulum game environment. The experimental environment was built using OpenAI Gym. Experiments have shown that Q-learning in the Off-Policy has a higher reward value than SARSA in the On-Policy method and keeps the game stable.

살리실산 나트륨이 Guinea Pig 내이 외림프의 Unknown Compoung에 미치는 영향

김상윤,황온유,김혜진,추광철 울산대학교 의과대학 1993 울산의대학술지 Vol.2 No.2

There are some evidences that temporary hearing loss is induced by disturbance of cochlear microcirculation due to the inhibition of prostaglandin synthesis by salicylate, however, the mechanism of salicylate ototoxicity has been still under study. We previously suggested that the increase of unknown compound concentration in perilymph of guinea pig might be important for the mechanism of salicylate ototoxicity. So we tried to compare the time course of unknown compound concentration to salicylate concentration in perilymph. Sodium salicylate(460mg/kg) was injected intraperitoneally to guinea pig. Unknown compound concentration in perilymph was measured before the injection and 1 hour, 3 hours, 5 hours, 7 hours and 24 hours after injection, using HPLC-ECD. The time course of unknown compound concentration reveals maximum level at 5 hours and nearly control level at 24 hours, and this is similar to the change of salicylate concentration in perilymph. These data might be another evidence to our understanding of unknown compound-related salicylate ototoxicity.

Proximate Composition, Mineral, Amino Acid, and Antioxidant Component from Fruiting Body of Pleurotus ostreatus Mutant Cultivated on Agricultural and Industrial By-Product

( On You Choi ),( Young Sook Moon ),( Young Cbeol Chang ) 조선대학교 공학기술연구원 2011 공학기술논문지 Vol.4 No.2

The purpose of this study was to investigate the proximate composition, mineral amino acid, and antioxidant component from fruiting body of Pleurotus ostreatus mutant cultivated based on agricultural and industrial by-product. The concentrations of dry matters, crude ash, crude protein, crude fat, and organic matter in PR media were 91.2, 5.3 23.4 4.8, and 84.6%, respectively. In the ca e of CAPR media, they were 91.4, 5.0, 25.1, 5.0, and 84.9%, respectively. The mineral concentrations followed the order of potasssium> phosphate calcium> sodium> iron> magnesium. The most major components of essential amino acids were lysine, leucine, and valine in PR and CAPR media. On the other hand, in the case of non­essential amino acids, they were aspartic acid and glutamic acid, respectively. The total phenol concentration in PR and CAPR media were 15.5 and 16.7 mg/g, respectively. The total flavonoid concentration in PR and CAPR media were 1.2 and 1.3 mg/g, respectively. In the case of polysaccharide, they were 2.3 and 2.2 mg/g, respectively. Overall, these results indicate that fruiting bodies of P. ostreatus mutant have been good potential as a source of material or additives for food, cosmetics, and drug compositions, in the agricultural and industrial sector.

Changes in upper limb muscle activity during smartphone usage while in stable and unstable positions and during gait

You Lim Kim,Suk Min Lee,Hyun-Soo Lee,Juyeong Song,Si-On Song,Min-Ji Seol,Yu-Mi Jang,Jin-Sik Im,Ji-Woo Im 물리치료재활과학회 2018 Physical therapy rehabilitation science Vol.7 No.3

Objective: The objective of this study was to describe the influence of smartphone usage on the upper limb muscles while in stable and unstable positions and during gait. Design: Cross-sectional study. Methods: The study was conducted with 20 right-handed university students between 20 and 27 years of age. Experiments were carried out on students who have used cell phones for more than a year. In this study, experiments were performed with one-handed and two-handed smartphone operations while on stable ground, and the same parameters were measured during smartphone use on unstable ground and during gait. Subjects were instructed to write a text message in Korean on the smartphone for 3 minutes. This was repeated 3 times, with a rest period of 10 seconds given between each 3-minute period. Electromyography was used to record the muscle activity of the upper trapezius, extensor carpi radialis, extensor pollicis longus, and abductor pollicis (AP) during phone operation. Results: The muscle activity value for the right AP in one-handed and the left-AP in two-handed operations was statistically higher than the other muscles in the stable position and during gait (p<0.0001). Also, the right AP in the one- handed operation condition was statistically higher than the two-handed condition (p<0.0001). Conclusions: The use of smartphones while in a stable position resulted in high muscle activity of the right upper limb AP. However, in an unstable position, there were no significant differences from the other muscle activities. Using smartphone while assuming various positions may lead to musculoskeletal disorders in different places.

A Study on Antioxidant and Nitrite Scavenging Effects of Fruiting Body Extract of Plellrotusostreatus Cultivated on Agricultural and Industrial By-Product

( On You Cboi ),( Young Sook Moon ),( Young Cheol Chang ) 조선대학교 공학기술연구원 2011 공학기술논문지 Vol.4 No.3

The aim of this study was to investigate the antioxidant and nitrite scavenging properties of Pleurotusostreatus fruiting body extracts cultivated on agricultural and industrial by-product. Among various extractants, the hot water extractant yield was the highest, 21.6% in the CAPR media and 20.9% in PR media, respectively. The vitamin B1 concentrations were 0.004 mg/100 g in CAPR media and 0.082mg/100 g in PR media, respectively. In the case of B2, they were 0.013 and 0.0 2 mg/100 g, respectively. The e tract showing DPPH radical scavenging activities were ranked as hot water (65.5-66.7%) ethanol (64.3-63.7%) > methanol (62.2-62.6%) > ethyl acetate (40.6-40.5%) > petroleum ether (20.1-17.5%) > chloroform (15.1-16.3%) > acetone (13.4-15.0%) > hexane (13.0-11.4%), irrespective to culture media. Also, superoxide dismutase activities in hot water extract were 55.5% in CAPR media and 58.8% in PR media, respectively. On the other hand, petroleum ether, chloroform, acetone, and hexane extract ranged from 6.8 to 9.8 % in CAPR media and from 5.9 to 8.3% in PR media, respectively. Among various extracts, the hot water extract had higher nitrite scavenging capability at pH 1.2. These results show that extracts of fruiting body of P.ostreatus proved to have the antioxidant and nitrite scavenging effects.

Changes in upper limb muscle activity during smartphone usage while in stable and unstable positions and during gait

Kim, You Lim,Lee, Suk Min,Lee, Hyun-Soo,Song, Juyeong,Song, Si-On,Seol, Min-Ji,Jang, Yu-Mi,Im, Jin-Sik,Im, Ji-Woo korean Academy of Physical Therapy Rehabilitation 2018 Physical therapy rehabilitation science Vol.7 No.3

Objective: The objective of this study was to describe the influence of smartphone usage on the upper limb muscles while in stable and unstable positions and during gait. Design: Cross-sectional study. Methods: The study was conducted with 20 right-handed university students between 20 and 27 years of age. Experiments were carried out on students who have used cell phones for more than a year. In this study, experiments were performed with one-handed and two-handed smartphone operations while on stable ground, and the same parameters were measured during smartphone use on unstable ground and during gait. Subjects were instructed to write a text message in Korean on the smartphone for 3 minutes. This was repeated 3 times, with a rest period of 10 seconds given between each 3-minute period. Electromyography was used to record the muscle activity of the upper trapezius, extensor carpi radialis, extensor pollicis longus, and abductor pollicis (AP) during phone operation. Results: The muscle activity value for the right AP in one-handed and the left-AP in two-handed operations was statistically higher than the other muscles in the stable position and during gait (p<0.0001). Also, the right AP in the one- handed operation condition was statistically higher than the two-handed condition (p<0.0001). Conclusions: The use of smartphones while in a stable position resulted in high muscle activity of the right upper limb AP. However, in an unstable position, there were no significant differences from the other muscle activities. Using smartphone while assuming various positions may lead to musculoskeletal disorders in different places.

에세이 창의성 분류를 위한 저빈도 서브워드 마스킹 기반 BERT 모델

이유빈(You-Bin Lee),온병원(Byung-Won On) 한국정보기술학회 2022 Proceedings of KIIT Conference Vol.2022 No.12

기존의 지식을 바탕으로 새로운 것을 만들어내는 중요성이 높아지고 있는 4차 산업혁명 시대인 만큼 우리는 창의적인 능력이 기를 필요가 있다. 이때 글쓰기는 창의력을 키우는 좋은 도구이다. 창의성은 명확하게 정의하기가 어렵기 때문에 최근 몇 년 동안 창의성에 대한 수많은 연구는 딥 러닝 모델에 기반을 두고 있다. 특히 BERT(Bi-directional Encoder Representation from Transformers)는 마스킹 할 일부 단어를 무작위로 선택하여 사전 훈련 단계를 수행한다. 본 논문에서는 마스킹과정에서 참신한 단어에 중점을 둔다. 추출된 참신한 단어를 기반으로 마스킹 하게되면 BERT 모델은 창의적 에세이를 잘 분류할 수 있는 단어에 더 집중할 수 있다. 따라서 우리는 낮은 빈도수에 해당하는 subword들을 참신한 단어로 여겨, 참신한 단어 마스킹 기반 BERT 모델을 제안한다. 실험 결과 제안한 모델이 기존 BERT 모델에 비해 최대 5% 향상되었다. In the era of the fourth Industrial Revolution, in which the importance of creating something new based on existing knowledge is increasing, we are required to be creative. At this time, writing is a good tool to develop creativity. Because creativity is very difficult to clearly define, numerous studies on creativity in recent years are based on deep learning models. In particular, Bi-directional Encoder Representation from Transformers (BERT) performs the pre-training step by randomly selecting some words for masking. In this paper, we focuses on novel words for masking. If the extracted novel words are masked, BERT model can focus more on them that can help it classify creative essays well. Therefore, we propose a new novel word masking-based BERT model in which novel words are considered as the corresponding subwords with low frequency. Our experimental results show that our proposed model improved up to 5%, compared to the existing BERT model.

A Study on the Application of SFCL on 22.9 ㎸ Bus Tie for Parallel Operation of Power Main Transformers in a Power Distribution System

온민귀(Min-Gwi On),김명후(Myoung-Hoo Kim),김진석(Jin-Seok Kim),유일경(Il-Kyoung You),임성훈(Sung-Hun Lim),김재철(Jae-Chul Kim) 대한전기학회 2011 전기학회논문지 Vol.60 No.1

This paper analyzed the application of Superconducting Fault Current Limiter (SFCL) on 22.9 [㎸] bus tie in a power distribution system. Commonly, the parallel operations of power main transformers offer a lot of merits. However, when a fault occurs in the parallel operation of power main transformer, the fault currents might exceed the interruption capacity of existing protective devices. To resolve this problem, thus, the SFCL has been studied as the fascinating device. In case that, Particularly, the SFCL could be installed to parallel operation of various power main transformers in power distribution system of the Korea Electric Power Corporation (KEPCO) on 22.9 [㎸] bus tie, the effect of the resistance of SFCL could reduce the increased fault currents and meet the interruption capacity of existing protective devices by them. Therefore, we analyzed the effect of application and proposed the proper impedance of the R-type SFCL on 22.9 [㎸] bus tie in a power distribution system using PSCAD/EMTDC.

A Study on Chitinase Production from Bacillus licheniformis H-89

( On You Cboi ),( Kian Cho ),( Shiru Jia ),( Young Cheol Chang ) 조선대학교 공학기술연구원 2012 공학기술논문지 Vol.5 No.3

This research was to investigate optimal condition for effective chitinase production from Bacillus licheniformis H-89 by changes using working volume, carbon source, nitrogen source, culture temperature, and initial pH of medium at flask for 48 hr. The chitinase production was appropriate when the working volume was 50 mL/500 mL of flask. Among various carbon sources, glucose was best carbon source. Especially, 20 or 25 g/L of glucose was optimum concentration for effective chitinase production. Among various nitrogen sources, yeast extract was also best one for chitinase production. The optimal pH and optimal temperature of chitinase production ranged of 6.0 to 7.0 and 35℃. These results suggest that all these optimal conditions can be applied for effective chitinase production from Bacillus licheniformis H-89.

Enhancing the Kasugamycin Production by Streptomyces kasugaensis using Olive Mill Waste

( On You Choi ),( Byeoog Kwon Ahn ),( Ki An Cho ),( Gwang Yeob Seo ),( In Sook Kang ),( Young Cheol Chang ) 조선대학교 공학기술연구원 2011 공학기술논문지 Vol.4 No.4

The purpose of this study was to produce the kasugamycin by using Streptomyces kasugaensis with olive mill waste containing oil as an energy source. Among the various concentrations of oil, only initial concentration of 20 g/l produced the highest concentration (125.7 mg/l) of kasugamycin. This result indicates that the kasugamycin production was strongly dependent with oil concentrations. Among these fatty acids, only oleic acid (0.1 g/l) produced the highest concentration of kasugamycin 69.7 mg/l, which resulted in 3.4 fold increase in the production than the absence of oleic acid. The corn steep liquor concentration (30 g/l) was found to be the most suitable source for kasugamycin production. Also, fed batch cultures using olive mill waste containing oil and glucose as the sole carbon source were carried out in 51 jar fermentor for seven days. In the case of olive mill waste the maximum kasugamycin concentration was 426.8mg/l, which was about 2.7 fold higher than that of the glucose medium.