Visual Analysis of Deep Q-network

( Dewen Seng ),( Jiaming Zhang ),( Xiaoying Shi ) 한국인터넷정보학회 2021 KSII Transactions on Internet and Information Syst Vol.15 No.3

In recent years, deep reinforcement learning (DRL) models are enjoying great interest as their success in a variety of challenging tasks. Deep Q-Network (DQN) is a widely used deep reinforcement learning model, which trains an intelligent agent that executes optimal actions while interacting with an environment. This model is well known for its ability to surpass skilled human players across many Atari 2600 games. Although DQN has achieved excellent performance in practice, there lacks a clear understanding of why the model works. In this paper, we present a visual analytics system for understanding deep Q-network in a non-blind matter. Based on the stored data generated from the training and testing process, four coordinated views are designed to expose the internal execution mechanism of DQN from different perspectives. We report the system performance and demonstrate its effectiveness through two case studies. By using our system, users can learn the relationship between states and Q-values, the function of convolutional layers, the strategies learned by DQN and the rationality of decisions made by the agent.

Seismic Performance of Column Supporting Single-Layer Reticulated Domes with Friction Pendulum Bearings

Dewen Kong,Lingling Wang,Feng Fan,Xudong Zhi 한국강구조학회 2017 International Journal of Steel Structures Vol.17 No.2

The friction pendulum bearing (FPB) has been widely studied as an effective dry friction sliding isolation device, due to its self-limit and self-reset capability. The refinement finite element models of FPB were applied to column supporting single-layer reticulated domes. The seismic performances of these structures with FPBs were systematically analyzed by finite element software LS-DYNA. Numerical results illustrate that the optimal friction coefficient of FPB increases with increasing earthquake intensity and the optimal range of friction coefficient locates between 0.025 and 0.15. The seismic effects of singlelayer reticulated domes with FPBs are strengthened with the increase of curvature radius, while isolation effect of FPBs has no obvious change as the curvature radius exceeds 1.5 m. Additionally, the parameter selection principles of friction pendulum bearings for column supporting single-layer reticulated domes are given by means of investigating the force of the slider of FPBs and dynamic analysis of single-layer reticulated domes with FPBs.

Infl uence of Column Supports on Seismic Performance of K8 Single-Layer Spherical Reticulated Domes with Friction Pendulum Bearings

Dewen Kong,Lingling Wang,Liao Wu,Yuxia Zhang 한국강구조학회 2019 International Journal of Steel Structures Vol.19 No.3

The friction pendulum bearing (FPB) has been proved to be good isolation equipment, and the friction pendulum bearings were applied to K8 single-layer reticulated domes which span was 80 m. By using the vibration reduction analysis method based on the refi ne element models of FPBs, the seismic performance of the single-layer spherical reticulated domes with FPBs was studied and the infl uence of the column height and radius of section was discussed on the seismic performance of structures under the horizontal earthquakes. The results indicate that, with the increasing of height and radius of section of the supporting column, the vibration reduction eff ect of column supporting K8 single-layer reticulated domes with FPBs is enhanced fi rst and then weakened. Under the horizontal earthquakes, the resonance phenomenon of K8 single-layer reticulated domes with column supports could be eff ectively avoided by the use of FPBs. For the cylindrical column supporting K8 single-layer reticulated domes with FPBs, compared with the corresponding hinge support structure, the vibration reduction eff ect of column support structure with FPBs which column height is 8 m is better when the column section radius is 0.50 m. However, the optimal column section radii are between 0.60 and 0.70 m when the column height is 10 m.

Seismic Performance of Single-Layer Lattice Shells with VF-FPB

Dewen Kong,Feng Fan,Xudong Zhi 한국강구조학회 2014 International Journal of Steel Structures Vol.14 No.4

Friction pendulum bearings (FPB) are recognized as a kind of effective isolation device for protecting and retrofitting anexisting structure, but they are not adaptive to different seismic inputs due to the constant friction coefficient and constant radiusof curvature. A new isolation bearing called variable-friction friction pendulum bearings (VF-FPB) was proposed which couldovercome this limitation while retaining all the advantages of an FPB. Unlike a conventional FPB, the sliding surface of a VFFPBwas divided into several parts with different surface materials which allowed the coefficient of friction to change whenthe slider went across different parts. This means that the stiffness of a VF-FPB changed as the relative displacement betweenthe slider and the sliding surface changes. In the current paper, the isolation mechanism of a typical VF-FPB was analyzed. The numerical model of a single-layer lattice shell was developed and the effect of VF-FPB installation on the structure wasstudied. The VF-FPBs used in the study had their surfaces divided into two parts, and it was found that an ‘optimal range’ existsfor the dimensions of two component parts for the best earthquake isolation effect. It was concluded that VF-FPBs couldimprove the seismic performance of a single layer lattice shell more effectively than conventional FPBs.

Study on Poverty Risk and Risk Avoidance of People Affected by Housing Relocation due to Urban Construction Projects in China Western Cities

He Dewen,Li Jingni,Song Fujian 연세대학교 빈곤문제국제개발연구원 2012 Journal of Poverty Alleviation and International D Vol.3 No.2

With accelerating urbanization in western provinces and cities of China, the number of urban construction projects, such as the old urban area reconstruction project, the development zone construction project, and the infrastructure construction project are on the rise. The project scale has been expanding and the amount of housing relocation involved in construction projects is growing year by year. As a result, more and more people are affected in this process. These people not only include the relocated people as specified by the Regulation on the Dismantlement of Urban Houses, namely, the owner of relocated houses, but also the actual users of the relocated housed who may be affected temporarily or permanently and the people influenced by relocation of houses for business purposes. To properly resettle the affected people in urban relocation projects, maintaining their right for survival and development, providing sustainable development opportunities for them, and eliminating or lowering their poverty risks is helpful for reducing social contradictions and conflicts and avoiding possible social risks. Large-scale construction projects have emerged in western provinces and cities with the implementation of the strategy of Development of Western Regions. Against this background, social risks caused by social contradictions due to relocation activities have occurred from time to time. Low-quality relocation and resettlement have become unstable factors and have stood in the way of harmonious development. Poverty risk of the affected people, as the fundamental and one of the most obvious social risks, relates to their right and interest for survival and development. To study the poverty risk and risk avoidance of people affected by housing relocation due to urban construction projects is helpful for not only maintaining social stability, but also achieving the“People Oriented” goal of urban construction projects. From the perspective of causal relation among the affecting factors for poverty risk of people affected by housing relocation due to urban construction projects, this paper adopts the method of a cause-and-effect tree diagram analysis to study the influence and combined action of four major affecting factors, namely, limitation of resettlement compensation policy objective, production difficulty of affected people, living difficulty of affected people and social marginalization of affected people, as well as the generation of poverty risk. It is found in the study that the limitation of resettlement compensation policy (with housing resettlement as the main objective) objective is the leading factor causing poverty risk. Social policy guidelines shall clearly specify that the government resettlement authorities shall assume main responsibilities in risk avoidance and play a leading role in solving production and living difficulties of affected people and improving the living environment of the vulnerable resettled groups.

Multiple Constrained Dynamic Path Optimization based on Improved Ant Colony Algorithm

Seng Dewen,Tang Meixia,Wu Hao,Fang Xujian,Xu Haitao 보안공학연구지원센터 2014 International Journal of u- and e- Service, Scienc Vol.7 No.6

Vehicle navigation system can effectively alleviate traffic congestion, reduce pollution, and reduce travel cost and other problems. As is known to all, the traditional ones are all just static path planning with problems of not only weak effectiveness but also lack of standard optimal path options. They usually provide only one path which represents the shortest time or shortest distance, and ignore the actual demands of the dirivers. Based on traffic data of the past, the upcoming traffic flows can be estimated. With the help of the improved ant colony algorithm, the dynamic optimal path planning results will meet the need of the travelers according with multiple actual constraints.

Process Backtracking and Reconstruction based on Task Chain Model

Jing Chen,Dewen Seng,Xujian Fang 보안공학연구지원센터 2016 International Journal of Grid and Distributed Comp Vol.9 No.4

Fundamental information of the design unit is described using the correlation between the nodes in the task chain. Data transfer between design units is formulated as fundamental data transfer, design rule transfer and path scheme transfer, respectively. The design process is stored with the node as the unit by using the algorithm for decomposing correlated nodes. The reconstruction method is employed to eliminate the redundant nodes that exist in various previous design processes, alleviating the degree of coupling. The performance of the proposed scheme is verified by applying it to the development of the low-voltage appliance.

Anti-seismic Effect of Lattice Grid Structure with Friction Pendulum Bearings Under the Earthquake Impact of Various Dimensions

Feng Fan,Dewen Kong,Menghan Sun,Xudong Zhi 한국강구조학회 2014 International Journal of Steel Structures Vol.14 No.4

As a typical base isolation device, the friction pendulum bearing (FPB) was applied to the lattice grid structures in this paper. The isolation mechanism of FPBs was analyzed from two aspects of force and energy consumption, while the static anddynamic mechanical properties of lattice grid structures with FPBs were numerically investigated using two models of regularand oblique quadrangular pyramid framed structures. The physical models of FPBs with fictional coefficient of 0.1 and acurvature radius of 1.0 m was established and applied to the lattice grid structures. Moreover, the mechanical properties of thesestructures with FPBs ware studied. Numerical results indicate that the seismic response of structures is remarkably weakenedby using FPBs, which means FPBs can be used to effectively control structural vibration.

Coupling Phenomenon Observed with NIRS in Bimanual Operation

Yaodong Pan,Dewen Hu,Katsuhisa Furuta,Takayoshi Komine 한국과학기술원 인간친화 복지 로봇 시스템 연구센터 2007 International Journal of Assistive Robotics and Me Vol.8 No.1

  In this paper, the activation of left and right motor cortex areas during bimanual and unimanual operations is investigated by using a near-infrared spectroscopy (NIRS), EGT-4000 Optical Topography System made by Hitachi Medical Corporation, Japan. It is found that only left or right side of the motor cortex area is active when a subject"s contralateral hand (or fingers of contralateral hand or contralateral arm) is moving in a slow or normal speed (i.e. a speed which is slower than or the same to the finger/hand/ arm moving speed in his/her dairy life), which coincides with previous research results found in literature. The most important discovery in this investigation is that the other side of the motor cortex area, i.e. the ipsilateral motor cortex area which is inactive for the low speed movement in unimanual operations becomes active when the movement is accelerated to a certain speed, which is much higher than finger/hand/arm moving speed in his/her dairy life and less than the speed limitation of his/her operation. This result reveals that the coupling between left and right motor cortex areas is the reason why the bimanual operation (two hands movement) effects with each other. For each subject participated in this research, a higher moving speed in bimanual operation results in more active in both sides of the motor cortex area. And the threshold speed with which the motor cortex area becomes much more active depends on the operation skill of the subject. More skilled the operator is, higher the threshold speed is.

Electromechanical coupled nonlinear dynamics of euler beam rails for electromagnetic railgun

Lizhong Xu,Dewen Wu 국제구조공학회 2017 Smart Structures and Systems, An International Jou Vol.19 No.2

The electromagnetic field can cause an essential change of the dynamic behavior of the railgun. The evaluation of the dynamics performance of railgun is a mandatory task. Here, a nonlinear electromagnetic force equation of the railgun is given in which the clearance, the thickness and the width of the rail are considered. Based on it, the nonlinear electromechanical coupled dynamics equations of Euler beam rails for the railgun are proposed. Using the equations, the nonlinear free vibration frequency of the railgun is investigated and the effects of the system parameters on the frequency are analyzed. The nonlinear forced responses of the rail to the electromagnetic excitation are investigated as well. The results show that as the nonlinearity of the railgun system is considered, the vibration frequencies of the railgun system increase; as the current in the rail increases, the difference between the natural frequencies and the nonlinear vibration frequencies increases significantly; the nonlinearity of the railgun system is more obvious for smaller distance between the two rails, smaller rail thickness, and smaller stiffness of the elastic foundation; the unstable dynamics state of the rail system occurs when the armature runs to the exit of the railgun. The results are useful for design and application of the railgun system.