Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

Tianle Zhang,Zhen Liu,Zhiqiang Pu,Jianqiang Yi,Yanyan Liang,Du Zhang 제어·로봇·시스템학회 2023 International Journal of Control, Automation, and Vol.21 No.7

Although deep reinforcement learning has recently achieved some successes in robot navigation, there are still unsolved problems. Particularly, a robot guided by a distant ultimate goal is easy to get stuck in danger or encounter collisions in dynamic crowded environments due to the lack of long-term perspectives. In this paper, a novel subgoal-guided approach based on two-level hierarchical deep reinforcement learning with spatial-temporal graph attention networks (ST-GANets), called SG-HDRL, is proposed for a robot navigating in a dynamic crowded environment with autonomous obstacles, e.g., crowd. Specifically, the high-level policy, that models the spatialtemporal relation between the robot and the obstacles using the obstacles’ trajectories by the designed high-level ST-GANet, generates intermediate subgoals from a longer-term perspective over higher temporal scales. The subgoals give a favorable and collision-free direction to avoid encountering danger or collisions while approaching the ultimate goal. The low-level policy, that similarly implements the designed low-level ST-GANet to implicitly predict the obstacles’ motions, takes the subgoals as short-term guidance through an intrinsic reward incentive to generate primitive actions for the robot. Simulation results demonstrate that SG-HDRL using ST-GANets has better performances compared with state-of-the-art baselines. Furthermore, the proposed SG-HDRL is deployed to an experimental platform based on omnidirectional cars, and experiment results validate the effectiveness and practicability of the proposed SG-HDRL.

A QoS Constraints Location-based Services Selection Model and Algorithm under Mobile Internet Environment

Mingjun Xin,Tianle Jiang,Rui Zhang 보안공학연구지원센터(IJGDC) 2014 International Journal of Grid and Distributed Comp Vol.7 No.2

Venture Capital Participation and Post-IPO Performance, an Empirical Research on Companies Listed in China

Tao Li,Fang Zhang,Hao Liu,Tianle Zhai 보안공학연구지원센터 2016 International Journal of u- and e- Service, Scienc Vol.9 No.5

Extensive studies from American and some European stock markets have pointed out that venture capital (VC) has a constructive influence on sustainable developments of their invested companies. In this paper, RBF neural network, principal component analysis, and mean variance analysis are functional and well utilized to test the effects of VCs on enterprises' post-IPO performances in China. The empirical results indicate that venture capitals in China brought undesirable effects to both operating performance and market performance, but performed well in controlling harmful volatility. Therefore, we believe VC in China has a certain positive external but not internal influence on sustainable developments of their invested companies.

Genetically predicted physical activity is associated with lower serum urate concentrations

Guan Ying,Wei Jiahe,Meng Lifeng,Li Yasong,Wang Tianle,Chen Dingwan,Qian Qilin 한국유전학회 2022 Genes & Genomics Vol.44 No.7

Background: Physical activity (PA) is considered to play an important role in the reduced gout risk. However, the epidemiology results are inconsistent and causality remains unclear. Objective: To investigate the causal relationship of PA with serum urate concentrations and gout risk by a bidirectional Mendelian randomization (MR) approach. Method: Two genome-wide association studies (GWASs) from UK Biobank were used to identify instrumental variables for self-reported moderate-to-vigorous PA (including 377,234 European individuals), accelerometer-measured 'average acceleration' PA (including 91,084 European individuals) and accelerometer-measured overall PA (including 91,105 European individuals). The summary data for serum urate (including 110,347 European individuals) and gout (including 2,115 cases and 67,259 controls) were derived from GWAS of Global Urate Genetics Consortium. Moreover, reverse direction Mendelian randomization study was conducted. The inverse-variance weighted, weighted median, Mendelian randomization Egger regression, simple mode and weighted mode and Mendelian Randomization Pleiotropy RESidual Sum and Outlier were methods we performed. Result: Genetic predisposition to accelerometer-measured 'average acceleration' PA [beta = -0.038; 95% confidence interval (CI) = -0.060,-0.015; P = 0.001] and accelerometer-measured overall PA (beta = -0.339; 95% CI = -0.522,-0.156; P = 2.8E-4) were significantly associated with decreased serum urate concentrations. Besides, there was no evidence supporting the causal association between PA and gout risk. In the reverse direction analysis, genetic predisposition to both urate and gout were not associated with PA being investigated. Conclusions: In MR study, we found that PA may reduce serum urate concentrations but not the risk of gout. Moreover, serum urate concentrations and gout were not associated with PA.

miR-143 inhibits proliferation and induces apoptosis of mammary epithelial cells in dairy goat

Zhibin Ji,Guizhi Wang,Lei Hou,Zhaohua Liu,Jianmin Wang,Tianle Chao 한국통합생물학회 2016 Animal cells and systems Vol.20 No.2

MicroRNAs are a class of post-transcriptional regulators of gene expression in multicellular organisms, which play important roles in cell fate, organ morphogenesis and carcinogenesis. In the present study, we demonstrated the critical roles of miR-143 on mammary epithelial cells of dairy goat. The test results of cell count, methylthiazolyldiphenyl-tetrazolium bromide, Hoechst33342/PI (propidium iodide) and flow cytometry showed miR-143-induced G0/G1 phase arrest, inhibited proliferation and promoted apoptosis of mammary epithelial cells; the qRT-PCR test of marker genes related to cell proliferation and apoptosis, BAX and BCL-2, supported the same conclusions. Our study presents evidence that miR-143 is an important post-transcription regulator involving in mammary cells survival, and it may have a value function in mammary gland development, lactation or involution.

Adaptive Source Time Synchronization for Low-Duty-Cycle Wireless Sensor Networks

Tian Le 보안공학연구지원센터 2015 International Journal of Future Generation Communi Vol.8 No.4

Time synchronization is critical for most distributed systems, especially for low-duty-cycle Wireless Sensor Networks(WSNs). Low-duty-cycle WSNs make use of time synchronization in many contexts(scheduling and sleeping, TDMA, Event Identification, data fusion, etc). A novel adaptive source time synchronization algorithm designed for low-duty-cycle WSNs, namely ASTS, is presented in the paper. The algorithm can be used for a small Low-Duty-Cycle WSN to be synchronized to a common clock, or used for a giant Low-Duty-Cycle WSN to be synchronized distributed. To improve the synchronization accuracy, all nodes estimate their time drifts relative to their neighbors using Maximum Likelihood Estimation, and get be synchronized to a common clock or their heads based on a vector of time drifts carried by the reference packet sent by the reference node. Simulation shows that the algorithm drastically improves the synchronization accuracy and scalability, and is much more applicable for low-duty-cycle WSNs than other synchronization algorithms.

A Data Fusion Algorithm Based on Neural Network Research in Building Environment of Wireless Sensor Network

Tian Le,Zhao Jing 보안공학연구지원센터 2015 International Journal of Future Generation Communi Vol.8 No.4

Data fusion in wireless sensor network is a effective method to reduce the network energy consumption. In order to build high performance of data fusion system, a data fusion algorithm using BP neural network to optimize fuzzy prediction and train the membership degree of collecting data is presented, which is used to determine which kind of dividing fusion mechanism is belonged for the sensor’s data collected at a given moment. First the fuzzy prediction is used for acquisition of knowledge which data is simplified to remove redundant properties and samples. The BP neural network is used to process fuzzy prediction and finally the patterns of muli-sensed-data(temperature as an example) fusion distribution are formed. Two kinds of different BP neural network are proposed and compared for more precision of the fuzzy prediction result. Second data fusion based on the fuzzy prediction will be implemented to reduce the number of data transmission in the network. Simulation results show that the algorithm has good precision and applicability.

Feature Selection to Mine Joint Features from High-dimension Space for Android Malware Detection

( Yanping Xu ),( Chunhua Wu ),( Kangfeng Zheng ),( Xinxin Niu ),( Tianling Lu ) 한국인터넷정보학회 2017 KSII Transactions on Internet and Information Syst Vol.11 No.9

Android is now the most popular smartphone platform and remains rapid growth. There are huge number of sensitive privacy information stored in Android devices. Kinds of methods have been proposed to detect Android malicious applications and protect the privacy information. In this work, we focus on extracting the fine-grained features to maximize the information of Android malware detection, and selecting the least joint features to minimize the number of features. Firstly, permissions and APIs, not only from Android permissions and SDK APIs but also from the developer-defined permissions and third-party library APIs, are extracted as features from the decompiled source codes. Secondly, feature selection methods, including information gain (IG), regularization and particle swarm optimization (PSO) algorithms, are used to analyze and utilize the correlation between the features to eliminate the redundant data, reduce the feature dimension and mine the useful joint features. Furthermore, regularization and PSO are integrated to create a new joint feature mining method. Experiment results show that the joint feature mining method can utilize the advantages of regularization and PSO, and ensure good performance and efficiency for Android malware detection.

Ferroelectric polarization effect on hysteresis behaviors of single-walled carbon nanotube network field-effect transistors with lead zirconate-titanate gating

Yilin Sun,Dan Xie,Ruixuan Dai,Mengxing Sun,Weiwei Li,Tianling Ren 한국물리학회 2018 Current Applied Physics Vol.18 No.3

We report the fabrication of single-walled carbon nanotube (SWCNT) network transistors by ferroelectric Pb(Zr0.4Ti0.6)O3 (PZT) bottom-gating and investigate the polarization effects of PZT on the transport properties of the transistor device. Our devices exhibit typical p-channel transistor characteristics and a large hysteresis loop with high ON/OFF current ratio and large ON current as well as memory window (MW) measured up to 5.2 V. The origin of clockwise hysteresis is attributed to ferroelectric polarization modulated charge trapping/de-trapping process in the interface states between SWCNT networks and PZT. The retention time about 104s with two high stable current states preliminarily demonstrates great potential for future non-volatile memory applications based on such SWCNT/PZT hybrid systems.