Applying Deep Reinforcement Learning to Improve Throughput and Reduce Collision Rate in IEEE 802.11 Networks

( Chih-heng Ke ),( Lia Astuti ) 한국인터넷정보학회 2022 KSII Transactions on Internet and Information Syst Vol.16 No.1

The effectiveness of Wi-Fi networks is greatly influenced by the optimization of contention window (CW) parameters. Unfortunately, the conventional approach employed by IEEE 802.11 wireless networks is not scalable enough to sustain consistent performance for the increasing number of stations. Yet, it is still the default when accessing channels for single-users of 802.11 transmissions. Recently, there has been a spike in attempts to enhance network performance using a machine learning (ML) technique known as reinforcement learning (RL). Its advantage is interacting with the surrounding environment and making decisions based on its own experience. Deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments (such as continuous state spaces or actions spaces) and to get optimum rewards. As a result, we present a new approach of CW control mechanism, which is termed as contention window threshold (CW_Threshold). It uses the DRL principle to define the threshold value and learn optimal settings under various network scenarios. We demonstrate our proposed method, known as a smart exponential-threshold-linear backoff algorithm with a deep Q-learning network (SETL-DQN). The simulation results show that our proposed SETL-DQN algorithm can effectively improve the throughput and reduce the collision rates.

myEvalSVC: an Integrated Simulation Framework for Evaluation of H.264/SVC Transmission

( Chih-heng Ke ) 한국인터넷정보학회 2012 KSII Transactions on Internet and Information Syst Vol.6 No.1

The ever-increasing demand for H.264 scalable video coding (H.264/SVC) distribution motivates researchers to devise ways to enhance the quality of video delivered on the Internet. Furthermore, researchers and practitioners in general depend on computer simulators to analyze or evaluate their designed network architecture or proposed protocols. Therefore, a complete toolset, which is called myEvalSVC, for evaluating the delivered quality of H.264/SVC transmissions in a simulated environment is proposed to help the network and video coding research communities. The toolset is based on the H.264 Scalable Video coding streaming Evaluation Framework (SVEF) and extended to connect to the NS2 simulator. With this combination, people who work on video coding can simulate the effects of a more realistic network on video sequences resulting from their coding schemes, while people who work on network technology can evaluate the impact of real video streams on the proposed network architecture or protocols. To demonstrate the usefulness of the proposed new toolset, examples of H.264/SVC transmissions over 802.11 and 802.11e are provided.

Improving Video Transmission in Software Defined Wired and Wireless Networks using Multi-Path Transmission

Chih-Heng Ke,Yeong-Sheng Chen,Yun-Shuai Yu 한국통신학회 2017 Journal of communications and networks Vol.19 No.6

Recently, increasing attention has been paid to the studyof applying software defined networks to improve video transmission. Compared with the existing studies which focus on a wirednetwork, this study proposed a multi-path transmission mechanismfor improving the performance of transmitting videos from awired network to a wireless one in a software defined network environment. In the proposed mechanism, a mobile host could applynetwork bonding to integrate several physical or virtual wirelessnetwork modules, each of which was connected to a different accesspoint, into a combined device. Thus, multiple wireless links ofthe mobile host could be established and hence multiple transmissionpaths from the video source in the wired network to the mobilehost were derived. In addition, different amount of video packetswere allocated onto different transmission paths so as to maximizethe performance of the multi-path transmission. According to theresults of our experiments, the proposed mechanism can enhancethe performance of transmission in terms of Peak Signal to NoiseRatio (PSNR).

Applying multi-agent deep reinforcement learning for contention window optimization to enhance wireless network performance

Ke Chih-Heng,Astuti Lia 한국통신학회 2023 ICT Express Vol.9 No.5

This paper investigates the Contention Window (CW) optimization problem in multi-agent scenarios, where the fully cooperative among mobile stations is considered. A partially observable environment is employed to model and analyze the CW optimization problem, and Smart Exponential-Threshold-Linear with Deep Q-learning Network (SETL-DQN) Multi-Agent (MA) algorithm is proposed to obtain the optimal system throughput through the CW Threshold optimization. In the determined scenarios, SETL-DQN(MA) can effectively cope with the mutual interaction among mobile stations. The simulation results show that our proposed method is superior from both static and dynamic scenarios and has the highest optimum packet transmission efficiency.

A reinforcement learning approach for widest path routing in software-defined networks

Ke Chih-Heng,Tu Yi-Hao,Ma Yi-Wei 한국통신학회 2023 ICT Express Vol.9 No.5

In this paper, a routing method based on reinforcement learning (RL) under software-defined networks (SDN), namely the Q-learning widest-path routing algorithm (Q-WPRA), is proposed. This algorithm processes the reward function according to the link bandwidth in the execution environment to find the optimal (i.e., widest) transmission path with the maximum bandwidth between the source and the destination through RL. The experimental results reveal that the Q-WPRA is outperformance than Dijkstra’s algorithm and Dijkstra’s widest-path algorithm to find the widest transmission path in SDN environment under different bandwidths, loss rates, and background traffic.

Antiproliferative and Antitumorigenic Activity of Toona sinensis Leaf Extracts in Lung Adenocarcinoma

Chih-Jen Yang,Yu-Jung Huang,Cheng-Yuan Wang,Chuan-Sheng Wang,Pei-Hui Wang,Jen-Yu Hung,Tung-Heng Wang,Hseng-Kuang Hsu,Hurng-Wern Huang,S.P. Anand Kumar,Ming-Shyan Huang,Ching-Feng Weng 한국식품영양과학회 2010 Journal of medicinal food Vol.13 No.1

Toona sinensis is a traditional Chinese herb, and the extracts of T. sinensis leaf possess a variety of biological functions. This study attempted to test the antiproliferative effect of TSL-1 (a bioactive fraction of T. sinensis) in H441 cells (lung adenocarcinoma). The data showed that the antiproliferative effect of TSL-1 on H441 cells is prominent using the 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide assay. TSL-1-induced apoptosis was confirmed by cell morphology, sub-G1 peak accumulation, cleavage of poly(ADP)-ribose polymerase, and propidium iodide-annexin V double staining. Furthermore, decreased Bcl-2 accompanied by increased Bax (in western blotting) was found with TSL-1 treatment of H441 cells. TSL-1 treatment-induced G1 arrest was concurrent with the down-regulation of protein levels of cyclin D1 and cyclin-dependent kinase 4 in H441 cells. Peroral and intraperitoneal administrations of TSL-1 were performed to evaluate the therapeutic efficacy, and peroral administration of TSL-1 was also used to elucidate the therapeutic efficacy in the H441 cell xenograft model in vivo. The data revealed that TSL-1 treatment inhibited H441 tumor growth in both therapeutic and preventive experiments. Taken together, these results demonstrate that TSL-1 possesses the capability of preventing and alleviating lung cancer proliferation in vitro and in vivo with proven nephrological and hepatic safety and has the potential to be developed as an anti–lung cancer drug.

Epidemiological trend in inflammatory bowel disease in Taiwan from 2001 to 2015: a nationwide population-based study

Hsu-Heng Yen,Meng-Tzu Weng,Chien-Chih Tung,Yu-Ting Wang,Yuan Ting Chang,Chin-Hao Chang,Ming-Jium Shieh,Jau-Min Wong,Shu-Chen Wei 대한장연구학회 2019 Intestinal Research Vol.17 No.1

Background/Aims: Incidences of inflammatory bowel disease (IBD), ulcerative colitis (UC), and Crohn’s disease (CD), havebeen increasing in Asia. In this study, we report the relevant clinical characteristics and determined the epidemiological trend ofIBD in Taiwan from 2001 to 2015. Methods: A retrospective study was conducted to analyze data recorded from January 2001through December 2015 in the registered database compiled by the National Health Insurance and provided by the Ministry ofHealth and Welfare, Taiwan. Results: A total of 3,806 patients with catastrophic IBD illness were registered from 2001 to 2015 inTaiwan (CD, 919; UC, 2,887). The crude incidence of CD increased from 0.17/100,000 in 2001 to 0.47/100,000 in 2015, whereasthat of UC increased from 0.54/100,000 in 2001 to 0.95/100,000 in 2015. The prevalence of CD increased from 0.6/100,000 in2001 to 3.9/100,000 in 2015, whereas that of UC increased from 2.1/100,000 in 2001 to 12.8/100,000 in 2015. The male-to-femaleratio in the study sample was 2.19 for CD and 1.62 for UC. The median age of those registered with CD was lower than that ofthose registered for UC: 38.86 and 44.86 years, respectively. A significantly greater increase in CD incidence rate was identifiedamong 20 to 39-year-old compared with other age groups. Conclusions: Using Taiwan’s nationwide insurance database, we determinedthat the number of patients with CD increased more rapidly during the study period than the number of patients withUC, especially among age 20 to 39-year-old, resulting in a decreased UC-to-CD ratio. (Intest Res 2019;17:54-62)

Treatment Retention Rates of 3-monthly Paliperidone Palmitate and Risk Factors Associated with Discontinuation: A Population-based Cohort Study

Chien-Heng Lin,Huang-Li Lin,Chih-Lin Chiang,Yi-Wen Chen,Yan-Fang Liu,Yen Kuang Yang,Chao-Hsiun Tang 대한정신약물학회 2023 CLINICAL PSYCHOPHARMACOLOGY AND NEUROSCIENCE Vol.21 No.3

Objective: Limited evidence exists regarding real-world 3-monthly paliperidone palmitate (PP3M) treatment retention and associated factors. Methods: We conducted a retrospective, nationwide cohort study using the Taiwan National Health Insurance Research Database between October 2017 and December 2019. Adult patients with schizophrenia initiated on PP3M were enrolled. The primary outcomes were time to PP3M discontinuation, time to psychiatric hospitalization, and the proportions of patients receiving the next PP3M dose within 120 days among first-, second-, and third-dose completers. Key covariates included prior PP1M duration and adequate PP3M initiation. Results: The PP3M treatment retention rates were 79.7%, 66.3%, and 52.5% after 6, 12, and 24 months, respectively, with 86.4%, 90.6%, and 90.0% of respective first-, second-, and third-dose completers receiving the next PP3M dose. Adequate PP3M initiation and prior PP1M treatment duration > 180 days were associated with favorable PP3M treatment retention. In multivariate analyses, PP1M durations of 180−360 days (adjusted relative risk [aRR], 1.76) or < 180 days (aRR, 2.79) were associated with PP3M discontinuation at the second dose. Inadequate PP3M initiation was associated with discontinuation at the third dose (aRR, 2.18). Patients fully adherent to PP3M treatment in the first year had a higher probability of being free from psychiatric hospitalization (86.7% at 2 years), compared with those partially adherent or non-adherent to PP3M in the first year. Conclusion: Prior PP1M duration and adequate PP3M initiation are major factors affecting PP3M treatment retention. Higher PP3M treatment retention is associated with a lower risk of psychiatric hospitalization.

Selective Photomechanical Detachment and Retrieval of Divided Sister Cells from Enclosed Microfluidics for Downstream Analyses

Chen, Yu-Chih,Baac, Hyoung Won,Lee, Kyu-Tae,Fouladdel, Shamileh,Teichert, Kendall,Ok, Jong G.,Cheng, Yu-Heng,Ingram, Patrick N.,Hart, A. John,Azizi, Ebrahim,Guo, L. Jay,Wicha, Max S.,Yoon, Euisik American Chemical Society 2017 ACS NANO Vol.11 No.5

<P>Considerable evidence suggests that self-renewal and differentiation of cancer stem-like cells, a key cell population in tumorgenesis, can determine the outcome of disease. Though the development of microfluidics has enhanced the study of cellular lineage, it remains challenging to retrieve sister cells separately inside enclosed microfluidics for further analyses. In this work, we developed a photomechanical method to selectively detach and reliably retrieve target cells from enclosed microfluidic chambers. Cells cultured on carbon nanotube polydimethylsiloxane composite surfaces can be detached using shear force induced through irradiation of a nanosecond-pulsed laser. This retrieval process has been verified to preserve cell viability, membrane proteins, and mRNA expression levels. Using the presented method, we have successfully performed 96-plex single-cell transcriptome analysis on sister cells in order to identify the genes altered during self-renewal and differentiation, demonstrating phenomenal resolution in the study of cellular lineage.</P>

Cross-Match as an Immuno-Oncological Risk Factor for Hepatocellular Carcinoma Recurrence and Inferior Survival after Living Donor Liver Transplantation: A Call for Further Investigation

( Cheng-maw Ho ),( Rey-heng Hu ),( Yao-ming Wu ),( Ming-chih Ho ),( Po-Huang Lee ) 대한간학회 2021 춘·추계 학술대회 (KASL) Vol.2021 No.1