A Fuzzy Q-Learning Algorithm for Storage Optimization in Islanding Microgrid

다국어 초록 (Multilingual Abstract)

In islanding microgrids, energy storage plays a key role in obtaining fl exible power control and operation. The energy storage solves the eff ects of randomness, intermittency and uncertainty of renewable energy through its peak regulation and frequency modulation. In order to better to improve the economics of the microgrid, this paper proposes a Q-learning algorithm based on fuzzy control. It is a model-free algorithm, without complicated modelling, the new algorithm can handle the state space explosion problem of the Q learning algorithm. This method improves the system’s renewable resource utilization rate by controlling the energy fl ow of the energy storage and adjusting the system’s energy scheduling. The results show that compared with the Q-learning, the fuzzy Q-learning algorithm reduced the use of fossil fuels by 33.7%, increases the utilization rate of renewable resources from 85.7 to 93.5%.

참고문헌 (Reference)

1 Etemad S, "Using reinforcement learning to make smart energy storage sources in microgrid" 4 (4): 45-55, 2015

2 Malik H, "Transmission line fault classifi cation using modifi ed fuzzy Q learning" 11 (11): 4041-4050, 2017

3 D’silva S, "Towards grid of microgrids. Seamless transition between grid-connected and islanded modes of operation" 1 : 66-81, 2020

4 Hong T, "Supplementary damping controller of grid connected dc micro-grids based on Q-learning" 1-5, 2016

5 Adewuyi OB, "Static voltage stability improvement with battery energy storage considering optimal control of active and reactive power injection" 172 : 303-312, 2019

6 Zhao B, "Service restoration for a renewable-powered microgrid in unscheduled island mode" 6 (6): 1128-1136, 2015

7 Milbrandt AR, "Renewable energy potential on marginal lands in the United States" 29 : 473-481, 2014

8 Zarrabian S, "Reinforcement learning approach for congestion management and cascading failure prevention with experimental application" 141 : 179-190, 2016

9 Maia R, "Regenerative braking system modeling by fuzzy Q-Learning" 93 : 103712-, 2020

10 Nieto A, "Power quality improvement in power grids with the integration of energy storage systems" 5 (5): 438-443, 2016

11 Paliwal NK, "Optimal sizing and operation of battery storage for economic operation of hybrid power system using artifi cial bee colony algorithm" 29 (29): 2685-, 2019

12 Xu Y, "Optimal control based energy management of multiple energy storage in a microgrid" 6 : 32925-32934, 2018

13 Gong Q, "Optimal confi guration of the energy storage system in ADN considering energy storage operation strategy and dynamic characteristic" 14 (14): 1005-1011, 2020

14 Hossain MA, "Modifi ed PSO algorithm for real-time energy ma-nagement in grid-connected microgrids" 136 : 746-757, 2019

15 Tan Y, "Microgrid stochastic economic load dispatch based on two-point estimate method and improved particle swarm optimization" 25 (25): 2144-2216, 2015

16 Boujoudar Y, "Intelligent controller-based energy management for stand-alone power system using artifi cial neural network" 30 (30): 12579-, 2020

17 Yunjun Yu, "Improved Elman Neural Network Short‑Term Residents Load Forecasting Considering Human Comfort Index" 대한전기학회 14 (14): 2315-2322, 2019

18 Islam M, "Grid power fluctuation reduction by fuzzy control-based energy management system in residential microgrids" 29 : 2758-, 2019

19 Jouffe L, "Fuzzy inference system learning by reinforcement methods" 28 (28): 338-355, 1998

20 Agamah S, "Energy storage system scheduling for peak demand reduction using evolutionary combinatorial optimization" 23 : 73-82, 2017

21 Jingwen Liu, "Energy management for households considering uncertainty in solar irradiance with various probability distribution" 대한전기학회 14 (14): 1943-1956, 2019

22 Du Y, "Distributed MPC for coordinated energy effi ciency utilization in microgrid systems" 10 (10): 1781-1790, 2019

23 Hossain MA, "DC-link voltage regulation of inverters to enhance microgrid stability during network contingencies" 147 : 233-244, 2017

24 Elsharafy MZ, "Back-feed power restoration using distributed constraint optimization in smart distribution grids clustered into microgrids" 206 : 1102-1117, 2017

25 Malysz P, "An optimal energy storage control strategy for grid-connected microgrids" 5 (5): 1785-1796, 2014

26 Anand H, "A real time pricing strategy for remote micro-grid with economic emission dispatch and stochastic renewable energy sources" 127 : 779-789, 2018

27 Jafari M, "A novel predictive fuzzy logic-based energy management system for gridconnected and off-grid operation of residential smart microgrids" 8 (8): 1391-1404, 2020

28 Elbidairi KS, "A hybrid energy management and battery size optimization for standalone microgrids : a case study for Flinders Island, Australia" 175 : 192-212, 2018

29 Agamah S, "A heuristic combinatorial optimization algorithm for load-leveling and peak demand reduction using energy storage systems" 45 (45): 2093-2103, 2018

30 Navin NK, "A fuzzy reinforcement learning approach to thermal unit commitment problem" 31 (31): 737-750, 2019

31 Usman Munawar, "A Framework of Using Machine Learning Approaches for Short‑Term Solar Power Forecasting" 대한전기학회 15 (15): 561-569, 2020