WEIGHT SPEEDY Q-LEARNING FOR FEEDBACK STABILIZATION OF PROBABILISTIC BOOLEAN CONTROL NETWORKS
http://dx.doi.org/10.17654/0972096023009
Keywords:
probabilistic Boolean control networks, feedback stabilization, weight speedy Q-learning, model-free techniqueAbstract
In this paper, a reinforcement learning (RL)-based scalable technique is presented to control the probabilistic Boolean control networks (PBCNs). In particular, we propose an improved Q-learning (QL) algorithm: weight speedy Q-learning (WSQL). Based on WSQL, the feedback stability problem of PBCN is solved, and the state feedback controller is designed to make the PBCN stable at the given equilibrium point. According to the design of the controller, the PBCN can have finite-time stability and asymptotic stability. The presented method is model-free and offers scalability. We also verify the convergence of the proposed algorithm. Finally, simulation results illustrate that compared with the QL, our proposed algorithm converges to the fixed point faster.
Received: February 3, 2023; Accepted: March 17, 2023; Published: April 19, 2023
References
S. A. Kauffman, Metabolic stability and epigenesis in randomly constructed genetic nets, J. Theoret. Biol. 22(3) (1969), 437-467.
H. Kitano, Computational systems biology, Nature 420(6912) (2002), 206-210.
B. Faryabi, E. R. Dougherty and A. Datta, On approximate stochastic control in genetic regulatory networks, IET Syst. Biol. 1(6) (2007), 361-368.
T. Akutsu, S. Kosub, A. A. Melkman and T. Tamura, Finding a periodic attractor of a Boolean network, IEEE/ACM Trans. Comput. Biol. Bioinf. 9(5) (2012), 1410-1421.
D. Cheng, H. Qi and Z. Li, Analysis and Control of Boolean Networks: A Semi-tensor Product Approach, Springer, London, U.K., 2011.
F. Li, H. Yan and H. R. Karimi, Single-input pinning controller design for reachability of Boolean networks, IEEE Trans. Neural Netw. Learn. Syst. 29(7) (2018), 3264-3269.
H. Li, Y. Wang and Z. Liu, Stability analysis for switched Boolean networks under arbitrary switching signals, IEEE Trans. Automat. Control 59(7) (2014), 1978-1982.
R. Liu, J. Lu, Y. Liu, J. Cao and Z.-G. Wu, Delayed feedback control for stabilization of Boolean control networks with state delay, IEEE Trans. Neural Netw. Learn. Syst. 29(7) (2018), 3283-3288.
H. Chen, J. Liang, T. Huang and J. Cao, Synchronization of arbitrarily switched Boolean networks, IEEE Trans. Neural Netw. Learn. Syst. 28(3) (2017), 612-619.
I. Shmulevich, E. R. Dougherty, S. Kim and W. Zhang, Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks, Bioinformatics 18(2) (2002), 261-274.
E. Fornasini and M. E. Valcher, Observability and reconstructibility of probabilistic Boolean networks, IEEE Control Syst. Lett. 4(2) (2020), 319-324.
Y. Guo, R. Zhou, Y. Wu, W. Gui and C. Yang, Stability and set stability in distribution of probabilistic Boolean networks, IEEE Trans. Automat. Control 64(2) (2019), 736-742.
R. Zhou, Y. Guo, Y. Wu and W. Gui, Asymptotical feedback set stabilization of probabilistic Boolean control networks, IEEE Trans. Neural Netw. Learn. Syst. 31 (2020), 4524-4537. doi: 10.1109/TNNLS.2019.2955974.
L. Wang, Y. Liu, Z.-G. Wu, J. Lu and L. Yu, Stabilization and finite-time stabilization of probabilistic Boolean control networks, IEEE Trans. Syst., Man, Cybern.: Syst. 51 (2021), 1559-1566. doi: 10.1109/TSMC.2019.2898880.
H. Li, Y. Wang and P. Guo, State feedback based output tracking control of probabilistic Boolean networks, Inform. Sci. 349 (2016), 1-11.
A. Yerudkar, C. Del Vecchio and L. Glielmo, Output tracking control of probabilistic Boolean control networks, Proc. IEEE Int. Conf. Syst. Man, Cybern. (SMC), Bari, Italy, 2019, pp. 2109-2114.
W. Stach, L. Kurgan, W. Pedrycz and M. Reformat, Genetic learning of fuzzy cognitive maps, Fuzzy Sets Syst. 153(3) (2005), 371-401.
M. U. Ahmed, S. Brickman, A. Dengg, N. Fasth, M. Mihajlovic and J. Norman, A machine learning approach to classify pedestrians events based on IMU and GPS, Int. J. Artif. Intell. 17(2) (2019), 154-167.
F. L. Lewis, D. Vrabie and K. G. Vamvoudakis, Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers, IEEE Control Syst. Mag. 32(6) (2012), 76-105.
B. Faryabi, A. Datta and E. Dougherty, On approximate stochastic control in genetic regulatory networks, IET Syst. Biol. 1(6) (2007), 361-368.
A. Sootla, N. Strelkowa, D. Ernst, M. Barahona and G.-B. Stan, On periodic reference tracking using batch-mode reinforcement learning with application to gene regulatory network control, Proc. 52nd IEEE Conf. Decis. Control, Florence, Italy, 2013, pp. 4086-4091.
Aivar Sootla, Natalja Strelkowa, Damien Ernst, Mauricio Barahona and Guy-Bart Stan, Toggling a genetic switch using reinforcement learning, 2013.
arXiv preprint arXiv:1303.3183.
Georgios Papagiannis and Sotiris K. Moschoyiannis, Deep reinforcement learning for control of probabilistic Boolean networks, International Workshop on Complex Networks and their Applications, 2019.
A. Acernese, A. Yerudkar, L. Glielmo and C. Del Vecchio, Reinforcement learning approach to feedback stabilization problem of probabilistic Boolean control networks, IEEE Control Syst. Lett. 5(1) (2021), 337-342.
doi: 10.1109/LCSYS. 2020.3001993
Tommi Jaakkola, Michael I. Jordan and Satinder P. Singh, On the convergence of stochastic iterative dynamic programming algorithms, Neural Comput. 6(6) (1994), 1185-1201.
M. G. Azar, R. Munos, M. Ghavamzadeh and H. Kappen, Speedy Q-learning, Advances in Neural Information Processing Systems, 2011.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Far East Journal of Applied Mathematics
This work is licensed under a Creative Commons Attribution 4.0 International License.
____________________________
Licensing Terms:
This work is published by Pushpa Publishing House and is subject to the following conditions:
Attribution: You must credit Pushpa Publishing House as the original publisher. Include the publication title and author(s) if applicable.
No Derivatives: Modifying the work or creating derivative works is not permitted without prior written permission.
For more information or permissions beyond the scope of this license, contact Pushpa Publishing House.
Publication count: 
Google h-index: 
Downloads: 