The basic principle of optimal traffic control is the appropriate real-time response to dynamic traffic flow changes. Signal plan efficiency depends on a large number of input parameters. An actuated signal system can adjust very well to traffic conditions, but cannot fully adjust to stochastic traffic volume oscillation. Due to the complexity of the problem analytical methods are not applicable for use in real time, therefore the purpose of this paper is to introduce heuristic method suitable for traffic light optimization in real time. With the evolution of artificial intelligence new possibilities for solving complex problems have been introduced. The goal of this paper is to demonstrate that the use of the Q learning algorithm for traffic lights optimization is suitable. The Q learning algorithm was verified on a road artery with three intersections. For estimation of the effectiveness and efficiency of the proposed algorithm comparison with an actuated signal plan was carried out. The results (average delay per vehicle and the number of vehicles that left road network) show that Q learning algorithm outperforms the actuated signal controllers. The proposed algorithm converges to the minimal delay per vehicle regardless of the stochastic nature of traffic. In this research the impact of the model parameters (learning rate, exploration rate, influence of communication between agents and reward type) on algorithm effectiveness were analysed as well.
Anžek M, Kavran Z, Badanjak D. Adaptive Traffic Control as Function of Safety. 12th World Congress on Intelligent Transport Systems and Services. San Francisco, 2005.
Robertson DI. Research on the TRANSYT and SCOOT Methods of Signal Coordination. ITE Journal. 1986;56(1):36-40.
Hunt PB, Roberetson DI. Bretherton RD, Winton RI. A traffic responsive method of coordinating signals. Crowthorne, Berkshire: Transport and Road Research Laboratory; 1981.
Lowrie PR. Scats, Sydney co-ordinated adaptive traffic system: a traffic responsive method of controlling urban traffic. Darlinghurst, NSW Australia: Roads and Traffic Authority NSW; 1990.
Gartner N. Opac: A demand-responsive strategy for traffic signal control. Transportation Research Board; 1983.
Veljanovska K, Bombol K, Maher T. Reinforcement Learning Technique in Multiple Motorway Access Control Strategy Design. PROMET - Traffic&Transportation. 2010;22(2):117-123.
Sen S, Head K. Controlled optimization of phases at an intersection. Transportation science. 1997;31(1):5-17.
Bingham E. Neurofuzzy Traffic Signal Control [Master’s thesis]. Helsinki, Finland: Dept. of Engineering Physics and Mathematics, Helsinki Univ. of Technology; 1998.
Wiering M. Multi-agent reinforcement learning for traffic light control. ICML ‘00 Proceedings of the 17th International Conference on Machine Learning; 2000.
Thorpe TL, Anderson CW. Traffic Light Control Using SARSA with Three State Representations. IBM Corporation; 1996.
Abdulhai B, Pringle R, Karakoulas GJ. Reinforcement Learning for True Adaptive Traffic Signal Control. Journal of Transportation Engineering. 2003;129(3):278–285.
Gregoire PL, Desjardins C, Laumonier J, Chaib-draa B. Urban Traffic Control Based on Learning Agents. IEEE Intelligent Transportation Systems Conference. IEEE; 2007. p. 916-921.
Lu S, Liu X, Dai S. Q-Learning for Adaptive Traffic Signal Control Based on Delay Minimization Strategy. 2008 IEEE International Conference on Networking, Sensing and Control; 2008. p. 687-91.
Arel I, Liu C, Urbanik T, Kohls AG. Reinforcement learning-based multi-agent system for network traffic signal control. IET Intelligent Transport Systems. 2010:4(2):128-135.
Prashanth, L.A., Bhatnagar, S.: Reinforcement Learning With Function Approximation for Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems. 2011;12(2):412-421.
Sutton RS, Barto A. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press; 1998.
Abdulhai B, Kattan L. Reinforcement learning: Introduction to theory and potential for transport applications. Canadian Journal of Civil Engineering. 2003;30(6):981–991.
Perez-Uribe A. Introduction to reinforcement learning [Internet]; 1998. Available from: http://ape.iict.ch/teaching/AIGS/AIGS_Labo/Labo5-RL/sarsa.html
Sutton RS. Learning to predict by the methods of temporal differences. Machine Learning. 1988;3:9–44
Wiering M, van Veenen J, Vreeken J, Koopman A. Intelligent traffic light control. Technical Report. Utrecht University, Institute of Information and Computing Sciences; 2004. 31 p.
Guest Editor: Eleonora Papadimitriou, PhD
Editors: Dario Babić, PhD; Marko Matulin, PhD; Marko Ševrović, PhD.
The mission of our journal is to provide high quality communication of research results, which is crucial for the effectiveness of scientific research, the promotion of scientific information, new ideas, and original research in the field of traffic and transportation engineering.
1848-4069
0353-5320
Proud publisher
Accelerating Discoveries in Traffic Science |
2024 © Promet - Traffic&Transportation journal
