Reinforcement learning approach for optimising traffic signal timings at isolated intersections

Abstract

One of effective ways to prevent congestion and delay on urban areas is signal control at intersections. Signal systems are operated according to state of intersections either isolated or coordinated signal systems. Many researches have been investigated to improve traffic signal systems based on delay minimization or capacity maximization throughput. Due to complexity of the system, new methods are needed to improve efficiency of signalization in aroad network.Signal setting parameters are usually obtained by minimizing total delay on anintersection. The delay is the key parameter which determines the level of service of an intersection. Delay is defined with two parts as an uniform and non-uniform. The uniform partof the delay is determined basically using conventional delay formulas. But the non-uniformpart is not easily determined and cannot be represent due to the nature of the problem and randomness in arrivals.In this study, Reinforcement Learning Signal Optimizer (RLSO) is used to optimize signal timings in isolated intersection because of reflecting the effect of non-uniform part ofdelay. Reinforcement Learning (RL) which is an approach to artificial intelligence that emphasizes learning by the individual from its interaction with its environment. This contrasts with classical approaches to artificial intelligence and machine learning, which have downplayed learning from interaction, focusing instead on learning from a knowledgeable teacher, or on reasoning from a complete model of the environment. RL is learning what todo-how to map situations to actions-so as to maximize a scalar reward signal. The learner isnot told which action to take, as in most forms of machine learning, but instead must discoverwhich actions yield the most reward by trying them.The aim of this paper is to minimize delay on intersections controlled by isolated signal system and to obtain operational parameters such as cycle time, green split rate. For thispurpose, the RLSO is applied to an example intersection which has four approaches and threestages. The results of RLSO were compared with field observations. The results showed thatthe RLSO is able to optimize traffic signal timings on an intersection. The proposed model also holds promise for successful application to optimize traffic signal timings at isolated intersections according to delay minimization.