Stochastic events are common in railway networks. The initial plan often needs to be rescheduled during operations as minor events such as delayed departure from train stations, various malfunctions on trains or infrastructure, or even problematic weather lead to delayed trains.
Malfunctions are implemented using a Poisson process to simulate delays by stopping agents at random times for random durations. Train that malfunction can’t move for a random, but known, number of steps. They of course block the trains following them 😬
The parameters necessary for the stochastic events are provided as a
stochastic_data = MalfunctionParameters( malfunction_rate=1/10000, # Rate of malfunction occurence min_duration=15, # Minimal duration of malfunction max_duration=50 # Max duration of malfunction )
The parameters are as follows:
malfunction_rateis the mean rate of the poisson process in number of environment steps.
max_durationset the range of malfunction durations. They are sampled uniformly.
You can then introduce stochasticity in an environment by using the
malfunction_generator parameter of the
RailEnv( ... malfunction_generator=ParamMalfunctionGen(stochastic_data), ... )
In your controller, you can then check whether an agent is malfunctioning:
obs, rew, done, info = env.step(actions) ... action_dict = dict() for a in range(env.get_num_agents()): if info['malfunction'][a] > 0: # agent is malfunctioning and can't move! # info['malfunction'][a] contains the number of steps this agent will still be blocked
You will quickly realize that this will lead to unforeseen difficulties which means that your controller needs to observe the environment at all times to be able to react to the stochastic events!