State machine

Flatland 3 introduces a state machine as a refactor of env.step function in Flatland. The functionality is essentially the same as Flatland 2, this change is primarily to help users who want to read and modify the code easily.

The goal is to provide a unified state machine which controls the state and transitions of the trains, rather than the older system which had a mix of agent.moving and agent.status.

The figure below illustrates the working of the state machine.

The principles we try to implement are as follows:

  1. Only a single state transtition is allowed per env.step.

  2. The action preprocessing ,position changes, speed counters and malfunctions are controlled by the state.

  3. All signals for transitioning between states are generated in one place, for easy readability.

State descriptions

The purpose of each state is described below:

State

Map Position

Description

WAITING

Off Map

No actions allowed here, when current timestep is behind earliest departure.

READY_TO_DEPART

Off Map

Train is ready to move and will start at initial_position when moving action is provided.

MALFUNCTION_OFF_MAP

Off Map

When a randomized malfunction occurs in an off map state, any moving actions provided here are stored and used when malfunction completes, unless stop action is provided.

MOVING

On Map

Indicates the train is moving, if speed is 1.0, the train changes position every timestep.

STOPPED

On Map

Indicates the train is stopped, this can occur when the user provides a stop action, or the train tries to move into a cell that is occupied or to a cell which does not have a track.

MALFUNCTION

On Map

When a randomized malfunction occurs in an on map state, any moving actions provided here are stored and used when malfunction completes, unless stop action is provided. No movement can occur during a malfunction state.

DONE

Depends on remove_agent_at_target

This is a terminal state which is activated when the target is reached.

State transition signals

The corresponding signals in the state machine are described below

State Transition Signal

Description

in_malfunction

Malfunction states start when in_malfunction is set to true

malfunction_counter_complete

Malfunction counter complete - Malfunction state ends this timestep and actions are allowed from next timestep

earliest_departure_reached

Earliest departure reached - Train is allowed to move now

stop_action_given

Stop Action Given - User provided a stop action. Action preprocessing can also change a moving action to a stop action if the train tries to move into a invalid or occupied position.

valid_movement_action_given

Movement action is provided and movement is allowed, note that a train can move into an occupied cell if that cell is being emptied by the train that was previously occupied it.

target_reached

Target position is reached

movement_conflict

Movement conflict - Multiple trains trying to move into same cell