NeurIPS Competition¶
These are the most common questions regarding the NeurIPS 2020 Flatland Challenge. If your questions are not answered please check the Forum and post your question there.
How can I win prizes in this challenge?¶
To be announced!
What are the deadlines for the flatland challenge?¶
To be announced!
How is the score of a submission computed?¶
The scores of your submission are computed as follows:
Mean number of agents done, in other words how many agents reached their target in time.
Mean reward is just the mean of the cummulated reward.
If multiple participants have the same number of done agents we compute a “nomralized” reward as follows:
normalized_reward =cumulative_reward / (self.env._max_episode_steps +self.env.get_num_agents()
The mean number of agents done is the primary score value, only when it is tied to we use the “normalized” reward to determine the position on the leaderboard.
How do I submit to the Flatland Challenge?¶
Follow the instructions in the starter kit to get your first submission.
Can I use env variables with my controller?¶
Yes you can. You can access all environment variables as you please. We recommend you use a custom observation builder to do so as explained here.
What are the time limits for my submission?¶
If there is no action on the server for 10 minutes the submission will be cancelled and a time-out error wil be produced.
If the submissions in total takes longer than 8 hours a time-out will occur.
What are the parameters for the environments for the submission scoring?¶
The environments vary in size and number of agents as well as malfunction parameters. The upper limit of these variables for submissions are:
(x_dim, y_dim) <= (150, 150)n_agents <= 400malfunction_interval >= 50