ReinforcementLearningEnvironments.jl

A one-stop package for different reinforcement learning environments
Author JuliaReinforcementLearning
Popularity
52 Stars
Updated Last
1 Month Ago
Started In
March 2019

ReinforcementLearningEnvironments.jl

⚠️ This package is moved into ReinforcementLearning.jl (2021-05-06)

CI

This package serves as a one-stop place for different kinds of reinforcement learning environments.

Install:

pkg> add ReinforcementLearningEnvironments

API

All the environments here are supposed to have implemented the AbstractEnv related interfaces in ReinforcementLearningBase.jl.

Supported Environments

By default, only some basic environments are installed and exported to make sure that they work on different kinds of platforms. If you want to use some other environments, you'll need to add those dependencies correspondingly.

Built-in Environments

Traits 1 2 3 4 5 6 7 8 9 10 11 12 13
ActionStyle MinimalActionSet
FullActionSet
ChanceStyle Stochastic
Deterministic
ExplicitStochastic
DefaultStateStyle Observation
InformationSet
DynamicStyle Simultaneous
Sequential
InformationStyle PerfectInformation
ImperfectInformation
NumAgentStyle MultiAgent
SingleAgent
RewardStyle TerminalReward
StepReward
StateStyle Observation
InformationSet
InternalState
UtilityStyle GeneralSum
ZeroSum
ConstantSum
IdenticalUtility
  1. MultiArmBanditsEnv
  2. RandomWalk1D
  3. TigerProblemEnv
  4. MontyHallEnv
  5. RockPaperScissorsEnv
  6. TicTacToeEnv
  7. TinyHanabiEnv
  8. PigEnv
  9. KuhnPokerEnv
  10. AcrobotEnv
  11. CartPoleEnv
  12. MountainCarEnv
  13. PendulumEnv

Note: Many traits are borrowed from OpenSpiel.

3-rd Party Environments

Environment Name Dependent Package Name Description
AtariEnv ArcadeLearningEnvironment.jl
GymEnv PyCall.jl
OpenSpielEnv OpenSpiel.jl
SnakeGameEnv SnakeGames.jl SingleAgent/Multi-Agent, FullActionSet/MinimalActionSet
#list-of-environments GridWorlds.jl Environments in this package use the interfaces defined in RLBae directly

Usage

julia> using ReinforcementLearningEnvironments

julia> using ReinforcementLearningBase

julia> env = CartPoleEnv()
# CartPoleEnv

## Traits

| Trait Type        |                  Value |
|:----------------- | ----------------------:|
| NumAgentStyle     |          SingleAgent() |
| DynamicStyle      |           Sequential() |
| InformationStyle  | ImperfectInformation() |
| ChanceStyle       |           Stochastic() |
| RewardStyle       |           StepReward() |
| UtilityStyle      |           GeneralSum() |
| ActionStyle       |     MinimalActionSet() |
| StateStyle        |     Observation{Any}() |
| DefaultStateStyle |     Observation{Any}() |

## Is Environment Terminated?

No

## State Space

`Space{Array{IntervalSets.Interval{:closed,:closed,Float64},1}}(IntervalSets.Interval{:closed,:closed,Float64}[-4.8..4.8, -1.0e38..1.0e38, -0.41887902047863906..0.41887902047863906, -1.0e38..1.0e38])`

## Action Space

`Base.OneTo(2)`

## Current State

[-0.032343893118127506, -0.04221525994544837, 0.024350079684957393, 0.04059943022508135]

julia> state(env)
4-element Array{Float64,1}:
  0.02688439956517477
 -0.0003235577964125977
  0.019563124862911535
 -0.01897808522860225

julia> action_space(env)
Base.OneTo(2)

julia> while !is_terminated(env)
           env(rand(legal_action_space(A)))
       end

julia> using ArcadeLearningEnvironment  # to use 3rd-party environments, you need to manually install and use the dependency first

julia> env = AtariEnv("pong");

Environment Wrappers

Many handy environment wrappers are provided to mimic the OOP style manipulation.

  • ActionTransformedEnv
  • DefaultStateStyleEnv
  • MaxTimeoutEnv
  • RewardOverriddenEnv
  • StateCachedEnv
  • StateOverriddenEnv

Application

Checkout experiments in ReinforcementLearningZoo.jl for how to apply modern reinforcement learning algorithms to these environments. You may also want to read this pluto notebook for how to write a customized environment.