ReinforcementLearningEnvironments.jl

⚠️ This package is moved into ReinforcementLearning.jl (2021-05-06)

This package serves as a one-stop place for different kinds of reinforcement learning environments.

Install:

pkg> add ReinforcementLearningEnvironments

API

All the environments here are supposed to have implemented the AbstractEnv related interfaces in ReinforcementLearningBase.jl.

Supported Environments

By default, only some basic environments are installed and exported to make sure that they work on different kinds of platforms. If you want to use some other environments, you'll need to add those dependencies correspondingly.

Built-in Environments

Traits		1	2	3	4	5	6	7	8	9	10	11	12	13
ActionStyle	MinimalActionSet	✔	✔	✔		✔		✔	✔	✔	✔	✔	✔	✔
ActionStyle	FullActionSet				✔		✔
ChanceStyle	Stochastic	✔		✔	✔						✔	✔	✔	✔
	Deterministic		✔			✔	✔
	ExplicitStochastic							✔	✔	✔
DefaultStateStyle	Observation	✔	✔	✔	✔	✔	✔		✔		✔	✔	✔	✔
DefaultStateStyle	InformationSet							✔		✔
DynamicStyle	Simultaneous					✔
DynamicStyle	Sequential	✔	✔	✔	✔		✔	✔	✔	✔	✔	✔	✔	✔
InformationStyle	PerfectInformation		✔				✔		✔
InformationStyle	ImperfectInformation	✔		✔	✔	✔		✔		✔	✔	✔	✔	✔
NumAgentStyle	MultiAgent					✔	✔	✔	✔	✔
NumAgentStyle	SingleAgent	✔	✔	✔	✔						✔	✔	✔	✔
RewardStyle	TerminalReward	✔	✔		✔	✔	✔	✔	✔	✔
RewardStyle	StepReward			✔							✔	✔	✔	✔
StateStyle	Observation	✔	✔	✔	✔	✔	✔		✔		✔	✔	✔	✔
	InformationSet							✔		✔
	InternalState			✔
UtilityStyle	GeneralSum	✔	✔	✔	✔						✔	✔	✔	✔
	ZeroSum					✔	✔			✔
	ConstantSum								✔
	IdenticalUtility							✔

Note: Many traits are borrowed from OpenSpiel.

3-rd Party Environments

Environment Name	Dependent Package Name	Description
`AtariEnv`	ArcadeLearningEnvironment.jl
`GymEnv`	PyCall.jl
`OpenSpielEnv`	OpenSpiel.jl
`SnakeGameEnv`	SnakeGames.jl	`SingleAgent`/`Multi-Agent`, `FullActionSet`/`MinimalActionSet`
#list-of-environments	GridWorlds.jl	Environments in this package use the interfaces defined in `RLBae` directly

Usage

julia> using ReinforcementLearningEnvironments

julia> using ReinforcementLearningBase

julia> env = CartPoleEnv()
# CartPoleEnv

## Traits

| Trait Type        |                  Value |
|:----------------- | ----------------------:|
| NumAgentStyle     |          SingleAgent() |
| DynamicStyle      |           Sequential() |
| InformationStyle  | ImperfectInformation() |
| ChanceStyle       |           Stochastic() |
| RewardStyle       |           StepReward() |
| UtilityStyle      |           GeneralSum() |
| ActionStyle       |     MinimalActionSet() |
| StateStyle        |     Observation{Any}() |
| DefaultStateStyle |     Observation{Any}() |

## Is Environment Terminated?

No

## State Space

`Space{Array{IntervalSets.Interval{:closed,:closed,Float64},1}}(IntervalSets.Interval{:closed,:closed,Float64}[-4.8..4.8, -1.0e38..1.0e38, -0.41887902047863906..0.41887902047863906, -1.0e38..1.0e38])`

## Action Space

`Base.OneTo(2)`

## Current State

[-0.032343893118127506, -0.04221525994544837, 0.024350079684957393, 0.04059943022508135]

julia> state(env)
4-element Array{Float64,1}:
  0.02688439956517477
 -0.0003235577964125977
  0.019563124862911535
 -0.01897808522860225

julia> action_space(env)
Base.OneTo(2)

julia> while !is_terminated(env)
           env(rand(legal_action_space(A)))
       end

julia> using ArcadeLearningEnvironment  # to use 3rd-party environments, you need to manually install and use the dependency first

julia> env = AtariEnv("pong");

Environment Wrappers

Many handy environment wrappers are provided to mimic the OOP style manipulation.

ActionTransformedEnv
DefaultStateStyleEnv
MaxTimeoutEnv
RewardOverriddenEnv
StateCachedEnv
StateOverriddenEnv

Application

Checkout experiments in ReinforcementLearningZoo.jl for how to apply modern reinforcement learning algorithms to these environments. You may also want to read this pluto notebook for how to write a customized environment.

Suggest Category