ReinforcementLearning.jl

A reinforcement learning package for Julia
Author JuliaReinforcementLearning
Popularity
217 Stars
Updated Last
1 Year Ago
Started In
June 2018


ReinforcementLearning.jl, as the name says, is a package for reinforcement learning research in Julia.

Our design principles are:

  • Reusability and extensibility: Provide elaborately designed components and interfaces to help users implement new algorithms.
  • Easy experimentation: Make it easy for new users to run benchmark experiments, compare different algorithms, evaluate and diagnose agents.
  • Reproducibility: Facilitate reproducibility from traditional tabular methods to modern deep reinforcement learning algorithms.

๐Ÿน Get Started

julia> ] add ReinforcementLearning

julia> using ReinforcementLearning

julia> run(
           RandomPolicy(),
           CartPoleEnv(),
           StopAfterStep(1_000),
           TotalRewardPerEpisode()
       )

The above simple example demonstrates four core components in a general reinforcement learning experiment:

Check out the tutorial page to learn how these four components are assembled together to solve many interesting problems. We also write blog occasionally to explain the implementation details of some algorithms. Among them, the most recommended one is An Introduction to ReinforcementLearning.jl, which explains the design idea of this package. Besides, a collection of experiments are also provided to help you understand how to train or evaluate policies, tune parameters, log intermediate data, load or save parameters, plot results and record videos. For example:

[TODO: Use ReinforcementLearningExperiments.jl instead]

Here are the full list of available experiments! [TODO: Insert Demo Link]
  • E`JuliaRL_BasicDQN_CartPole`
  • E`JuliaRL_DQN_CartPole`
  • E`JuliaRL_PrioritizedDQN_CartPole`
  • E`JuliaRL_Rainbow_CartPole`
  • E`JuliaRL_IQN_CartPole`
  • E`JuliaRL_A2C_CartPole`
  • E`JuliaRL_A2CGAE_CartPole` (Thanks to @sriram13m
  • E`JuliaRL_MAC_CartPole` (Thanks to @RajGhugare19)
  • E`JuliaRL_PPO_CartPole`
  • E`JuliaRL_VPG_CartPole` (Thanks to @norci)
  • E`JuliaRL_DDPG_Pendulum`
  • E`JuliaRL_TD3_Pendulum` (Thanks to @rbange)
  • E`JuliaRL_SAC_Pendulum` (Thanks to @rbange)
  • E`JuliaRL_PPO_Pendulum`
  • E`JuliaRL_BasicDQN_MountainCar` (Thanks to @felixchalumeau)
  • E`JuliaRL_DQN_MountainCar` (Thanks to @felixchalumeau)
  • E`JuliaRL_Minimax_OpenSpiel(tic_tac_toe)`
  • E`JuliaRL_TabularCFR_OpenSpiel(kuhn_poker)`
  • E`JuliaRL_DeepCFR_OpenSpiel(leduc_poker)`
  • E`JuliaRL_DQN_SnakeGame`
  • E`JuliaRL_BC_CartPole`
  • E`JuliaRL_BasicDQN_EmptyRoom`
  • E`Dopamine_DQN_Atari(pong)`
  • E`Dopamine_Rainbow_Atari(pong)`
  • E`Dopamine_IQN_Atari(pong)`
  • E`rlpyt_A2C_Atari(pong)`
  • E`rlpyt_PPO_Atari(pong)`
  • ๐ŸŒฒ Project Structure

    ReinforcementLearning.jl itself is just a wrapper around several other subpackages. The relationship between them is depicted below:

    +-----------------------------------------------------------------------------------+
    |                                                                                   |
    |  ReinforcementLearning.jl                                                         |
    |                                                                                   |
    |      +------------------------------+                                             |
    |      | ReinforcementLearningBase.jl |                                             |
    |      +----|-------------------------+                                             |
    |           |                                                                       |
    |           |     +--------------------------------------+                          |
    |           +---->+ ReinforcementLearningEnvironments.jl |                          |
    |           |     +--------------------------------------+                          |
    |           |                                                                       |
    |           |     +------------------------------+                                  |
    |           +---->+ ReinforcementLearningCore.jl |                                  |
    |                 +----|-------------------------+                                  |
    |                      |                                                            |
    |                      |     +-----------------------------+                        |
    |                      +---->+ ReinforcementLearningZoo.jl |                        |
    |                            +----|------------------------+                        |
    |                                 |                                                 |
    |                                 |     +-------------------------------------+     |
    |                                 +---->+ DistributedReinforcementLearning.jl |     |
    |                                       +-------------------------------------+     |
    |                                                                                   |
    +------|----------------------------------------------------------------------------+
           |
           |     +-------------------------------------+
           +---->+ ReinforcementLearningExperiments.jl |
           |     +-------------------------------------+
           |
           |     +----------------------------------------+
           +---->+ ReinforcementLearningAnIntroduction.jl |
                 +----------------------------------------+
    
    

    ๐Ÿ–– Supporting

    ReinforcementLearning.jl is a MIT licensed open source project with its ongoing development made possible by many contributors in their spare time. However, modern reinforcement learning research requires huge computing resource, which is unaffordable for individual contributors. So if you or your organization could provide the computing resource in some degree and would like to cooperate in some way, please contact us!

    โœ๏ธ Citing

    If you use ReinforcementLearning.jl in a scientific publication, we would appreciate references to the CITATION.bib.

    โœจ Contributors

    Thanks goes to these wonderful people (emoji key):


    jbrea

    ๐Ÿ’ป ๐Ÿ“– ๐Ÿšง

    Jun Tian

    ๐Ÿ’ป ๐Ÿ“– ๐Ÿšง ๐Ÿค”

    Aman Bhatia

    ๐Ÿ“–

    Alexander Terenin

    ๐Ÿ’ป

    Sid-Bhatia-0

    ๐Ÿ’ป

    norci

    ๐Ÿ’ป ๐Ÿšง

    Sriram

    ๐Ÿ’ป

    Pavan B Govindaraju

    ๐Ÿ’ป

    Alex Lewandowski

    ๐Ÿ’ป

    Raj Ghugare

    ๐Ÿ’ป

    Roman Bange

    ๐Ÿ’ป

    Felix Chalumeau

    ๐Ÿ’ป

    Rishabh Varshney

    ๐Ÿ’ป

    Zachary Sunberg

    ๐Ÿ’ป ๐Ÿ“– ๐Ÿšง ๐Ÿค”

    Jonathan Laurent

    ๐Ÿค”

    Andriy Drozdyuk

    ๐Ÿ“–

    Ritchie Lee

    ๐Ÿ›

    Xirui Zhao

    ๐Ÿ’ป

    Nerd

    ๐Ÿ“–

    Albin Heimerson

    ๐Ÿ’ป

    michelangelo21

    ๐Ÿ›

    GuoYu Yang

    ๐Ÿ“– ๐Ÿ’ป

    Prasidh Srikumar

    ๐Ÿ’ป

    Ilan Coulon

    ๐Ÿ’ป

    Jinrae Kim

    ๐Ÿ“– ๐Ÿ›

    luigiannelli

    ๐Ÿ›

    Jacob Boerma

    ๐Ÿ’ป

    Xavier Valcarce

    ๐Ÿ›

    Ashwani Rathee

    ๐Ÿ’ป

    This project follows the all-contributors specification. Contributions of any kind welcome!