POMDPStressTesting.jl

Adaptive stress testing of black-box systems, implemented within the POMDPs.jl ecosystem.

See the documentation for more details.

Citation

If you use this package for research purposes, please cite the following:

@article{moss2021pomdpstresstesting,
  title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
  author = {Robert J. Moss},
  journal = {Journal of Open Source Software},
  year = {2021},
  volume = {6},
  number = {60},
  pages = {2749},
  doi = {10.21105/joss.02749}
}

Interface

To stress test a new system, the user has to define the GrayBox and BlackBox interface outlined in src/GrayBox.jl and src/BlackBox.jl.

GrayBox Interface

The GrayBox simulator and environment interface includes:

GrayBox.Simulation type to hold simulation variables
GrayBox.environment(sim::Simulation) to return the collection of environment distributions
GrayBox.transition!(sim::Simulation) to transition the simulator, returning the log-likelihood

BlackBox Interface

The BlackBox system interface includes:

BlackBox.initialize!(sim::Simulation) to initialize/reset the system under test
BlackBox.evaluate!(sim::Simulation) to evaluate/execute the system under test
BlackBox.distance(sim::Simulation) to return how close we are to an event
BlackBox.isevent(sim::Simulation) to indicate if a failure event occurred
BlackBox.isterminal(sim::Simulation) to indicate the simulation is in a terminal state

Functions ending with ! may modify the Simulation object in place.

Solvers

Several solvers are implemented.

Reinforcement learning solvers

MCTSPWSolver

Deep reinforcement learning solvers¹

Stochastic optimization solvers

CEMSolver

Baseline solvers

RandomSearchSolver

Example

An example implementation of the AST interface is provided for the Walk1D problem:

Julia source: test/Walk1D.jl
Jupyter notebook: Walk1D.ipynb
Descriptive tutorial-style write-up: walk1d.pdf (created using TeX.jl)

Installation

Install RLInterface.jl then the POMDPStressTesting.jl package via:

using Pkg
pkg"add https://github.com/JuliaPOMDP/RLInterface.jl"
pkg"add https://github.com/sisl/POMDPStressTesting.jl"

Testing

To run the test suite, you can use the Julia package manager.

] test POMDPStressTesting

Contributing

We welcome contributions! Please fork the repository and submit a new Pull Request.

Package maintained by Robert Moss: mossr@cs.stanford.edu

¹ TRPO and PPO thanks to Shreyas Kowshik's initial implementation.

Suggest Category