POMDPStressTesting.jl

Adaptive stress testing of black-box systems within POMDPs.jl
Author sisl
Popularity
12 Stars
Updated Last
12 Months Ago
Started In
February 2020

POMDPStressTesting.jl

Documentation Build Status codecov

Adaptive stress testing of black-box systems, implemented within the POMDPs.jl ecosystem.

See the documentation for more details.

Citation

If you use this package for research purposes, please cite the following:

status

@article{moss2021pomdpstresstesting,
  title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
  author = {Robert J. Moss},
  journal = {Journal of Open Source Software},
  year = {2021},
  volume = {6},
  number = {60},
  pages = {2749},
  doi = {10.21105/joss.02749}
}

Interface

To stress test a new system, the user has to define the GrayBox and BlackBox interface outlined in src/GrayBox.jl and src/BlackBox.jl.

GrayBox Interface

The GrayBox simulator and environment interface includes:

  • GrayBox.Simulation type to hold simulation variables
  • GrayBox.environment(sim::Simulation) to return the collection of environment distributions
  • GrayBox.transition!(sim::Simulation) to transition the simulator, returning the log-likelihood

BlackBox Interface

The BlackBox system interface includes:

  • BlackBox.initialize!(sim::Simulation) to initialize/reset the system under test
  • BlackBox.evaluate!(sim::Simulation) to evaluate/execute the system under test
  • BlackBox.distance(sim::Simulation) to return how close we are to an event
  • BlackBox.isevent(sim::Simulation) to indicate if a failure event occurred
  • BlackBox.isterminal(sim::Simulation) to indicate the simulation is in a terminal state

Functions ending with ! may modify the Simulation object in place.

Solvers

Several solvers are implemented.

Reinforcement learning solvers

Deep reinforcement learning solvers1

Stochastic optimization solvers

Baseline solvers

Example

Example Notebook

An example implementation of the AST interface is provided for the Walk1D problem:

Installation

Install the POMDPStressTesting.jl package via:

] add POMDPStressTesting

Testing

To run the test suite, you can use the Julia package manager.

] test POMDPStressTesting

Contributing

We welcome contributions! Please fork the repository and submit a new Pull Request.


Package maintained by Robert Moss: mossr@cs.stanford.edu

1 TRPO and PPO thanks to Shreyas Kowshik's initial implementation.