POMDPStressTesting.jl
Adaptive stress testing of black-box systems, implemented within the POMDPs.jl ecosystem.
See the documentation for more details.
Citation
If you use this package for research purposes, please cite the following:
@article{moss2021pomdpstresstesting,
title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
author = {Robert J. Moss},
journal = {Journal of Open Source Software},
year = {2021},
volume = {6},
number = {60},
pages = {2749},
doi = {10.21105/joss.02749}
}
Interface
To stress test a new system, the user has to define the GrayBox
and BlackBox
interface outlined in src/GrayBox.jl
and src/BlackBox.jl
.
GrayBox Interface
The GrayBox
simulator and environment interface includes:
GrayBox.Simulation
type to hold simulation variablesGrayBox.environment(sim::Simulation)
to return the collection of environment distributionsGrayBox.transition!(sim::Simulation)
to transition the simulator, returning the log-likelihood
BlackBox Interface
The BlackBox
system interface includes:
BlackBox.initialize!(sim::Simulation)
to initialize/reset the system under testBlackBox.evaluate!(sim::Simulation)
to evaluate/execute the system under testBlackBox.distance(sim::Simulation)
to return how close we are to an eventBlackBox.isevent(sim::Simulation)
to indicate if a failure event occurredBlackBox.isterminal(sim::Simulation)
to indicate the simulation is in a terminal state
Functions ending with !
may modify the Simulation
object in place.
Solvers
Several solvers are implemented.
Reinforcement learning solvers
Deep reinforcement learning solvers1
Stochastic optimization solvers
Baseline solvers
Example
An example implementation of the AST interface is provided for the Walk1D problem:
- Julia source:
test/Walk1D.jl
- Jupyter notebook:
Walk1D.ipynb
- Descriptive tutorial-style write-up:
walk1d.pdf
(created using TeX.jl)
Installation
Install RLInterface.jl
then the POMDPStressTesting.jl
package via:
using Pkg
pkg"add https://github.com/JuliaPOMDP/RLInterface.jl"
pkg"add https://github.com/sisl/POMDPStressTesting.jl"
Testing
To run the test suite, you can use the Julia package manager.
] test POMDPStressTesting
Contributing
We welcome contributions! Please fork the repository and submit a new Pull Request.
Package maintained by Robert Moss: mossr@cs.stanford.edu
1 TRPO and PPO thanks to Shreyas Kowshik's initial implementation.