BayesianExperiments.jl provides a toolbox for running various types of Bayesian AB testing experiments.
Current features include:
- Hypothesis testing with Bayes factor. Support the effect size model with Normal distribution prior and JZS prior.
- Bayesian decision making with conjugate prior models. Support expected loss and probability to beat all as the stopping rule.
- Flexible experiment design for both fixed horizon experiments and sequential test experiment.
- Efficient simulation tools to support power analysis and sensitivity analysis.
For usage instructions and tutorials, see documentation.
For detailed discussions on many topics in the field, see the Jupyter notebooks in the examples folder:
- Sequential Experiment with Two Models
- Type S Error in Fixed Horizon and Sequential Test Experiment
- Bayes Factor Experiment with Optional Stopping
Or you can go to binder to directly play with the Jupyter notebooks.
Open source projects in R related to our project: