MCMCDebugging.jl: debugging utilities for MCMC samplers
This package implements a few utilities for debugging MCMC samplers, which includes
- Geweke test
- See the references [1,2] or this blog for details
- Central limit theorem test
See the notebook for an example.
The example notebook covers most of the usages. Some details on the model definition via DynamicPPL is explained below.
Defining test models via DynamicPPL.jl
MCMCDebugging.jl allows using DynamicPPL.jl to define test models. In the example notebook, the test model is defined as
@model function BetaBinomial(θ=missing, x=missing) θ ~ Beta(2, 3) x ~ Binomial(3, θ) return θ, x end
There are a few requirements from MCMCDebugging.jl to use the defined model.
- The model should take
xas inputs (in order) and optionally being
- So that the model can be used to generate the marginal sampler as e.g.
BetaBinomial()and conditional sampler as e.g.
- The model should return the parameter
θand the data
xas a tuple.
With these two points, MCMCDebugging.jl can generate several functions used by lower-level APIs.
xas a tuple
logjoint(θ, x): computing the log-joint probability of
1 and 2 are used to perform the Geweke test and 3 is used to make the Q-Q plot.
Defining the Geweke test
cfg = GewekeTest(n_samples::Int)
n_samples is the number of samples used for testing.
Performing the Geweke test
res = perform(cfg::GewekeTest, rand_marginal, rand_x_given, rand_θ_given; g=nothing, progress=true)
xas a tuple
g(θ, x)is the test function
Making the Q-Q plot
logjoint(θ, x)computes the log-joint probability of
In case models are defined by DynamicPPL.jl, you can use
plot(res, BetaBinomial()). Note we have to pass an instantiated model (i.e. BetaBinomial()) here, for now, to make Julia correctly dispatch the plot recipe.
 Geweke J. Getting it right: Joint distribution tests of posterior simulators. Journal of the American Statistical Association. 2004 Sep 1;99(467):799-804.
 Grosse RB, Duvenaud DK. Testing mcmc code. arXiv preprint arXiv:1412.5218. 2014 Dec 16.