Simple Probabilistic Programs
Simple implementation of probabilistic programs for the Julia programming language with support for traces allocated on the stack.
Status
This package is very much work in progress and it is likely to change in the future. Comments, suggestions, and pull requests are welcome!
Quickstart
julia> using SimpleProbabilisticPrograms
julia> using Distributions: Beta, Bernoulli
julia> @probprog function beta_bernoulli_model(a, b, n)
bias ~ Beta(a, b) # sample a number between 0 and 1
coin = Bernoulli(bias) # create biased coin as Bernoulli distribution
throws ~ iid(coin, n) # throw the coin n times
return (; bias, throws) # return the trace of the program execution
end
beta_bernoulli_model (generic function with 1 method)
julia> model = beta_bernoulli_model(3, 4, 10)
ProbProg(...)
julia> trace = rand(model)
(bias = 0.15035879436791896, throws = Bool[0, 1, 0, 0, 0, 0, 0, 0, 0, 0])
julia> logpdf(model, trace)
-3.5451416292361504
Introduction and Motivation
Probabilistic programming is a great way of making the power of Bayesian statistics more accessible by drawing from research in programming languages.
A probabilistic program represents a distribution over the random choices that are made in a standard execution of the program. The collection of all such choices is called a trace of the probabilistic program.
While probabilistic programming systems like Gen.jl and Turing.jl (also Pyro and PyMC3 in python) focus on the implementation and integration of inference methods, the motivation of this package is to provide a minimal implementation of probabilistic programs with a simple API, full compositionality, and fast performance through type stability.
This can be helpful, for example, for applications that go beyond the use cases of established probabilistic programming systems, if such systems seem to be too complex for your use case, or if you want to learn/teach how probabilistic programming works.
In particular, if you want to write down a distribution that is just a bit more complicated than the ones provided by Distributions.jl
, this package might help you.
Implementing an extensive library of inference methods that work out of the box is not the goal of this package.
High-Level API
This package builds on the fact that all you need to get started with using probabilistic programs are 3 things:
- a way to define them
- a function for drawing a trace randomly, and
- a function for evaluating the log probability of a trace.
The specification of this high-level API is:
- Macro
@probprog function_definition
: Define functions that construct probabilistic programs. This macro transforms sample statements indicated by the binary operator~
into calls to a more low-levelsample
method (see the source code). Call the function generated by this macro to create a probabilistic program object (see example in Quickstart above). - Function
rand([rng,] prog)
: draw a random sample - Function
logpdf(prog, x)
: evaluate log probability (density) of a random sample. If the return values ofprog
are its execution traces, thenlogpdf(prog, trace)
works out of the box. Otherwise, the functionrecover_trace(prog, x)
needs to be implemented.