DiscreteMarkovChains is a package that supports various functions relating to discrete Markov chains. In particular, it deals with discrete-time discrete-space time-homogenous finite Markov chains.

This library also deals with continuous Markov chains. Any function in the documentation that takes "some kind of Markov chain" as an argument can be a `DiscreteMarkovChain`

or a `ContinuousMarkovChain`

. Sadly there are very few examples for continuous Markov chains but they operate in the same way as discrete Markov chains.

`DiscreteMarkovChains`

should be up on Julia's package registry.

Simply type `] add DiscreteMarkovChains`

into the Julia REPL.

See the documentation hosted on GitHub Pages.

We want to find out if this chain is an absorbing chain.

```
using DiscreteMarkovChains
transition_matrix = [
0.0 1.0 0.0;
0.5 0.0 0.5;
0.0 1.0 0.0;
]
chain = DiscreteMarkovChain(transition_matrix)
is_absorbing(chain)
# output
false
```

Let's try find the communication classes, see if they are recurrent and what their periodicity is.

```
periodicities(chain)
# output
([[1, 2, 3]], Any[true], Any[2])
```

This means that we have one communication class with 3 recurrent states. Their periodicity is 2.

Since we have a single communication class, we can calculate the mean recurrence times.

```
mean_recurrence_time(chain)
# output
3-element Array{Float64,1}:
4.0
2.0
4.0
```

So the first and third states take an average of 4 time steps to return to itself. The second state takes an average of 2 steps to return to itself.

There is support for continuous Markov chains as well.

```
generator = [
-3 1 2;
0 -1 1;
1 1 -2;
]
chain = ContinuousMarkovChain(generator)
communication_classes(chain)
# output
([[1, 2, 3]], Any[true])
```

So we have one communication class that is recurrent.

Calculate the stationary distribution of the chain.

```
stationary_distribution(chain)
# output
3-element Array{Float64,1}:
0.125
0.5
0.375
```

Calculate the mean first passage time of the chain.

```
round.(mean_first_passage_time(chain), digits=2)
# output
3×3 Array{Float64,2}:
0.0 1.0 0.67
3.0 0.0 1.0
2.0 1.0 0.0
```

- Chris du Plessis

MIT