Implementation of a drone surveillance problem^{1} with the POMDPs.jl.
^{1} M. Svoreňová, M. Chmelík, K. Leahy, H. F. Eniser, K. Chatterjee, I. Černá, C. Belta, " Temporal logic motion planning using POMDPs with parity objectives: case study paper", International Conference on Hybrid Systems: Computation and Control (HSCC), 2015.
using Pkg
Pkg.add(PackageSpec(url="https://github.com/JuliaPOMDP/DroneSurveillance.jl"))
using DroneSurveillance
using POMDPs
# import a solver from POMDPs.jl e.g. SARSOP
using SARSOP
# for visualization
using POMDPGifs
import Cairo
pomdp = DroneSurveillancePOMDP() # initialize the problem
solver = SARSOPSolver(precision=1e3) # configure the solver
policy = solve(solver, pomdp) # solve the problem
makegif(pomdp, policy, filename="out.gif")
A drone must survey two region (in green) while avoiding to fly over a ground agent. The drone has a limited field of view.

States: position of the drone and the agent in the grid world

Actions: Moving up, down, left, right, or hovering over the current cell.

Transition model: The drone moves deterministically to the desired cell depending on the action chosen. The agent follows a random policy, it can stay in the same cell or move to a neighboring cell with equal probability. In addition the agent cannot enter the area to survey.

Observation model: if the agent is in the field of view of the drone, the drone observes the position of the agent within its field of view with probability 1, otherwise it does not observe the agent (value
OUT
). 
Initial state: the drone starts in the bottom left corner, the agent is initialized outside the field of view of the drone.

Reward model: a naive reward model is implemented, the drone receives +1 for reaching the second area to survey, and 1 if it flies over the ground agent.
 constructor:
DroneSurveillancePOMDP(kwargs...)
 keyword arguments:
size::Tuple{Int64, Int64} = (5,5)
size of the grid worldregion_A::DSPos = [1, 1]
first region to survey, initial state of the quadregion_B::DSPos = [size[1], size[2]]
second region to surveyfov::Tuple{Int64, Int64} = (3, 3)
size of the field of view of the droneagent_policy::Symbol = :random
policy of the other agent, only random is implementedterminal_state::DSState = DSState([1, 1], [1, 1])
a sentinel state to encode terminal statesdiscount_factor::Float64 = 0.95
the discount factor
DSPos
represents a position in the grid as a static array of 2 integersDSState
represents the state of the environment, the fieldquad
represents the position of the drone and the fileagent
the position of the ground agent