Julian hybrid constraint programming solver enhanced by a reinforcement learning driven search.
Author corail-research
28 Stars
Updated Last
2 Years Ago
Started In
May 2020


SeaPearl is a Constraint Programming solver that can use Reinforcement Learning agents as value-selection heuristics, using graphs as inputs for the agent's approximator. It is to be seen as a tool for researchers that gives the possibility to go above and beyond what has already been done with it.

The paper accompanying this solver can be found on the arXiv. If you use SeaPearl in your research, please cite our work.

The RL agents are defined using ReinforcementLearning.jl, their inputs are dealt with using GeometricFlux.jl and Flux.jl. The CP part, inspired from MiniCP, is focused on readability. The code is meant to be clear and modulable so that researchers could easily get access to CP data and use it as input for their ML model.


]add SeaPearl


Working examples can be found in SeaPearlZoo.

SeaPearl can be use either as a classic CP solver that uses predefined variable and value selection heuristics or as Reinforcement Learning driven CP solver that is capable of learning trought solving automatically generated instances of a given problem ( knapsack, tsptw, graphcoloring, nurse rostering ...).

SeaPearl as a classic CP solver :

To use SeaPearl as a classic CP solver, one needs to :

  1. declare a variable selection heuristic :
YourVariableSelectionHeuristic{TakeObjective} <: SeaPearl.AbstractVariableSelection{TakeObjective}
  1. declare a value selection heuristic :
BasicHeuristic <: ValueSelection
  1. create a Constraint Programming Model :
trailer = SeaPearl.Trailer()
model = SeaPearl.CPModel(trailer)

#create variable : 

#add constraints : 
push!(model.constraints, SeaPearl.AbstractConstraint(...))

#add optionnal objective function : 
model.objective = ObjectiveVar

SeaPearl as a RL-driven CP solver :

To use SeaPearl as a RL-driven CP solver, one needs to :

  1. declare a variable selection heuristic :
CustomVariableSelectionHeuristic{TakeObjective} <: SeaPearl.AbstractVariableSelection{TakeObjective}
  1. declare a value selection learnedheuristic :
LearnedHeuristic{SR<:AbstractStateRepresentation, R<:AbstractReward, A<:ActionOutput} <: ValueSelection
  1. optionnaly, declare some classic value selection heuristic for benchmarking purposes
basicHeuristic = SeaPearl.BasicHeuristic((x; cpmodel) -> your_function(...))
  1. define an agent :
agent = RL.Agent(
  1. optionnaly, declare a custom reward :
CustomReward <: SeaPearl.AbstractReward 
  1. optionnaly, declare a custom StateRepresentation ( instead of the Default tripartite-graph representation ) :
CustomStateRepresentation <: SeaPearl.AbstractStateRepresentation
  1. optionnaly, declare a custom featurization for the StateRepresentation :
CustomFeaturization <: SeaPearl.AbstractFeaturization
  1. create a generator for your given problem, that will create different instances of the specific problem used during the learning process.
CustomProblemGenerator <: AbstractModelGenerator
  1. set a number of training epochs, declare an evaluator, a Strategy, a metric for benchmarking
nb_epochs = 3000
CustomStrategy <: SearchStrategy #or use predefined one : SeaPearl.DFSearch
CustomEvaluator <: AbstractEvaluator #or use predefined one : SeaPearl.SameInstancesEvaluator(...)
function CustomMetricsFun
  1. launch the training :
bestsolutions, nodevisited,timeneeded, eval_nodevisited, eval_timeneeded = SeaPearl.train!(
valueSelectionArray=[learnedHeuristic, basicHeuristic], 


Contributing to SeaPearl

All PRs and issues are welcome. This repo contains and images to facilitate the understanding of the code. To contribute to Sealpearl, follow these steps:

  1. Fork this repository.
  2. Create a branch: git checkout -b <branch_name>.
  3. Make your changes and commit them: git commit -m '<commit_message>'
  4. Push to the original branch: git push origin <project_name>/<location>
  5. Create the pull request.

Alternatively see the GitHub documentation on creating a pull request.