A light wrapper around the DFO-LS (Derivative-Free Optimizer for Least-Squares Minimization) Python package written by the Numerical Algorithms Group at Oxford University. See here for the paper and the github repository.

- Installation
- Usage
- Constraints and Stochastic Objectives
- Advanced Usage

**Note:** This package is GPL3 licensed, to comply with the underlying Python.

Simply run

```
] add DFOLS
```

**Note:** The build script assumes that `$(PyCall.pyprogramname) pip`

is a valid command. This is automatically true on Windows and macOS, but needs to be verified on Linux (i.e., make sure it isn't `pip3`

, or `python-pip3`

, or something). You can do this by aliasing `pip=pip3`

or setting up a symbolic link `pip -> pip3`

.

We define a type `DFOLSResults`

to store the solver output.

```
struct DFOLSResults{TI, TF}
x::Array{TF, 1}
resid::Array{TF, 1}
f::TF
jacobian::Union{Nothing, Matrix{TF}} # jacobian is nothing if convergence is immediate
nf::TI
nx::TI # differs from nf if sample averaging is used
nruns::TI # > 1 if multiple restarts
flag::TI
msg::String
EXIT_SUCCESS::TI
EXIT_MAXFUN_WARNING::TI
EXIT_SLOW_WARNING::TI
EXIT_FALSE_SUCCESS_WARNING::TI
EXIT_INPUT_ERROR::TI
EXIT_TR_INCREASE_ERROR::TI
EXIT_LINALG_ERROR::TI
end
```

And we define a set of convenience functions to interact with it

```
converged, optimizer, optimum, residuals, jacobian, nf, nruns, nx, flag, msg
```

You can run the solver by calling the `solve`

function, as below

```
rosenbrock = x -> [10. * (x[2]-x[1]^2), 1. - x[1]]
sol = solve(rosenbrock, [-1.2, 1.])
```

Options for `solve`

include

```
function solve(objfun, x0::Array{TF, 1};
bounds = nothing,
npt = nothing,
rhobeg = nothing,
rhoend = 1e-8,
maxfun = nothing,
nsamples = nothing,
user_params = nothing, # see https://numericalalgorithmsgroup.github.io/dfols/build/html/advanced.html
objfun_has_noise = false,
scaling_within_bounds = false) where {TF <: AbstractFloat}
```

You can impose constraints on the solution space

```
solve(rosenbrock, x0, bounds = ([-5., -5.], [5., 5.])) # two-sided box
solve(rosenbrock, x0, bounds = ([-5., -5.], nothing)) # one-sided constraint
```

And note that the objective is stochastic

```
σ = 0.01
μ = 1.
rosenbrock_noisy = x -> rosenbrock(x) .* (μ .+ σ*randn(2))
solve(rosenbrock_noisy, x0, objfun_has_noise=true)
```

**Note:** The solver will determine the stochasticity of the objective only by examining the `objfun_has_noise`

flag, and not by looking at the actual function supplied.

The `user_params`

should be a Julia dict (see here for valid key, value pairs). For example:

```
solve(rosenbrock, x0, user_params = Dict("init.random_initial_directions" => false,
"model.abs_tol" => 1e-20,
"noise.quit_on_noise_level" => false))
```