Reverse mode differentiation for pre-defined functions.

Using reverse mode differentiation is very simple, just call,

`reversediff(function, args...)`

for instance, to differentiate `f`

, defined by,

```
f(a, b) = begin
c = a*b
dot(c, c) + dot(c, b)
end
```

at, `[1. 2; 3 4], [1., 2]`

, call,

`reversediff(f, [1. 2; 3 4], [1., 2])`

which returns a tuple containing the return value, and the differential of the return value with respect to every argument,

```
(173.0,(
2x2 Array{Float64,2}:
11.0 22.0
24.0 48.0,
[88.0,129.0])
```

It is good practice, at this early stage, to test that `reversediff`

produces the correct results for your problem.
To do such a test on your function, simply call,

`testdiff(g, args...)`

for instance,

`testdiff(f, [1. 2; 3 4], [1., 2])`

An error will be generated if finite difference and our method give different results.

Of course, things aren't always quite that simple.
There are three common sources of bugs.
First, `reversediff`

differentiates with respect to every argument, so every argument should have type `Float64`

, or `Array{Float64}`

. Notice the care taken in the example to make sure that we don't get an array of `Int`

. Second, the type signature of the function you're trying to differentiate may be too constrained - the function needs to let values of type `Call`

propagate through until they reach known functions. Notice, `f`

, defined above had no type constraints. Third, you may be trying to use a function whose differential is not yet defined. You can provide define new definitions using the macro `@d1`

, for one argument functions, or `@d2`

, for two argument functions, for instance, to redefine the differentials for `*`

, we might use,

`ReverseDiff.@d2(*, d*y', x'*d)`

Where `x`

is the first argument to the function, `y`

is the second argument, and `d`

is the differential of the objective with respect to the result of the function call. Note that you can also annotate the types of `x`

and `y`

, using,

`ReverseDiff.@d2(*, d*y', x'*d, AbstractArray, AbstractArray)`