DeepReshapes.jl

Reshape arbitrarily nested structures of Tuples and Arrays in Julia
Popularity
0 Stars
Updated Last
8 Years Ago
Started In
November 2014

DeepReshapes

Extends reshape to arbitrarily nested structures of `Tuple`s and `Array`s, both in source and target. Also provides a deep `flatten` function that transforms these structures into a flat `Vector`.

As I am pretty new to Julia, before I consider registering this package, I would like a code review to know whether I am actually doing this "right". Please just have a look, and if you think this is useful and ready, open an issue or something like that.

Note that this only works on Julia 0.4 right now.

What?

Provides a `deep_reshape` function that can transform the structure of data:

``````A = [1 2; 3 4; 5 6]
b = [1, 2, 3, 4]
deep_reshape((A, b), (2, 5))
# => [1 5 4 1 3;
#     3 2 6 2 4]

deep_reshape([1:25], ((3, 3), (4, 4)))
# => ([ 1  4  7;
#       2  5  8;
#       3  6  9],
#     [10 14 18 22;
#      11 15 19 23;
#      12 16 20 24;
#      13 17 21 25])

α, β, c = deep_reshape([1.23, 2.34, 3, 4, 5], (Float64, Float64, (Int, 3)))
# => (1.23,2.34,[3,4,5])
``````

This works on all (potentially nested) structures of `Tuple`s and `Array`s, as long as the actual scalar values contained within are `Number`s (for now).

Why?

Say you want to optimize a non-linear function. Many optimization frameworks (NLopt, Optim) require you to supply cost and gradient functions and expect them to operate on plain `Vector{Float64}`s for that purpose. However, some algorithms are expressed more naturally in terms of more structured data.

Consider for example the popular [backpropagation algorithm] (http://ufldl.stanford.edu/wiki/index.php/Backpropagation_Algorithm) for training neural networks. The outline of the gradient calculation might look like this:

``````function cost_and_gradient!(
W::Vector{Matrix{Float64}},  # weight matrices for each neuron layer
b::Vector{Vector{Float64}},  # bias vectors for each neuron layer
)
# ...do feedforward and backpropagation, obtain some intermediate results
# ...calculate gradients and fill the result vectors ∇W and ∇b
# ...calculate and return the cost
end
``````

For optimization, we cannot use this function directly, because the optimization package expects it to work on plain number vectors:

``````using NLopt

W, b = initialize_parameters()
# ...we need to flatten W, b to number vector θ

optimization = Opt(:LD_LBFGS, length(θ))
min_objective!(optimization, cost_and_gradient!) # <- we need to define this
result = optimize(optimization, θ)
``````

Flattening the initial parameters is easy with `flatten`:

``````using DeepReshapes

θ = flatten(Float64, W, b)
``````

As for `cost_and_gradient!`, we can simply wrap the original calculation with `deep_reshape`s:

``````function cost_and_gradient!(θ::Vector{Float64}, ∇θ::Vector{Float64})
W, b = deep_reshape(θ, s) # <- s is a specification of the original structure
# which can be obtained by calling describe on the
# initial parameters before flattening them.

# ...do the original calculation
∇θ[:] = flatten(Float64, ∇W, ∇b)
# ... calculate and return the cost
end
``````

How?

A deep reshape consists of two independent processes: a producer that recursively walks the input to produce scalar values, and a consumer that consumes these values and puts them into a new object according to a structure specification:

``````result = deep_reshape(input, specification)
``````

Source

The input can be any object, but by default, the producer only descends into Arrays and Tuples and considers anything else to be a scalar:

``````deep_reshape([1:4], (2, 2)) # => Any[1 3; 2 4]
deep_reshape(1:4, (2, 2))   # => Any[1:4 (); () ()]
``````

What objects to descend into can be overridden through the optional `Deep` argument:

``````deep_reshape(1:4, (2, 2), Deep = Range)
``````

Any input of type `Deep` will be considered iterable and all contained values will be produced. Any other input will be considered scalar and produced directly, without further recursion.

Target

The produced scalars will then be assigned into the objects under construction according to the specification, which is of the following format:

• An empty tuple `()` describes `Any` value.
• A `Type` describes a value of that type.
• A tuple of `(Integer...)` dimensions describes an `Any[]` with these dimensions.
• A tuple `(Type, Integer...)` of a type and dimensions describes an Array with that element type and these dimensions.
• Any other `Tuple` recursively describes a tuple, where each contained value describes an entry of the result.
• An `Array` recursively describes an array in the same way.

For simple inputs (recursively) consisting only of `Tuple`s, `Array`s and scalars, `describe()` returns the corresponding specification:

``````s = describe(([1:10], [1 2; 3 4])) # => ((Int, 10), (Int, 2, 2))
``````

These can be used directly as `deep_reshape` specifications:

``````nested = deep_reshape([1:14], s)
# => ([1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
#     [11 13;
#      12 14])
``````

flatten and pack

There is also a convenience function `flatten` that can recursively flattens the input to a `Vector`, optionally with a fixed target type that the scalars are to be converted to:

``````flatten(nested)
# => Any[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]

flatten(Int, nested)
# => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
``````

The very similar `pack` function returns both the flattened `Vector` and the original structure as defined by `describe`. This can be used to later reverse the flattening:

``````flattened, s = pack(Int, ([1:10], [1 2; 3 4]))
# => ([1,2,3,4,5,6,7,8,9,10,1,3,2,4],(((Int,10),(Int,2,2)),))

deep_reshape(flattened, s)
# => ([1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
#     [11 13;
#      12 14])
``````