Flatten.jl

Flatten nested Julia objects to tuples, and reconstruct them later
Author rafaqz
Popularity
13 Stars
Updated Last
9 Months Ago
Started In
July 2018

Flatten

Build Status codecov.io Coverage Status

Flatten.jl converts data from arbitrary nested structs to tuples, using flatten(), reconstruct(), update!() and modify() functions. This facilitates building modular, composable structs while allowing access to solvers and optimisers that require flat lists of parameters. Importantly it's type-stable and very fast. It is not intended for use with arrays, as we do not know their length at compile time.

Flatten is also flexible. The types to return and ignore can be specified, and individual fields can be ignored using field-level traits like flattenable from FieldMetadata.jl. Method overrides can also be defined for custom types.

Type queries

Flatten allows a kind of querying to extract some types and ignore others, here using flatten:

julia> using Flatten

julia> struct Foo{A,B,C}
           a::A
           b::B
           c::C
       end

julia> struct Bar{X}
           x::X
       end

julia> obj = Foo(1, :two, Foo(Bar(3), 4.0, 5.0f0));

julia> use = Union{Int, Float32}; # Return Int and Float32 fields

julia> ignore = Bar;              # Dont return Bar or iterate over Bar fields

julia> flatten(obj, use, ignore)  # `flatten` all Int and Float32 except fields of Bar
(1, 5.0f0)

julia> modify(string, obj, Int)   # `modify`: convert all Int to String
Foo{String,Symbol,Foo{Bar{String},Float64,Float32}}("1", :two, Foo{Bar{String},Float64,Float32}(Bar{String}("3"), 4.0, 5.0f0))

The default type used is Real. These rules also apply in reconstruct, update! and modify.

Field removal

There are often cases where you want to ignore certain fields that have the same type as the fields you want to extract. Flatten.jl also FieldMetadata.jl to provide @flattenable macro and methods, allowing you to choose fields to include and remove from flattening. The default is true for all fields.

using Flatten
import Flatten: flattenable

@flattenable struct Bar{X,Y}
    x::X | true
    y::Y | false
end

flatten(Bar(1, 2))

# output
(1,)

Custom @metadata methods from FieldMetadata can be used, if they return a Bool. You can also use custom functions that follow the following form, returning a boolean:

f(::Type, ::Type{Var{:fieldname}}) = false

Metatdata flattening

Flatten also provides metaflatten() to flatten any FieldMetadata.jl metadata for the same fields flatten() returns. This can be useful for attaching information like descriptions or prior probability distributions to each field. Regular field data can also be collected with convenience versions of metaflatten: fieldnameflatten, parentflatten, fieldtypeflatten and parenttypeflatten functions provide lists of fieldnames and types that may be useful for building parameter display tables.

This package was started by Robin Deits (@rdeits), and its early development owes much to discussions and ideas from Jan Weidner (@jw3126) and Robin Deits. """