This is a pure Julia implementation of the Apache Arrow data standard. This package provides Julia AbstractVector objects for
referencing data that conforms to the Arrow standard. This allows users to seamlessly interface Arrow formatted data with a great deal of existing Julia code.
Please see this document for a description of the Arrow memory layout.
The package can be installed by typing in the following in a Julia REPL:
julia> using Pkg; Pkg.add("Arrow")When developing on Arrow.jl it is recommended that you run the following to ensure that any changes to ArrowTypes.jl are immediately available to Arrow.jl without requiring a release:
julia --project -e 'using Pkg; Pkg.develop(path="src/ArrowTypes")'This implementation supports the 1.0 version of the specification, including support for:
- All primitive data types
- All nested data types
- Dictionary encodings and messages
- Extension types
- Streaming, file, record batch, and replacement and isdelta dictionary messages
It currently doesn't include support for:
- Tensors or sparse tensors
- Flight RPC
- C data interface
Third-party data formats:
- CSV, parquet and avro support via the existing CSV.jl, Parquet.jl and Avro.jl packages
- Other Tables.jl-compatible packages automatically supported (DataFrames.jl, JSONTables.jl, JuliaDB.jl, SQLite.jl, MySQL.jl, JDBC.jl, ODBC.jl, XLSX.jl, etc.)
- No current Julia packages support ORC
See the full documentation for details on reading and writing arrow data.