This is small package aimed to simplify process of data downloading and postprocessing, without intermediate files storing. Additionally
UrlDownload.jl provides progress bar for big files with long download time.
UrlDownload.jl has integrated support for the processing of various data formats, such as CSV, JSON, various image formats. Everything that is not supported by default, can be easily extended with custom parsers. Also, package automatically decompress and process various compressed data with the help of TranscodingStreams.jl. And it can process files located on
http resources and local.
UrlDownload either do
using Pkg Pkg.add("UrlDownload")
or switch to
Pkg mode with
] and issue
pkg> add UrlDownload
Note: this package uses many different packages for data processing, which should be installed separately. So, if you receive message like
ERROR: ArgumentError: Package CSV not found in current path, install CSV.jl manually and error will go away. No additional work is needed, since
UrlDownload.jl import necessary packages on it's own.
Usage is quite simple
using UrlDownload using DataFrames url = "https://raw.githubusercontent.com/Arkoniak/UrlDownload.jl/master/data/ext.csv" df = urldownload(url) |> DataFrame # 2×2 DataFrame # │ Row │ x │ y │ # │ │ Int64 │ Int64 │ # ├─────┼───────┼───────┤ # │ 1 │ 1 │ 2 │ # │ 2 │ 3 │ 4 │
More details and examples can be found in the documentation.