ParallelUtilities.jl

Fast and easy parallel mapreduce on HPC clusters
Author jishnub
Popularity
31 Stars
Updated Last
8 Months Ago
Started In
July 2019

ParallelUtilities.jl

Build status codecov Stable Dev DOI

Parallel mapreduce and other helpful functions for HPC, meant primarily for embarassingly parallel operations that often require one to split up a list of tasks into subsections that may be processed on individual cores.

Installation

Install the package using

pkg> add ParallelUtilities
julia> using ParallelUtilities

Quick start

Just replace mapreduce by pmapreduce in your code and things should work the same.

julia> @everywhere f(x) = (sleep(1); x^2); # some expensive calculation

julia> nworkers()
2

julia> @time mapreduce(f, +, 1:10) # Serial
 10.021436 seconds (40 allocations: 1.250 KiB)
385

julia> @time pmapreduce(f, +, 1:10) # Parallel
  5.137051 seconds (863 allocations: 39.531 KiB)
385

Usage

See the documentation for examples and the API.