HealpixMPI.jl: an MPI-parallel implementation of the Healpix tessellation scheme in Julia
Welcome to HealpixMPI.jl, an MPI-parallel implementation of the main functionalities of HEALPix spherical tessellation scheme, entirely coded in Julia.
This package constitutes a natural extension of the package Healpix.jl, providing an MPI integration of its main functionalities, allowing for simultaneous shared-memory (multithreading) and distributed-memory (MPI) parallelization leading to high performance sperical harmonic transforms.
Read the full documentation for further details.
Installation
From the Julia REPL, run
import Pkg
Pkg.add("HealpixMPI")
Usage Example
The example shows the necessary steps to set up and perform an MPI-parallel alm2map
SHT with HealpixMPI.jl.
Set up
We set up the necessary MPI communication and initialize Healpix.jl structures:
using MPI
using Random
using Healpix
using HealpixMPI
#MPI set-up
MPI.Init()
comm = MPI.COMM_WORLD
crank = MPI.Comm_rank(comm)
csize = MPI.Comm_size(comm)
root = 0
#initialize Healpix structures
NSIDE = 64
lmax = 3*NSIDE - 1
if crank == root
h_map = HealpixMap{Float64, RingOrder}(NSIDE) #empty map
h_alm = Alm(lmax, lmax, randn(ComplexF64, numberOfAlms(lmax))) #random alm
else
h_map = nothing
h_alm = nothing
end
Distribution
The distributed HealpixMPI.jl data types are filled through an overload of MPI.Scatter!
:
#initialize empty HealpixMPI structures
d_map = DMap{RR}(comm)
d_alm = DAlm{RR}(comm)
#fill them
MPI.Scatter!(h_map, d_map)
MPI.Scatter!(h_alm, d_alm)
SHT
We perform the SHT through an overload of Healpix.alm2map
and, if needed, we Gather!
the result in a HealpixMap
:
alm2map!(d_alm, d_map; nthreads = 16)
MPI.Gather!(d_map, h_map)
This allows the user to adjust at run time the number of threads to use, typically to be set to the number of cores of your machine.
Run
In order to exploit MPI parallelization run the code through mpirun
or mpiexec
as
$ mpiexec -n {Ntask} julia {your_script.jl}
To run a code on multiple nodes, specify a machine file machines.txt
as
$ mpiexec -machinefile machines.txt julia {your_script.jl}