ProgressTMap

ProgressTMap provides a map function that will spread the workload evenly across all available threads. Via ProgressLogging.jl, progress events are emitted as each thread completes its task. These events can be displayed in the VSCode status bar via the Julia VSCode extension or as a progress bar in the Julia REPL via the TerminalLoggers.jl package.

To install ProgressTMap, you can use the Julia package manager. Open the Julia REPL and run the following:

using Pkg
Pkg.add("ProgressTMap.jl")

Say we have a costly function f that we want to compute using all available threads on our machine. Luckily evaluations of f are independent of each other, so we can don't have any requirement on the of calls of f. Using this package, we can multithread this computation as follows.

using ProgressTMap

f(t) = sleep(t)  # example slow function

ts = 1:100
mapped_result = progress_tmap(f, ts; name="Sleeping")

The function divides the input into n even chunks, where n is the number of threads the julia worker is started with (Threads.nthreads()). This is done with the ChunkSplitters.jl package. ¹ For each chunk, a new Task is spawned, on which the function is broadcasted over the chunk. The results of each task are then combined at the end and returned.