Dispatcher is a tool for building and executing a computation graph given a series of dependent operations.
run! builds and runs a computation graph of
DispatchNodes represent units of computation that can be run.
The most common
Op, which represents a function call on some arguments.
Some of those arguments may exist when building the graph, and others may represent the results of other
Executor executes a whole
Executors are provided.
AsyncExecutor executes computations asynchronously using Julia
ParallelExecutor executes computations in parallel using all available Julia processes (by calling
Frequently Asked Questions
How is Dispatcher different from ComputeFramework/Dagger?
Dagger is built around distributing vectorized computations across large arrays. Dispatcher is built to deal with discrete, heterogeneous data using any Julia functions.
How is Dispatcher different from Arbiter?
Arbiter requires manually adding tasks and their dependencies and handling data passing. Dispatcher automatically identifies dependencies from user code and passes data efficiently between dependencies.
Isn't this just Dask?
The plan is to implement another
Executor and integrate with the
dask.distributed scheduler service to piggyback off of their great work.
How does Dispatcher handle passing data?
Dispatcher uses Julia
RemoteChannels to pass data between dispatched
For more information on how data transfer works with Julia's parallel tools see their documentation.