Treat Jupyter notebooks as visual scripts which can be executed from the command line or from a script. JupyterParameters creates and executes a new copy of the notebook with the parameters that have been passed and preserves the original.
My main use case for JupyterParameters is for batch processes that I also want to generate inline sophiscated plots. This essentially creates log files of my data analysis along with plots. Running Jupyter notebooks from the command line is already possible using
jupyter nbconvert --to notebook --execute mynotebook.ipynb
The issue with using nbconvert
in this fashion, is you can not pass arguments to the notebook.
Using jjnbparam
provided by JupyterParameters you are able to pass variables to a notebook.
using JupyterParameters
jjnbparam(["notebook_orig.ipynb","notebook_new.ipynb","--varname1","varvalue1","--varname2","varvalue2",...]
We can create an alias in .bashrc
as
alias jjnbparam='julia -e "using JupyterParameters; jjnbparam()"'
or add the following executable script (named jjnbparam
) to your PATH
.
julia --color=yes -e '
try
using JupyterParameters
catch
import Pkg; Pkg.add("JupyterParameters")
using JupyterParameters
end
jjnbparam()' "$@"
The command (from the shell) becomes
jjnbparam notebook_orig.ipynb notebook_new.ipynb --varname1 varvalue1 --varname2 varvalue2 ...
The command above creates and executes a new copy of the notebook with the parameters that have been passed and preserves the original. If one wants to overwrite the original then
jjnbparam notebook.ipynb notebook.ipynb --varname1 varvalue1 --varname2 varvalue2 ...
The target notebook needs to include a parameters
cell (this does not have to be the first cell):
To create a parameters cell simply edit the cell's metadata to include the following:
{
"tags": [
"parameters"
]
}
It is also helpful (for the user) to have a comment inside of the cell like so
# this is the parameters cell
foo = 10
bar = "hi"
In the cell above foo
and bar
are defined with what can be thought of as default values which will be used if the user does not replace them.
This project was inspired by papermill
The execution of the notebook can be customized with
jjnbparam refnote.ipynb outnote.ipynb \
--kernel_name julia-nodeps-1.1 \
--timeout -1 \
--var1 1234 \
--var2 "abcd"
where kernel_name
specifies the IJulia kernel and timeout defines the maximum time (in seconds) each notebook cell is allowed to run.
These values are passed under-the-hood to jupyter nbconvert
as traitlets.
If not passed the default values for jupyter nbconvert
are used (again, see traitlets).