The Pardiso.jl package provides an interface for using Panua Pardiso, it's predecessors from pardiso-project.org, and Intel MKL PARDISO from the Julia language.
You cannot use Pardiso.jl without either having a valid license for Panua Pardiso or
having the MKL library installed. This
package is available free of charge and in no way replaces or alters any
functionality of the linked libraries.
The package itself is installed with Pkg.add("Pardiso") but you also need to
follow the installation instructions below to install a working PARDISO
library.
By default, when adding "Pardiso.jl" to the active environmnent, Julia will automatically install a suitable MKL for your platform by loading MKL_jll.jl.
Note that if you use a mac you will need to pin MKL_jll to version 2023.
If you instead use a self installed MKL, follow these instructions:
- Set the
MKLROOTenvironment variable. See the MKL set environment variables manual for a thorough guide how to set this variable correctly, typically done by executing something likesource /opt/intel/oneapi/setvars.sh intel64or running"C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\bin\mklvars.bat" intel64 - Run
Pkg.build("Pardiso", verbose=true) - Eventually, run
Pardiso.show_build_log()to see the build log for additional information. - Note that the
MKLROOTenvironment variable must be set, andLD_LIBRARY_PATHmust contain$MKLROOT/libwhenever using the library this way.
PARDISO from panua.ch ("PanuaPardiso", formerly "ProjectPardiso")
- Unzip the download file
panua-pardiso-yyyymmdd-os.zipto some folder and set the environment variableJULIA_PARDISOto thelibsubdirectory of this folder. For example, create an entryENV["JULIA_PARDISO"] = "/Users/Someone/panua-pardiso-yyyymmdd-os/lib"in.julia/config/startup.jl. If you have a valid license for the predecessor from pardiso-project.org, put the PARDISO library to a subdirectory denoted byENV["JULIA_PARDISO"]and evenutally rename it tolibpardiso.so. - Perform the platform specific steps described below
- Run
Pkg.build("Pardiso", verbose=true) - Eventually, run
Pardiso.show_build_log()to see the build log for additional information.
Note: In the past, weird errors and problems with MKL Pardiso had been observed when PanuaPardiso is enabled
(likely because some library that is needed by PanauaPardiso was problematic with MKL).
In that case, if you want to use MKL Pardiso it is better to just disable PanuaPardiso by not setting
the environment variable JULIA_PARDISO (and rerunning Pkg.build("Pardiso")).
- Make sure that the version of
gfortrancorresponding to the pardiso library is installed. - Make sure OpenMP is installed.
- Install a (fast) installation of a BLAS and LAPACK (this should preferably be single threaded since PARDISO handles threading itself), using for example OpenBLAS
gfortran and OpenMP usually come with recent version of gcc/gfortran. On Linux, Panua Pardiso
looks for libraries libgfortran.so and libgomp.so . They may be named differently on your system.
In this situation you may try to create links to them with names known to
Pardiso.jl (bash; pathnames serve as examples here):
$ mkdir $HOME/extralibs
$ ln -s /usr/lib64/libgomp.so.1 $HOME/extralibs/libgomp.so
$ ln -s /usr/lib64/libgfortran.so.5 $HOME/extralibs/libgfortran.so
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/extralibs/
This section will explain how to solve equations using Pardiso.jl with the default settings of the library. For more advanced users there is a section further down.
A PardisoSolver is created with PardisoSolver() for solving with PanuaPardiso or MKLPardisoSolver() for solving with MKL PARDISO. This object will hold the settings of the solver and will be passed into the solve functions. In the following sections an instance of a PardisoSolver or an MKLPardisoSolver() will be referred to as ps.
Solving equations is done with the solve and solve! functions. They have the following signatures:
solve(ps, A, B)solvesAX=Band returnsXsolve!(ps, X, A, B)solvesAX=Band stores it inX
The symbols :T or :C can be added as an extra argument to solve the transposed or the conjugate transposed system of equations, respectively.
Here is an example of solving a system of real equations with two right-hand sides:
ps = PardisoSolver()
A = sparse(rand(10, 10))
B = rand(10, 2)
X = zeros(10, 2)
solve!(ps, X, A, B)which happened to give the result
julia> X
10x2 Array{Float64,2}:
-0.487361 -0.715372
-0.644219 -3.38342
0.465575 4.4838
1.14448 -0.103854
2.00892 -7.04965
0.870507 1.7014
0.590723 -5.74338
-0.843841 -0.903796
-0.279381 7.24754
-1.17295 8.47922Given a partitioned matrix M = [A B; C D], the Schur complement of A in M is S = D-CA⁻¹B.
This can be found with the function schur_complement with the following signatures:
schur_complement(ps, M, n)returns Schur complement of submatrixAinM, wherenis the size of submatrixD(and therefore also of Schur complement)schur_complement(ps, M, x)returns Schur complement of submatrixAinM, where submatrixDis defined by nonzero rows ofSparseVectororSparseMatrixx.
The symbols :T or :C can be added as an extra argument to solve the transposed or the conjugate transposed system of equations, respectively.
Here is an example of finding the Schur complement:
ps = PardisoSolver()
m = 100; n = 5; p = .5; T = Float64
rng = MersenneTwister(1234);
A = I + sprand(rng,T,m,m,p)
A⁻¹ = inv(Matrix(A))
B = sprand(rng,T,m,n,p)
C = sprand(rng,T,n,m,p)
D = sprand(rng,T,n,n,p)
M = [A B; C D]
S = schur_complement(ps,M,n)which gives
julia> S
5×5 Array{Float64,2}:
-0.121404 1.49473 -1.25965 7.40326 0.571538
-19.4928 -7.71151 12.9496 -7.13646 -20.4194
9.88029 3.35502 -7.2346 1.70651 13.9759
-9.06094 -5.86454 7.44917 -2.54985 -9.17327
-33.7006 -17.8323 20.2588 -19.5863 -37.6132We can check the validity by comparing to explicity form:
julia> norm(D - C*A⁻¹*B - S)
5.033075778861378e-13At present there seems to be an instability in the Schur complement computation for complex matrices.
The number of threads to use is set in different ways for MKL PARDISO and PanuaPardiso.
set_nprocs!(ps, i) # Sets the number of threads to use
get_nprocs(ps) # Gets the number of threads being usedThe number of threads are set at the creation of the PardisoSolver by looking for the environment variable OMP_NUM_THREADS. This can be done in Julia with for example ENV["OMP_NUM_THREADS"] = 2. Note: OMP_NUM_THREADS must be set before Pardiso is loaded and can not be changed during runtime.
The number of threads used by a PardisoSolver can be retrieved with get_nprocs(ps)
This section discusses some more advanced usage of Pardiso.jl.
For terminology in this section please refer to the PanuaPardiso manual and the oneMKL PARDISO manual.
After using functionality in this section, calls should no longer be made to the solve functions but instead directly to the function
pardiso(ps, X, A, B)This will ensure that the properties you set will not be overwritten.
If you want, you can use get_matrix(ps, A, T) to return a matrix that is suitable to use with pardiso depending on the matrix type that ps has set. The parameter T is a symbol representing if you will solve the normal, transposed or conjugated system. These are represented by :N, :T, :C) respectively.
For ease of use, Pardiso.jl provides enums for most options. These are not exported so has to either be explicitly imported or qualified with the module name first. It is possible to both use the enum as an input key to the options or the corresponding integer as given in the manuals.
The matrix type can be explicitly set with set_matrixtype!(ps, key) where the key has the following meaning:
| enum | integer | Matrix type |
|---|---|---|
| REAL_SYM | 1 | real and structurally symmetric |
| REAL_SYM_POSDEF | 2 | real and symmetric positive definite |
| REAL_SYM_INDEF | -2 | real and symmetric indefinite |
| COMPLEX_STRUCT_SYM | 3 | complex and structurally symmetric |
| COMPLEX_HERM_POSDEF | 4 | complex and Hermitian positive definite |
| COMPLEX_HERM_INDEF | -4 | complex and Hermitian indefinite |
| COMPLEX_SYM | 6 | complex and symmetric |
| REAL_NONSYM | 11 | real and nonsymmetric |
| COMPLEX_NONSYM | 13 | complex and nonsymmetric |
The matrix type for a solver can be retrieved with get_matrixtype(ps).
PanuatPardiso supports direct and iterative solvers. The solver is set with set_solver!(ps, key) where the key has the following meaning:
| enum | integer | Solver |
|---|---|---|
| DIRECT_SOLVER | 0 | sparse direct solver |
| ITERATIVE_SOLVER | 1 | multi-recursive iterative solver |
Depending on the phase calls to solve (and pardiso which is mentioned later) does different things. The phase is set with set_phase!(ps, key) where key has the meaning:
| enum | integer | Solver Execution Steps |
|---|---|---|
| ANALYSIS | 11 | Analysis |
| ANALYSIS_NUM_FACT | 12 | Analysis, numerical factorization |
| ANALYSIS_NUM_FACT_SOLVE_REFINE | 13 | Analysis, numerical factorization, solve, iterative refinement |
| NUM_FACT | 22 | Numerical factorization |
| SELECTED_INVERSION | -22 | Selected Inversion |
| NUM_FACT_SOLVE_REFINE | 23 | Numerical factorization, solve, iterative refinement |
| SOLVE_ITERATIVE_REFINE | 33 | Solve, iterative refinement |
| SOLVE_ITERATIVE_REFINE_ONLY_FORWARD | 331 | MKL only, like phase=33, but only forward substitution |
| SOLVE_ITERATIVE_REFINE_ONLY_DIAG | 332 | MKL only, like phase=33, but only diagonal substitution (if available) |
| SOLVE_ITERATIVE_REFINE_ONLY_BACKWARD | 333 | MKL only, like phase=33, but only backward substitution |
| RELEASE_LU_MNUM | 0 | Release internal memory for L and U matrix number MNUM |
| RELEASE_ALL | -1 | Release all internal memory for all matrices |
Advanced users likely want to explicitly set and retrieve the IPARM and DPARM (PanuaPardiso only) parameters.
This can be done with the getters and setters:
get_iparm(ps, i) # Gets IPARM[i]
get_iparms(ps) # Gets IPARM
set_iparm!(ps, i, v) # Sets IPARM[i] = v
# PanuaPardiso only
get_dparm(ps, i) # Gets DPARM[i]
get_dparms(ps) # Gets DPARM
set_dparm!(ps, i, v) # Sets DPARM[i] = vTo set the default values of the IPARM and DPARM call pardisoinit(ps). The default values depend on what solver and matrix type is set.
It is possible for Pardiso to print out timings and statistics when solving. This is done by set_msglvl!(ps, key) where key has the meaning:
| enum | integer | Solver |
|---|---|---|
| MESSAGE_LEVEL_OFF | 0 | no statistics printed |
| MESSAGE_LEVEL_ON | 1 | statistics printed |
PanuaPardiso comes with a few matrix and vector checkers to check the consistency and integrity of the input data. These can be called with the functions:
printstats(ps, A, B)
checkmatrix(ps, A)
checkvec(ps, B)In MKL PARDISO this is instead done by setting IPARM[27] to 1 before calling pardiso.
These are set and retrieved with the functions
set_mnum!(ps, i)
get_mnum(ps)
set_maxfct!(ps, i)
get_maxfct(ps)
get_perm(ps)
set_perm!(ps, perm) # Perm is a Vector{Int}The pardiso(ps,...) syntax can be used to compute the Schur compelement (as described below). The answer can be retrieved with pardisogetschur(ps).
To use the low-level API to compute the Schur complement:
- use custom IPARMS (
set_iparm!(ps,1,1)), set the Schur complement block size ton(set_iparm!(ps,38,n)), and set the phase to analyze & factorize (set_phase!(ps,12)). - compute the Schur complement by calling
pardiso(ps,X,M,X), whereBis a dummy vector withlength(X)=size(M,1)that shares element type withM. - retrieve with
pardisogetschur(ps)
- Julia uses CSC sparse matrices while PARDISO expects a CSR matrix. These can be seen as transposes of each other so to solve
AX = Bthe transpose flag (IPARAM[12]) should be set to 1. - For symmetric matrices, PARDISO needs to have the diagonal stored in the sparse structure even if the diagonal element happens to be 0. The manual recommends adding an
epsto the diagonal when you suspect you might have 0 values diagonal elements that are not stored in the sparse structure. - Unless
IPARM[1] = 1, all values inIPARMwill be ignored and default values are used. - When solving a symmetric matrix, Pardiso expects only the upper triangular part. Since Julia has CSC matrices this means you should pass in
tril(A)to thepardisofunction. Usecheckmatrixto see that you managed to get the matrix in a valid format.
If you have suggestions or idea of improving this package, please file an issue or even better, create a PR!