This post lists resources for using MPI with Python.
An application built with the hybrid model of parallel programming can run on a computer cluster using both OpenMP and Message Passing Interface (MPI), such that OpenMP is used for parallelism within a (multi-core) node while MPI is used for parallelism between nodes. There have also been efforts to run OpenMP on software distributed shared memory systems,[6] to translate OpenMP into MPI[7][8] and to extend OpenMP for non-shared memory systems.
cython.parallel is built on top of OpenMP (see Using Parallelism)
- MPI for Python(mpi4py)
- MPI -python guide
Please read Laurent Duchesne’s excellent step-by-step guide for parallelizing your Python code using multiple processors and MPI.
On our cluster, to run MPI Python programs, mpi4py has been compiled against OpenMPI 1.10.1 therefore we need to load that additional package:
module load python/3.4.3 mpi/openmpi/1.10.1-gcc
Create the the test MPI example file as described in Laurent’s guide above, using the same name mpi.py:
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
print("I am rank", rank, "of", size)
Create the SLURM submission script submit.sh:
#SBATCH -n 4 mpirun python mpi.py
You should get output similar to:
I am rank 3 of 4 I am rank 0 of 4 I am rank 1 of 4 I am rank 2 of 4
Craig Finch has a more practical example for high throughput MPI on GitHub.
- Multi-process parallelization on a single node, Parallelization across multiple nodes
- Parallel Python
References:
- https://en.wikipedia.org/wiki/OpenMP
- https://en.wikipedia.org/wiki/Message_Passing_Interface
- https://athena.brynmawr.edu/wiki/Tutorial_on_using_Python_and_MPI
- Parallel Processing with Python and OpenMP
- Using Parallelism
