This post introduces Parallel Programming using MPI in Python.
The library is mpi4py (MPI and python extensions of MPI), see here for its code repo on bitbucket.
Laurent Duchesne provides an excellent step-by-step guide for parallelizing your Python code using multiple processors and MPI. Craig Finch has a more practical example for high throughput MPI on GitHub. See here for more mpi4py examples from Craig Finch.
An example of TensorFlow using MPI can be found here.
References:
- A Python Introduction to Parallel Programming with MPI
- MPI for Python (mpi4py)
- Python Guide – MPI
- Setup python (including mpi4py) without admin rights
- Python for High Performance
While Python is a scripting language, it has plenty of facilities for high performance computing. This article covers some of its features and libraries that are particularly helpful when moving scientific code to a large cluster resource. It also includes specific recipes for compilation and execution on the TACC clusters.
