This post introduces Parallel Programming using MPI in Python.
Laurent Duchesne provides an excellent step-by-step guide for parallelizing your Python code using multiple processors and MPI. Craig Finch has a more practical example for high throughput MPI on GitHub. See here for more mpi4py examples from Craig Finch.
An example of TensorFlow using MPI can be found here.
- A Python Introduction to Parallel Programming with MPI
- MPI for Python (mpi4py)
- Python Guide – MPI
- Setup python (including mpi4py) without admin rights
- Python for High Performance
While Python is a scripting language, it has plenty of facilities for high performance computing. This article covers some of its features and libraries that are particularly helpful when moving scientific code to a large cluster resource. It also includes specific recipes for compilation and execution on the TACC clusters.