Math578 - Alexiades
              Towards Parallelism: Part 1
Parallelizing is painful... especially the first time... it takes time, so let's start preparing...

  • Start by reading about Parallel Computing and about Distributed Computing on Wikipedia.
      Our interest is on distributed memory computing via message passing, so pay more attention to those...
  • It will be done by "domain decomposition", using the MPI library.
  • Current Intel terminology on clusters-nodes-sockets-cores-...

  • Parallelization can be done and tested on your own laptop, but don't expect to see speedup...
  • After debugging/testing your code on your laptop, you will port it to a real cluster, which takes some work...
  • Got us a class account on UT's ISAAC cluster (successor to Newton cluster), see below.
      If you already have an account on ISAAC/Newton cluster, let me know.
  • Also take a look at the Guide to High Performance Computing by Shane Sawyer
  • Then take a look at the excellent, extensive tutorial from Cornell:
        Parallel Programming Concepts and High-Performance Computing
                  About MPI ( Message Passing Interface )
  • Read about MPI on wikipedia, to get an idea. We'll talk about it in class...
  • Check if your Unix/Linux system has some MPI installed.
      Try: which mpiexec it should return the path to it. If you get nothing, it may still be there somewhere...
      Try: which mpirun , Try: which mpicc
      If your Linux system has some MPI installed, fine ... BUT be aware:  
       mpicc and mpirun must come from the same MPI implementation.
      It is best to install your own openMPI (or MPICH) and put the path to it in your Makefile.

  • In any case, you can easily install your own 'openMPI', under Linux, as follows:
      1. Get the latest stable version ( v4.1.0 ?) from open-MPI.org
      2. Unpack it: gtar zxf openmpi-X.Y.Z.tar.gz
      3. cd openmpi-X.Y.Z
      4. read the INSTALL file and follow the instructions.
      Another option is to install MPICH , which is another implementation of the MPI standard.
      This also exists for MS_Win, but then you'll need MS_Win compiler... not sure if this can work out...
      best by far is to install VirtualBox and Linux (ubuntu is probably the most user-friendly).

    Note: Do NOT confuse "openMPI" (an implementation of MPI) with "OpenMP" (shared memory programming standard)!

    MPI has native bindings for Fortran and for C/C++. However, now Python is also an option with:
  • mpi4py , you can call Fortran and C code directly through wrappers
    or
  • boost module, has native python bindings built directly on the C++ mpi bindings.

  • A good, detailed, reference is this   MPI tutorial , from LLNL.
                    ISAAC: UTK's Advanced Computing Facility (ACF)
  • Connect to ISAAC portal with your NetID:   account request and ask to be added to class account: ACF-UTK0151
  • After your account is set, direct login to : ISAAC portal to ACF   UTK's Advanced Computing Facility