Math578 - Alexiades

              Towards Parallelism: Part 1
Parallelizing is painful... especially the first time... it takes time, so let's start preparing...

  • Start by reading about Parallel Computing and about Distributed Computing on Wikipedia.
      Our interest is on distributed memory computing via message passing, so pay more attention to those...
  • It will be done by "domain decomposition" and the MPI library.
  • Current Intel terminology on clusters-nodes-sockets-cores-...

  • Parallelization can be done and tested on your own laptop, but won't see speedup...
  • So, trying to get us access to a cluster at the NICS ACF... so far no luck...
      If you already have an account on some cluster, let me know.
                  About MPI ( Message Passing Interface )
  • Read about MPI on wikipedia, to get an idea. We'll talk about it in class...
  • Check if your Unix/Linux system has some MPI installed.
      Try: which mpiexec it should return the path to it. If you get nothing, it may still be there somewhere...
      Try: which mpirun , Try: which mpicc
      If your Unix/Linux system has some MPI installed, so much the better... BUT be aware:  
       mpicc and mpirun must come from the same MPI implementation.
      It's best to install your own openMPI (or MPICH) and put the path to it in your Makefile.
  • In any case, you can easily install your own 'openMPI', under Linux, as follows:
      1. Get the latest stable version from
      2. Unpack it: gtar zxf openmpi-X.Y.Z.tar.gz
      3. cd openmpi-X.Y.Z
      4. read the INSTALL file and follow the instructions.
      Another option is to install MPICH , another implementation of the MPI standard.
      This also exists for MS_Win, but then you'll need MS_Win compiler... not sure if this can work out...
      best by far is to install VirtualBox and Linux (ubuntu is probably the most user-friendly).

    Note: Do NOT confuse "openMPI" (an implementation of MPI) with "OpenMP" (shared memory programming standard)!

    MPI has native bindings for Fortran and for C/C++. However, now Python is also an option with:
  • mpi4py , you can call Fortran and C code directly through wrappers
  • boost module, has native python bindings built directly on the C++ mpi bindings.

  • A good, detailed, ref is this   MPI tutorial , from LLNL.
                    ACF info and getting ACF account
  • ACF   UTK's Advanced Computing Facility
  • Acceptable Usage Policy

  • Access and Login   read about Duo Setup Directions (will need a smart phone...)
  • Request an ACF account using your UTK NetID, Project "ACF-UTK0011", and install Duo.
  • Then login (using SSH on linux) via Duo:   ssh
  • Once connected, you will be on
  • You can SCP files to/from ACF:   scp -p   FILE
            ...if I forgot anything let me know...       last updated on 6sep17