====== Compiling and Running C++ MPI Applications ======

This guide explains how to load the Open MPI module, compile a distributed C++ program using the MPI wrapper, and submit an MPI job to the **unite** partition on the cluster.

===== IMPORTANT: Compilation Policy =====

^ WARNING: Do not run compilation commands directly on the Login Node! ^
| High-performance libraries like Open MPI can cause heavy overhead during compilation. Always allocate an interactive compute node session on the **unite** partition using **`srun`** before building your code. |

===== Available Modules =====

To see the available Open MPI configurations, run:
<code>
$ module avail unite/mpi
</code>

===== Step-by-Step Guide =====

==== 1. Request an Interactive Node for Compilation ====
Switch to a safe environment on a compute node under the **unite** partition:

<code>
$ srun --partition=unite --cpus-per-task=2 --time=00:30:00 --pty bash
</code>
Once your command prompt updates, you are active on a compute node.

==== 2. Load the Required Modules ====
MPI modules require a compatible compiler toolchain. Load both the GCC compiler and the corresponding Open MPI library:

<code>
$module load unite/compilers/gcc-14.3.0
$ module load unite/mpi/4.1.0
</code>

To verify your environment's active C++ MPI compiler wrapper, run:
<code>
$ mpicxx --version
</code>

==== 3. Create a Sample MPI C++ File ====
Create a parallel C++ file named `mpi_hello.cpp`. This example initializes the MPI environment, determines the rank of each process, and identifies how many total processes are running.

<file cpp mpi_hello.cpp>
#include <mpi.h>
#include <iostream>

int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(&argc, &argv);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank (ID) of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    // Get the name of the processor node
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

    // Print off a hello world message from each process
    std::cout << "Hello world from process rank " << world_rank 
              << " out of " << world_size 
              << " processors on node " << processor_name << std::endl;

    // Finalize the MPI environment.
    MPI_Finalize();
    return 0;
}
</file>

==== 4. Compilation ====
Compile your application using the `mpicxx` wrapper script, which handles adding all necessary linking flags for Open MPI automatically.

<code>
$ mpicxx -O3 -std=c++17 mpi_hello.cpp -o mpi_hello_executable
</code>

* **`mpicxx`**: The C++ compilation wrapper script for Open MPI.
* **`-O3`**: High-level optimization flag.
* **`-o mpi_hello_executable`**: The resulting binary output.

Exit your interactive session to return to the login node when the build completes successfully:
<code>
$ exit
</code>

===== Running Distributed Jobs via Slurm Batch =====

To launch an MPI job across multiple processors or nodes via Slurm, create a batch script named `submit_mpi.sh`:

<file bash submit_mpi.sh>
#!/bin/bash
#SBATCH --job-name=mpi_cpp_job
#SBATCH --partition=unite           # Target partition for the UNITE cluster
#SBATCH