OpenFOAM: Running Applications in Parallel

OpenFOAM is an open-source CFD software, that allows users to solve complex fluid flow problems through the use of numerical simulations. One way to speed up simulations and reduce the computational time required is to run applications in parallel.

To run applications in parallel in OpenFOAM, a parallel computing framework such as MPI (Message Passing Interface) or OpenMPI is needed. These frameworks allow distributing the computational load across multiple processors or cores, enabling the solution of more significant and complex problems in less time.

To run an OpenFOAM simulation in parallel using MPI, you will need to follow these general steps: decomposing the domain, running the simulation, then reconstructing the domain.

Decompose the problem into smaller domains can be done using the domain decomposition tool available in OpenFOAM: decomposePar utility. Both the mesh and fields are decomposed according to a set of parameters specified in a dictionary named decomposeParDict that must be located in the system directory of the case of interest. Hereafter is an example of a decomposeParDict file.

/*--------------------------------*- C++ -*----------------------------------*
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     | Website:  https://openfoam.org
    \\  /    A nd           | Version:  8
     \\/     M anipulation  |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 4;

method          scotch;

simpleCoeffs
{
    n               (2 2 1);
    delta           0.001;
}

hierarchicalCoeffs
{
    n               (1 1 1);
    delta           0.001;
    order           xyz;
}

manualCoeffs
{
    dataFile        "";
}

// *************************************************************\        

Four methods of decomposition are available in OpenFOAM, specified by the method keyword.

• simple: Simple geometric decomposition in which the domain is split into sub-domains by direction, for example, the line below means two sub-domains in the x-direction, two in the y-direction, and one in the z-direction.

method          simple; 
n               (2 2 1);        

• hierarchical

Hierarchical geometric decomposition is the same as simple except the user specifies the order in which the directional split is done using the keyword order. In the example below, first decompose in the x-direction, then in the y-direction and z-direction.

method              hierarchocal;
hierarchicalCoeff
{
    n               (4 2 1);
    delta           0.001;
    order           xyz;
}        

• scotch

Scotch decomposition which requires no geometric input from the user and attempts to minimize the number of processor boundaries. The user can specify a weighting for the decomposition between processors, through an optional processorWeights keyword which can be useful on machines with differing performance between processors. There is also an optional keyword entry strategy that controls the decomposition strategy through a complex string supplied to Scotch. For more information, see the source code file: $FOAM_SRC/parallel/decompose/scotchDecomp/scotchDecomp.C

• manual

Manual decomposition, where the user directly specifies the allocation of each cell to a particular processor.

To run the domain decomposition, the command below is used:

decomposePar        

The simulation is run using the mpirun command, which launches the MPI executable and distributes the computational load across the specified number of processors or cores.

mpirun -np 4 pisoFoam -parallel        

Once the simulation is complete, the reconstructPar utility is used to reconstruct the decomposed results into a single output file, which can be visualized using ParaView.

reconstructPar        

Overall, running applications in parallel can significantly reduce the computational time required for OpenFOAM simulations, by solving more complex problems and obtaining results more quickly.

For more details please check the OpenFOAM user guide.