Translations of this page:

OpenFOAM

Start version which you want to use by loading correct environment module. Modules are named with following scheme: openfoam/MPI-version/compiler/version

Job script example

Example of the runscript needed for Slurm.

#!/bin/bash -l
### NOTE: if you are running on cluster STYX, you need to add
### -l flag after the /bin/bash !!!!
### job script example with 4 cores on exactly 1 node
### parallel computation of damBreak case with finer mesh
###
 
## name of your job
#SBATCH -J foamjobname
 
## system error message output file
## leave %j as it's being replaced by JOB ID number
#SBATCH -e foamjobname.std.err_%j
 
## system message output file
#SBATCH -o foamjobname.std.out_%j
 
## send mail after job is finished
#SBATCH --mail-type=end
#SBATCH --mail-user=<lut_user_name>@lut.fi
 
## memory limit per allocated CPU core
## try to put this limit as low as reasonably achievable
## too low calculation will fail, too high resources are wasted
## limit is specified in MB
## example: 1 GB is 1000
#SBATCH --mem-per-cpu=1000
 
## how long a job takes, wallclock time d-hh:mm:ss
## here 1 hour is used
#SBATCH -t 0-01:00:00
 
## number of nodes (if necessary)
## -N 1 (job run on exactly one node)
## -N <minnodes:maxnodes>
#SBATCH -N 1
 
## number of cores
#SBATCH -n 4
 
## name of queue 
#SBATCH -p phase_name
 
## load necessary environment modules 
module load openmpi/1.10.4-gcc-local
module load openfoam/openmpi-1.10.4/gcc-4.8.5/5.0
 
## compiling custom solver
## you might want to compile custom solver before actual calculation job starts
## cd $WM_PROJECT_USER_DIR/application/solvers/<yoursolvername>
## wmake
 
## change directory to your calculation directory
## note that the case has been already initialized and decomposed before
cd $FOAM_RUN/tutorials/multiphase/interFoam/laminar/damBreakFine/
## Or this should work also:
## run
## cd tutorials/multiphase/interFoam/laminar/damBreakFine/
 
## run my MPI executable
srun --mpi=pmi2 interFoam -parallel

Compilation of your own solver

  • Load OpenFOAM version which you want to use
module load openmpi/1.10.4-gcc-local
module load openfoam/openmpi-1.10.4/gcc-4.8.5/5.0
  • Create OpenFOAM run and application directories if you don't have them already
mkdir -p $FOAM_RUN
mkdir -p $FOAM_USER_APPBIN
  • First change to main directory of your currently selected OpenFOAM installation. Create applications/solvers and cd to that.
cd $WM_PROJECT_USER_DIR
mkdir -p applications/solvers
cd applications/solvers
  • Create a new directory <yoursolvername> containing your new solver under $WM_PROJECT_USER_DIR/applications/solvers directory. In this example we just copy icoFoam solver from OpenFOAM main installation.
cp -R $FOAM_SOLVERS/incompressible/icoFoam $WM_PROJECT_USER_DIR/applications/solvers
  • Compile your new solver
Interactive run

Reserve interactive run and compile your solver on node by:

srun -p styx -n 1 -t 00:30:00 --pty $SHELL
module load openmpi/1.10.4-gcc-local
module load openfoam/openmpi-1.10.4/gcc-4.8.5/5.0
cd $WM_PROJECT_USER_DIR/application/solvers/<yoursolvername>
wmake
exit
Automatic

Try to compile solver as part of a SLURM job. Add following lines after loading OpenFOAM modules but before OpenFOAM run command in job script.

cd $WM_PROJECT_USER_DIR/application/solvers/<yoursolvername>
wmake

Caveats

If your solver does not compile:

  • Login nodes does not necessarily contain all tools and libraries needed for compilation tasks. Use calculation nodes.
  • Error that output files can't be written. You can't write your custom changes to common apps directory. Check that in <yoursolvername>/<subdir>/Make/files you are not trying to write to common directories. In practise you need to change FOAM_LIBBIN → FOAM_USER_LIBBIN
 
/opt/webdata/webroot/wiki/data/pages/en/hpc/software/openfoam.txt · Last modified: 2018/04/09 11:21 by vrintala
[unknown button type]
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki