open TELEMAC-MASCARET The mathematically superior suite of solvers

Dear All,

I am trying to set-up the parallel version of Telemac, has any one got the configuration file corresponding to intel-fortran 10.1.013 and intel mpi 3.1.

Our cluster is a 64 bit-linux running CentOS release 5.4.

Any and all help is greatly apprecitaed.

Thanks,
Sumit

We are not supporting MPI-3 (yet).

Could yo ugo back to the MPI-2 sandard and try again ?
Can you let us know whether you use the python or perl installation procedure ? ... as the configuration files are a bit different.

Hope this helps.

Sébastien

Hello Sebastien,

Thanks a lot for your reply, I am using python to get things going.

We have openmpi-1.3.3 and intel mpi 3.1 already installed.

As I don't have all the admin rights on the cluster I was looking for a config file which will help me compile the parallel version of Telemac with all the rights that I have.

As for fortran we have intel-fortran 10.1.013 already installed, can you give me an idea what config file will work best for me. I am already getting good result with the serial version of Telemac and I want to make the jump to the next (parallel) level.

I already installed the required metis version.

Best regards,
Sumit

Hello,

I need to talk to our IT Department (monday) but in the meantime here are a few pointers:

For Intel on windows:

cmd_obj: ifort.exe /c /Ot /iface:cref /iface:nomixed_str_len_arg /nologo /names:uppercase /convert:big_endian /extend_source:132 <mods> <incs> <f95name>
cmd_lib: xilib.exe /nologo /out:<libname> <objs>
cmd_exe: xilink.exe /nologo /subsystem:console /stack:536870912 /out:<exename> <objs> <libs>

For parallel on CentOS (with gfortran and mpich2)

options: parallel mpi
#mpi_hosts -localonly
mpi_hosts: -machinefile /share/hostfile
mpi_cmdexec: /usr/bin/mpiexec <hosts> <ncsize> <wdir> <exename>
#
cmd_obj: gfortran -c -O3 -ffixed-line-length-132 -fconvert=big-endian -frecord-marker=4 <mods> <incs> <f95name>
cmd_lib: ar cru <libname> <objs>
cmd_exe: gfortran -fconvert=big-endian -frecord-marker=4 -lpthread -v -lm -lz -o <exename> <objs> <libs>
#
mods_all: -I <config>
#
incs_parallel: -I /usr/include/mpich2-x86_64
libs_parallel: /share/opentelemac/lib/metis-4.0.3/libmetis.a
libs_all : /usr/lib64/mpich2/lib/libmpich.a


For parallel Fedora (with gfortran and OpenMPI):

options: parallel mpi
#mpi_hosts -localonly
mpi_cmdexec: /usr/lib64/openmpi/bin/mpiexec <wdir> <ncsize> <hosts> <exename>
#
cmd_obj: gfortran -c -O3 -ffixed-line-length-132 -fconvert=big-endian -frecord-marker=4 <mods> <incs> <f95name>
cmd_lib: ar cru <libname> <objs>
cmd_exe: /usr/lib64/openmpi/bin/mpif90 -fconvert=big-endian -frecord-marker=4 -lpthread -v -lm -o <exename> <objs> -Wl,--start-group <libs> -Wl,--end-group
#
mods_all: -I <config>
#
incs_parallel: -I /usr/include/openmpi-x86_64/
libs_parallel: /home/telemac/metis-4.0.3/libmetis.a
libs_all : /usr/lib64/openmpi/lib/libmpi.so


The important parts are:

- incs_parallel (to point to the inlcude directory for you MPI or OpenMPI install, where mpif.h will be)
- libs_parallel (to point to where the /metis-4.0.3/libmetis.a, which you need to compile yourself based on your compiler -- this is easy on linux by the way, just follow the metis standard instructions)
- libs_all (to point to the libmpi, appropriately MPI or OpenMPI).

Then you need to make sure you have your command call right. For instance /usr/lib64/openmpi/bin/mpif90 is used in the last configuration to create the executable instead of gfortran.

Then you just need to add options (parallel mpi -- whether you use michp2 or openmpi) and then:
- mpi_hosts: -machinefile /share/hostfile (see another post on this forum)
- mpi_cmdexec: /usr/bin/mpiexec <hosts> <ncsize> <wdir> <exename>
for the execution.

Voilà -- Hope this helps.

Sébastien

Thanks a lot Sebastien, I am going to give this a shot and see where I get stuck.

Hello Sebastien,

I was able to compile things using intel mpi 3.1 and gfortran, I came to a stage where the compileTELEMAC.py gave me the end result as, "My work is done" and I can see that all libraries and depedencies are made.

Now I need a little guidance about how to set up mpi_hosts, right now in the config file (attached with this post) it is commented as -localonly. In nut shell can you kindly let me know how to submit the cases in parallel, is it exactly same as submitting through runcode.py but then how and where do I specify the number of processors and things like that.

I am very thankful for your help.

Best regards,
Sumit

Nice job !

In the command for mpi (mpi_cmdexec) the <hosts> key is repalced by what is in the command mpi_hosts.

For instance, you may find:

So since you are just writting up there the mpiexec command and its options, you can just have:

... for example, where /share/hostfile is a text file with the list of the names of your nodes (you have to duplicate the name of a node if you wish to use more than one processor of that node). For instance:

gcl01
glc02
glc02
glc02

If I remember correctly mpiexec can also be used with the list of nodes in the command line, with -n 4 gcl01 glc02 glc02 glc02 for instance ...

Hope this help.

Sébastien.

Hello Sebastien,

I did make the host file but my simulation is not taking of; the command I give to run telemac2d on cluster is

python2.7 /scratch/sinha6/opentelemac/v6p1/pytel/runcode.py telemac2d -c Cengfopenmpi -f /scratch/sinha6/opentelemac/v6p1/config/systel.cis-CentOS.cfg -s NewLem2D.cas

The erros that I get is as follows

Loading Options and Configurations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

... parsing configuration file: /scratch/sinha6/opentelemac/v6p1/config/systel.cis-CentOS.cfg
Traceback (most recent call last):
File "/scratch/sinha6/opentelemac/v6p1/pytel/runcode.py", line 616, in <module>
cfgs = parseConfigFile(options.configFile)
File "/scratch/sinha6/opentelemac/v6p1/pytel/config.py", line 175, in parseConfigFile
configDict = getConfigs(file)
File "/scratch/sinha6/opentelemac/v6p1/pytel/config.py", line 151, in getConfigs
parser.error("Could not access required parameters in config file")
NameError: global name 'parser' is not defined

I am not sure what am I doing wrong, I have ofcourse switched on the PARALLEL PROCESSOR keyword in the cas file. Also I want to let you know that I have already ran the same simulation on windows machine with the scalar version but as I said I wanted to explore the parallel version hence all the bothersome mails to you .

Thanks a ton,
Sumit

You seem to have an error in your cfg file -- can you attach the new one ?

Please find attached my new cfg file, I hope I am close to the solution :)