open TELEMAC-MASCARET The mathematically superior suite of solvers

Dear Users,

Somebody might have come across to similar problem I have when I try to run Telemac in Parallel and could give me advice. It compiled fine but it does not run and gives me an errror:

sh: gfortran: command not found

---

mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.


I was searching on internet and sombody said that when you enter mpif90 -show and if you get ...../include -twice, as I have (see below) then system is confused and it might be the couse. I am not even sure how it appeared there twice, some cluster original settings? Or it could come from my configuration file, please see attached.

gfortran -I/gpfs/apps/openmpi_gnu/1.6.4/include -I/gpfs/apps/openmpi_gnu/1.6.4/include -pthread -I/gpfs/apps/openmpi_gnu/1.6.4/lib -L/gpfs/apps/openmpi_gnu/1.6.4/lib -Wl,-rpath=/gpfs/apps/openmpi_gnu/1.6.4/lib -lmpi_f90 -lmpi_f77 -lmpi -ldl -lm -Wl,--export-dynamic -lrt -lnsl -lutil -lm -ldl

My platfort is Linux
Compiler - gfortran
Lilbary - openmpi_gnu/1.6.4
language - python
Cluster - Blue Ice 2 HPC
Please advise.

Thanks.

Kind Regards!

Violeta

Hello,

Sorry for the delay. Your configuration seems OK. I do not know why you are getting the duplicate include or whether it is the cause of your troubles.

Here below is a config with gfortran and openmpi 1.6.4. I use gfortran 4.4.6. I would not recommend any below that (you can have above that). Also you have to make sure that everything is compiled with the same compiler (libmpi.so and libmetis.)

One more thing: note that I have sfx_lib: .lib and not .a
It may have a bigger influence than we think. Do let us know if so.

Hope this helps,
Sébastien.

Thank you Sebastien. I cannot explain why .../include is twice on cluster and nobody here can't, but I hope it does not cause any problems, the same is for ifort ..include is twice. It is probably cluster setting.

However, I still have problems to install in parallel on cluster. Thanks for looking at my config file. I have tried on cluster with Openmpi and gfortran but parallel did not work, it gave be constant error that gfortran: command not found. And I thought that something was wrong with gfortran set up not on every node so I moved to intel but now I cannot compile at all. The new error is:

undefined reference to 'metis_partmeshdual_'.

I looked on forum and there was note that to avoid this problem the metis should be 4.0.3 as it compiles partel. But also it is advised to use metis-5.x.x so my metis is 5.0.2 but I still have this problem.

Another question is, do I need to install MUMPS for be able to work with Telemac in parallel? I understood that it is not necessary step.

To install metis I needed Cmake and both of them are easy to install on desktop pc but not on cluster as both of the programs are looking for /usr/local.. but it is not possible on cluster since you do not have authorization and you cannot use sudo. So error might have occured there as I had to redirect them manually. It was installed but I never sure if it was done 100%, so I am going to re-install them again, as I cannot see any other solutions at the moment.


Actually just another quick question: for ifort in cmd_exe line should it be mpif90 as well?


Thanks for your reply.

Kind Regards!

Violeta

OK - I would stick to gfortran if you can. Indeed you seem to have problem with incompatibility between metis, mpi, gfortran (or intel). And indeed, by default a compilation of METIS will produce:

In order to change that if you do not have access to /usr, you can do as follows:(after having created ~/opentelemac/libs/)
and then
With that, it should create a bin, a include and a lib directory, and the lib includes libmetis.a which you can point to in your TELEMAC config file.

Note, you can probably install metis with intel as well.

Hope this helps,
Sébastien.

Thank you very much Sebastien. I will follow your advice, it is something different that I have done before so it might work.

I have just changed back to gfortran and it worse, even scalar cannot run with gfortran, but it runs with intel. So I will try with intel with metis and cmake from scratch now. Thanks.

Kind Regards!

Violeta

Hello,

When you recompile TELEMAC after changing options - make sure you use the --clean option of compileTELEMAC.py just to make sure.

Good luck.
Sébastien.

Thanks a mil Sebastien. That is probably my mistake as I have never used clean option before. I will definitely use it from now on. Thanks.

Kind Regards!

Violeta

Hi Sebastien,

I wonder if you have come accross other users having similar problem as mine. I have cleaned and resistalled cmake-2.8.11 and metis-5.0.2. With metis I created libs in opentelemac and after metis installation I got new directories: bin, include and lib with libmeetis.a in it which path I entered in config file.

When I started to compile with 'clean system', the all modules are compiling fine, but when it comes to PARTEL_PRELIM I get the following error: undefined reference to `metis_partmeshdual_'. I even removed par_cmdexec: <config>/partel < PARTEL.PAR >> <partel.log> line from config file but still getting same error. Is there is anything else I have to do to link metis and telemac? I include my latest config file, just in case as there could be some new errors as I changed to intel. Please advise. Thanks.


Compiling the program PARTEL_PRELIM and dependents
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/special/plante.o
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/special/check_allocate.o
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/partel/partitioner.o
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/partel/partel_prelim.o
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/special/partel_prelim.lib
- created /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/partel/partel_prelim.lib
ifort: command line remark #10010: option '-pthread' is deprecated and will be removed in a future release. See '-help deprecated'
partitioner.o: In function `partitioner_':
/home/violeta.moloney/opentelemac/v6p3/sources/utils/partel/partitioner.F:(.text+0x5ce): undefined reference to `metis_partmeshdual_'
... The following command failed for the reason above
/gpfs/apps/openmpi_intel/1.6.4/bin/mpif90 -o /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/bin/partel_prelim partitioner.o partel_prelim.o -Wl,--start-group /home/violeta.moloney/opentelemac/v6p3/builds/ubuifortopenmpi/lib/utils/special/partel_prelim.lib /gpfs/apps/openmpi_intel/1.6.4/lib/libmpi.so -Wl,--end-group
[violeta.moloney@mg01 python27]$

OK it looks like METIS is your only remaining problem.

If you are compiling TELEMAC with ifort, you need to compile metis with ifort as well -- otherwise ifort cannot read the lib created by gfortran) -- I believe the default is gcc (including gfortran). I am not sure how to switch to ifort so that cmake use it to compile metis. Maybe something like make config ...

With ifort you may have to add /fpp (even on Linux) to your cmd_obj command.
(do a --clean after that).

I think you are almost there ...

Sébastien.

Hello again,

Note also that you do not need the -Wl,--start-group ... -Wl,--end-group anymore, although it doe snot hurt, as we have done a much better job at sorting dependencies throughout the system.

Sébastien.