open TELEMAC-MASCARET The mathematically superior suite of solvers

Hi, I'm working at the Leibniz Supercomputer Center in Garching near Munich. Recently, I got the task to run and benchmark a parallelized telemac on SuperMUC, which has a SLES 11.3 OS. This is to check scalability of telemac.
I noticed that the partitioning (partel) takes rather long (>40 minutes in my case) and runs in a serial manner - which is seriously a waste of resources when running maybe hundreds of nodes in parallel. From the configuration file I edited I conclude that partel is linked to metis. But there is also a parallel version - parmetis. I wonder now whether it's possible to link partel also against parmetis? Unfortunately, I didn't succeed to do so so far. I also didn't find any documentation about this at Dr. Google.

Btw. the idea is not completely and genuinely mine:
wiki.opentelemac.org/doku.php?id=downloa...em_for_your_computer
In the optionals folder, there is a parmetis folder sketched. So I hope, someone had the same idea as I had.

Thanks for any hint and help.
Cheers,
Martin

Hi there is a parallel version of partel called partel_para that uses parmetis.

However it is not fully validated adn not complitely up to date.

I'll help if i can.

Thank you very much for the fast reply and for the good news. And I'm really willing to try partel_para.
I indeed discovered also an executable ./builds/susgfintelmpi/bin/partel_para. But as I've not specified any parmetis-library, I've no clue how that's possible.
How do I correctly build and link partel_para against parmetis, i.e. how looks the config file? Is libs_partel still the correct tab?

And second, I guess I've to replace
par_cmdexec : <config>/partel < PARTEL.PAR >> <partel.log>
by
par_cmdexec : mpiexec <config>/partel_para < PARTEL.PAR >> <partel.log>
?

to add parmetis do the following in your configuration file:

cmd_obj: ... -DHAVE_PARMETIS ...
libs_partel: ... <path-to-parmetis>/lib/libparmetis.a ...

par_cmdexec : mpiexec -n <ncsize> <config>/partel_para < PARTEL.PAR >> <partel.log>

That should do the trick

Hello,
I want to say that I'm also interested.
Some weeks ago for a benchmark test at our cluster I tried to use parmetis but already in the compilation process I stucked.
Configuration:
1. I compiled parmetis 4.0.3 at the cluster with gcc 4.8.2

2. Relevant lines in the config file:
- libs_partel: /home/.../build/Linux-x86_64/libparmetis/libparmetis.a
- cmd_obj: mpif90 -c -O2 -mavx -DHAVE_MPI -DHAVE_PARMETIS -fconvert=little-endian -frecord-marker=4 <mods> <incs> <f95name>

Error:
________________________________________________________________
partitioner.o: In function `partitioner_':
partitioner.F:(.text+0x3ba): undefined reference to `metis_partmeshdual_'
collect2: error: ld returned 1 exit status
compileTELEMAC::main:
+> creating executable: homere_partel:
|createExeFiles: something went wrong, I am not sure why (runcode=1).
_________________________________________________________________
So it seems that it doesn't call partitioner_para.F?
I would be happy for any hints.

Best regards,
Clemens

My bad i forgot to say that you need to keep metis in libs_partel.
As parmetis needs metis.

Hope it helps.

Thank you for the hints,
however with
libs_partel: /home/.../build/Linux-x86_64/libmetis/libmetis.a

I get the following error:

partitioner_para.o: In function `partitioner_para_':
partitioner_para.F:(.text+0x9af): undefined reference to `parmetis_v3_partmeshkway_'
collect2: error: ld returned 1 exit status
compileTELEMAC::main:
+> creating executable: partel_para:
|createExeFiles: something went wrong, I am not sure why (runcode=1).

Or should I keep also libparmetis.a?
Best regards,
Clemens

Yes you should keep both and in the following order:
/.../lubparmetis.a /.../libmetis.a

Thank you!
Compilation is successful.

Best regards,
Clemens

Compilation fails at sources/utils/partel/partitioner_para.F with
/home/hpc/pr28fa/di49zop/Progs/opentelemac3/v6p3r2/sources/utils/partel/partitioner_para.F(94): error #6404: This name does not have a type, and must have an explicit type. [ELMDIST]
ALLOCATE (ELMDIST(NPARTS+1),STAT=ERR)
I'm unfortunately no Fortran programmer, but it seems as if some kind of header information is missing. For completeness, I use telemac version v6p3r2 (because my client uses it). So, stupid question: Is this version already complete enough to compile with parmetis? Or do I have simply forgotten something in my config file (attached)?

(Btw. I explicitly removed "-n <ncsize>" etc. because mpiexec is a wrapper coupled to the batch queueing system where it drags the necessary host file info.)

Thanks, and best regards!