open TELEMAC-MASCARET The mathematically superior suite of solvers

Hi,

This is definitively an interesting topic, didn't know that a parallel implementation of metis existed! As Martin said, sometimes splitting/merging could take around 1 hr+ in our case!

I'll try to compile telemac with Parmetis using GCC 5.4 in our linux cluster and report back!

Regards,

José D.

ohlerich wrote:
We had the same problem on our cluster some time ago. I'm not sure but I think the problem was the compiler. Do you use the Intel compiler? Which version? Maybe you get better results if you use an older version.

O.T.
One problem of ParMETIS is the license.

Regards,

Leo

Currently, I use intel 14.0 compiler - as telemac fails completely with newer versions (I tried 15 and 16). Should I use a still older version?

ParMETIS license is no issue as we are non-profit

Thanks, Martin.

Hi Leo,

we had also very long partitioning times at our cluster, like Martin wrote.
In our case definitely it didn't depend on the compiler since I tried
GCC 4.8.2, Intel 14.0.2 and Intel 16.0.3.
We used Telemac v7p1r1.

Best regards,
Clemens

Hi Clemens, hi Martin

have you allways had such long partitioning times? We observed this problem a while ago on our cluster after we had a change of our default compilers. I've asked Rebekka, she said that we use an old version of the Intel compiler (composer_xe_2013_sp1.2.144).

Maybe the problem lies somewhere else? But sure, ParMETIS is a good way instead of fiddling around with all possible compiler versions and options.

Best regards,

Leo

Hi Leo,
I don't know since in the past I didn't pay much attention to this because for the daily use we don't use a lot of cores.
The case here was a benchmark case involving more cores than usual for which I did a comparison of different compilers with the focus only on simulation times.

Best regards,
Clemens

Hi Leo, same as Clemens case - I try a benchmark with up to about 128*28 cores. I did not use telemac before - so have no knowledgebase.

Hi Martin,
yes, there were some issues with Intel compilers 15 and above when trying to compile older versions of Telemac, in your case v6p3r2.
I remember some forum posts about this.

Best regards,
Clemens

Hi Martin,

128*28 cores is a lot we use between 80 and 200 cores. Usually we say that we use between 5000-10000 elements per core. So your systems is much larger (~20 million elements?).

So maybe there is another limitation/problem with partitioning such large grids. We observed the extreme long times for partitioning on our daily use problems (200 thousand - 1 million elements)

Well, maybe Wasn't my idea:
www.wb.bgu.tum.de/fileadmin/w00boi/www/P...htshefte/Band131.pdf
(sorry, don't find any English version - but is about modeling the Danube area in Bavaria).
I'm currently only testing a 100 km channel to check scalability. As it was not generated by me, I still have to find out how many cells are this in total. But we required from our customer enough numerical cells to represent and meet the final project requirements. So, parmetis is in this case worth to think about

Thanks for your advice!
Cheers, Martin