open TELEMAC-MASCARET The mathematically superior suite of solvers

Hi Yoann,

I tried your suggestions and with --ncsize=4 the example started to run but was extremely slow, it took more than 2 hours to finished while in sequential mode it only needs 5 sec...please refer to HPC_STDIN attached. Then I turned to python27 when compiling v8p1r0 and it worked well, I'll just stick to it for the moment.

Best,
Yunhao

This seems weird.
Is your HPC_STDIN the same in python2 ?

Almost the same, with Python27 the srun was used, please check the file below.

Ok I think the issue could come from anaconda.
I had some issues sometimes because anaconda contains a lot of pre-compiled library that can cause some issues.
TAs it is the only difference between the two.

Dear Yugi,

I allowed myself continuing this post as the parallel-related problem hasn't been solved when using the latest v8p2 on HPC. As you mentioned in the previous posts that this could be caused by Anaconda, should I give up using Anaconda 3.7 and install python3 and Numpy on the HPC myself?

Before this update we have been running parallel models using v8p0 with Anaconda 2.7 and intel parallel studio and everything went well, just FYI.

Any suggestion is appreciated.
Yunhao

If you are not using the API there are less issues with anaconda.

Internally we use a local installation of Python and use pip to install addtional packages.
This does the job as you can tell at the installation what compiler/lib to use.

You quickly have issues combining telemac and anaconda because Anaconda will use its own Fortran compiler and its own MPI. We also have issue with hdf5 as well as it is using its own.

So yes I would recommend installing it yourself.

Hi Yugi,

To check out if the problem in my case was caused by anaconda, I installed Python 3.9.1 locally and used pip3.9 to install NumPy 1.19.4, but the error was still there.

The command used to run the example case is
The maximum number of CPUs allocated for my account is 16 so it's really weird...Please find the STDIN, pysource, config, bash_profile and partel log files in the attachment, now I really don't know where to start...

HI,

Could you post the run log ?
Also can you try running a simple program with the same HPC_STDIN ? something like a simple hello world.

Hi yugi,

Does the run log mean the sortie file? In the slurm-[jobID].out file there is nothing but:
I did the hello world mpi test as you mentioned and the result seemed suspicious either, the node id is the same 0 instead of (0;1;2;3) as shown in the snapshot.

To bypass this problem I switched into Python27+v8p2r0 and the telemac example was finished in parallel with no error. The comparison between slurm jobs information of Python27+v8p2r0 and Python3+v8p2r0 was displayed in the second picture, it's noticeable that Python3+v8p2r0 failed to pass the value of [ncsize] to the job submitting system thus only one cpu was allocated, triggering the errorI think this is where the problem came from...Could you shed more light on how to solve this?

Many thanks,
Yunhao

yugi wrote:
I used the command: telemac2d.py -s CASFILE.cas