Greetings,
I am currently trying to run TELEMAC-3D (v8p4) on my university’s cluster, but I am facing a major issue. The simulations in parallel do not produce the same results as in sequential mode. The same problem occurs with the provided TELEMAC-3D examples. This issue only affects 3D simulations, 2D ones work fine.
I would be very grateful for any insights on this issue.
Hi [name removed],
How different are the results? You will likely get more help if you provide specific examples and values and figures.
[name removed]
First of all, thank you very much. I agree with you. Previously, I have tried to attach some figures, but each time I was blocked from publishing my question. I hope this time it worked.
The figures compare the results of a simulation I am working on: one run was done on my own computer, the second on a cluster without parallel processing, and the last one used the cluster with just two cores. I noticed that the same problem occurred even with the Telemac 3D examples. I hope the problem is now clearer. Thank you in advance.
Hi
You compare the listing information but in parallel and in sequential this is not related to the same domain so this sounds logical to see some differences.
What about the comparison of results files?
Hi,
Thank you very much for your reply. Would you mind elaborating a bit more? I didn’t quite understand why it’s considered a different domain, even though I’m running the exact same simulation. How does running in parallel change the domain, and how does that specifically affect the listing output? I’m asking because I didn’t encounter this issue with TELEMAC-2D — the listing results are identical in both parallel and sequential modes.
To answer your question: yes, even the results files are different when I read them — and the differences are visible even at the initial timesteps.
Hi
Parallel computation is made by domain decomposition and you will have one listing per domain. Only the one coming from the master node is printed on screen, the other are redirected to files.
If the results are different at the initial time step, this means there is another problem somewhere…
Hi,
Thank you very much for your insights. I would appreciate any suggestions you might have on how to proceed or further investigate this issue.
Hard to say how investigate precisely!
If the difference occur at the initial time step, I would try to look the splitted file and the results at T=0
Try to run your parallel simulation in dedicated directory.
If this occurs with telemac example, please indicate which one to let us check on our side but as the validation process include a comparison between results in parallel and in sequential, I’m unsure we will be able to reproduce your problem, otherwise the telemac validation will highlight a problem…
Thank you for the clarification. I’ll run some tests based on your suggestions and get back to you with more details soon.
Hello,
I have tried your suggestions, but the problem still persists. I tested several TELEMAC 3D examples, and the issue appears consistently. For instance, I ran the ‘Vent’ example both in sequential mode and in parallel mode using 2 cores on the cluster, and the results differ.
I’ve attached screenshots of the listing outputs, the attribute tables in BlueKenue (for an arbitrarily chosen node), and the partel_T3DGEO file from the same example.
Since these examples are part of the validated TELEMAC suite, could the issue be related to how TELEMAC-3D is installed on the cluster?
partel-T3DGEO.txt (3.05 KB)
Hello [name removed],
Unfortunately, the issue of discrepancies between serial and parallel simulations is a well-known one in the TELEMAC system. That said, the resulting error is typically very small and therefore manageable.
From what I understand of your results, the error is less than 1E-4 over a volume of 500,000 m³. That’s actually very good, and so far, the TELEMAC team hasn’t found a way to improve this further. Additionally, you’re displaying elevation values around 0.000289, which is below a millimetre – predicting to the centimetre would already be quite fortunate!
Given this, I’m not sure why this is causing significant delays in your research. These kinds of differences are not something you should be concerned about.
Hope this helps,
[name removed].
Good morning Mr. [name removed],
Thank you very much for your reply. I completely agree with you that the discrepancies in these results remain small. However, these values correspond to the results of a TELEMAC-3D example, the Vent case, which is highly simplified. In contrast, when I run my simulation in the Strait of Gibraltar, where the spatial resolution is much higher, the discrepancies between the parallel and sequential versions become significantly larger.
For further clarification, I have attached the results of both simulations: one using the sequential version and the other using the parallel version with only 2 cores. The attachments include screenshots of the listing files and attribute tables showing the velocities at an arbitrary point on the surface.
Thank you in advance
I apologize, please disregard the attachment in the previous post, as I mistakenly uploaded the wrong figures. In this post, I am attaching the correct ones.
This could be great if you could share all the files required to run your case because in your case the differences are significant but this remain strange for Telemac.
Nevertheless, without the case, it’s hard to investigate and help you
Regards
Good morning,
Thank you very much for your response. Would you mind sharing your email address so I can send you the files related to my case?
Best regards,
It depends on the file size but you could join a zip of all the file here or give a link to a shared drive.
Greetings,
Please find below a Drive link to a folder related to one of the simulations I’m working on for the Strait of Gibraltar. I’ve also attached screenshots showing the file listings and attribute tables for two simulation runs on the cluster—one without parallelization and one using parallelization with only 2 cores. I hope this helps provide an overview or offers some clues about the issue.
Thank you in advance.
Link: SOG_TPXO10 – Google Drive
Hello,
I have run your input files for a few configurations (laptop/cluster, intel/gnu/nag compilers, 1 core/2 cores) and every computation writes the same initial volume of water: 0.5364304E+12 m3 contrary to your 2 core cluster configuration.
The installation on your cluster is suspicious to my point of view and should be investigated.
Anyway, it is strange that you do not see discrepancies for TELEMAC-2D example whereas you see them for TELEMAC-3D example.
Your modelling is close to the tide examples. You should see same behaviours when using TELEMAC-2D and TELEMAC-3D for the t2d_tide-jmj_type.cas and t3d_tide-jmj_type.cas examples.
Hope this helps,
[name removed]
Greetings,
First of all, thank you very much for your reply and for the time and effort you put into running these simulations. I would also like to thank everyone who was involved in investigating this issue — I truly appreciate the collaboration and support of this community 
Regarding the problem, it has now been resolved. I reinstalled TELEMAC from scratch on the cluster, this time using version v9 instead of the older one, and switching from MPICH to OpenMPI. Interestingly, the issue disappeared. It seems there was indeed a problem with the previous installation on the cluster.
Once again, thank you very much for your time, help, and consideration, and I apologize for any inconvenience caused.
[name removed]
[name removed], can you upload your cfg file you used for installation. I run into trouble for the installation of telemac on cluster. Thx in advance
