open TELEMAC-MASCARET The mathematically superior suite of solvers

Hi Riadh,

Thanks very much for your response earlier. I wonder whether you can help me with one more point and also perhaps spot if I am doing something wrong with your Choice 2?

I have tried running a full 3-way coupled continuation with each of t2d, tom and sis continuing from an initial solution (COMPUTATION CONTINUED, NEXT COMPUTATION). I have tried this in parallel mode (16 processors).

If I use your Choice 2,

EITHER with: IF(TISPEF…. moved to line 247
OR by declaring WAC_FILES and including at line 247: IF(WAC_FILES(NSPE)%NAME.NE.'')THEN ,

the same unit 17 error arises as before. So Choice 2 doesn’t seem to have affected the outcome.

However, if instead I use Choice 1 : providing a filename in keyword 1D SPECTRA RESULTS FILE =...
the continuation run then almost completes: as you can see in the error log below:
- It gives a Correct End of Run. BUT:
- although T2DRES and WACRES are created and merged correctly, they are not moved. They simply remain in the working directory.
- WACRBI is created in the working directory but is an empty file (0KB). Its 16 component files are listed but are all empty also (0KB).
- SISRES is NOT created but its 16 components SISRES00015-00000 to 00015 are listed, apparently correct, in the working directory.
- the working directory remains open at the end of the run.

So my final (hopefully!) query to you on this topic is whether you can understand why gretel is not doing its job (maybe because of WACRBI according to the error listing below??). The fact that Choice 2 is not working worries me here. One point that I notice below is that NUMBER OF PLANES is given as 0. Is this correct, I wonder??

I’ll be delighted if you can come up with some solution to this issue!!

With best regards, Alan

Error listing:
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
END OF TIME LOOP

EXITING MPI

CORRECT END OF RUN

ELAPSE TIME :
1 HOURS
5 MINUTES
30 SECONDS


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

... merging separated result files

+> NorthWales_t2d.cas
collecting: T2DRES
collecting: WACRES
collecting: WACRBI
runRecollection:
|runGRETEL: Could not split your file WACRBI (runcode=1) with the error as follows:
|
|... The following command failed for the reason above (or below)
|/app/environment/TELEMAC/v7p2r0/el6/AVX/intel-16.0/intel-5.1/builds/hpcw.ifortmpi/bin/gretel < gretel_WACRBI.par >> gretel_WACRBI.log
|
|
|Here is the log:
|
|
| +

---

+
|
| GRETEL: TELEMAC MERGER
|
|
|
| VERSION V7P2R0
|
| HOLGER WEIL BEER (BAW)
|
| JEAN-MICHEL HERVOUET (LNHE)
|
| YOANN AUDOUIN (LNHE)
|
| GRETEL (C) COPYRIGHT 2003-2012
|
| BUNDESANSTALT FUER WASSERBAU, KARLSRUHE
|
|
|
| +

---

+
|
|
|
|
|
| MAXIMUM NUMBER OF PARTITIONS: 100000
|
|
|
| +

---

+
|
|
|
| --GLOBAL GEOMETRY FILE:
|
| INPUT: WACGEO
|
| --GEOMETRY FILE FORMAT <FFORMAT> [MED,SERAFIN,SERAFIND]:
|
| INPUT: SERAFIN
|
| --RESULT FILE:
|
| INPUT: WACRBI
|
| --RESULT FILE FORMAT <FFORMAT> [MED,SERAFIN,SERAFIND]:
|
| INPUT: SERAFIN
|
|--NUMBER OF PARTITIONS <NPARTS> [2 -100000]:
|
| INPUT: 16
|
| --NUMBER OF PLANES:
|
| INPUT: 0
|
| ERREUR -1 LORS DE L APPEL A OPEN_MESH_SRF:READ
|
| TEXTE DE L'ERROR :
|
| UNKONWN ERROR
|
|
|
|
|
|
|
|
|
|
|
| PLANTE: PROGRAM STOPPED AFTER AN ERROR
|
| RETURNING EXIT CODE: 2
|

Hi Riadh,

A correction to what I wrote in the previous post:

At the end of the 3-way coupled continuation run (16 processors), the working directory remains open and includes files:

- T2DRES which is a correct merged solution
- SISRES00015 appears but in 16 parts for the subdomains. I have used gretel directly to merge this into SISRES which is then also a correct merged solution according to BlueKenue
- WACRES is created with the correct size and structure. BUT after the resumption timestep the merged file exhibits incorrect times and scrambled data. So WACRES is being corrupted somehow.
- WACRBI is also created in the working directory but is empty (0KB) as are its 16 components WACRBI00015.
So there is still a tomawac issue in ecrspe.f (and CALL P_SYNC) that remains unresolved for 3-way coupled continuation runs unfortunately.

Regards, Alan

Hello Alan

This looks like all results from Tomawac are corrupted. That's why gretel is unable to finish the job.

Did you observe any suspecting data in the listing of Tomawac during the computation?
what does it mean 'incorect timesteps and scrambled data'?

If I remember well, tomawac works well in standalone, is it ?
if it is the case, this means that there is some issue with the coupled version.

final question: do you use spherical coodinates ?

kind regards

Riadh

Hi Riadh,

In answer to your questions:

Yes, tomawac works fine in standalone mode.

When coupled with t2d alone, the initial solution is fine (for my coastal domain and also for the littoral test case). A continuation run (I have only tried this for littoral) then gives a correct solution APART FROM the global results file which is empty (0KB).

For continuation in 3-way coupling, what I mean by 'incorrect timesteps and scrambled data' is as follows:

- The solutions T2DRES and SISRES (merged manually) are both correct according to BlueKenue.

- When I view WACRES in BlueKenue, the solution is read in with the correct size and structure. The data and time at the resumption timestep are correct. However the data at all subsequent timesteps are incorrect and each timestep appears as t = 0 (even though the correct times appear for tomawac in the listing output file). At the first few timesteps following the resumption timestep, wave height HM0, for example, is zero throughout the domain. Then, for later timesteps, HM0 is non-zero but has no coherent pattern, just a randomised set of numbers as if the 16 domains have been shuffled together. Also, as said before, WACRBI is empty (0KB).

Finally, no, I do not use spherical coordinates.

I hope all this is clearer and thanks for your response!

All the best, Alan

Hello Alan

Just to unblock the situation of Gretel, you can just comment the keyword 'GLOBAL RESULT FILE' in the steering file of Tomawac.
Can you send me or to Thierry your case to check it up. Without the model it will be hard to investigate.

kind regards

Riadh

Hi Riadh,

For my continuation run (3-way coupled, 16 processors), I have now added wave outputs (HM0,TP,UWB) to the Sisyphe output (SISRES). These outputs are absolutely fine in the continuation solution, indicating that wave effects are being represented correctly by Sisyphe. In other words, the tomawac continuation simulation is running correctly, and it is simply the writing of this data to the tomawac output files (WACRES,WACRBI) that is going wrong for some reason.

Thank you for your offer to check my case. It will probably be best if I send you a stripped down version of the model (without bed roughness feedback etc) that will hopefully run at Chatou and focus on the tomawac output issue. I'll try to send you the files later this week, if possible.

Thanks again, Alan

Hi Riadh,

Following your last post, I attach a set of files that I ran here this morning on HPC Wales which exemplify the difficulty that I have been discussing concerning continuation runs for 3-way coupling T2D<>Tom<>Sis. The simulation is for 2 hours initially, followed by a continuation for a further two hours. With 16 processors each step has a run time of 11-12 min.

The cas files are set up for the initial run which terminates correctly for me using v7p2r0, including correct, merged wave output files.

For the continuation run, each cas file needs to have the initial run outputs replaced by the lines commented with /*/. The continuation runs give me a correct end of run. However while T2DRES and SISRES are both merged and correct, WACRES has the wrong size and content and WACRBI is empty.

I'll be very grateful if can try this example and spot what is going wrong. Although I haven't run this test using v7p2r3, past experience tells me that the same error would arise.

Best regards, Alan

Unfortunately it looks as if my grid .slf file is too large to send to the website (6.3MB). Can I send it to you directly? Alan

This sounds familiar. Can you check the WACRES files from the subdomains? I remember that I had a similar problem because with continued computations TOMAWAC tries to add the new results to the previous result file (starting at timestep N+1), whereas TELEMAC, SISYPHE (and Gretel) assume starting at time 1. Therefore, the first N steps of TOMAWAC are filled with garbage and Gretel can't complete the procedure.

Hi Michiel,

Yes, this is what is happening, I think. As I indicated in an earlier post, the first part of WACRES contains zeroes and the later part is scrambled data. WACRES has the wrong size. I have tried re-merging the subdomain parts using gretel directly, but that fails.

All the best, Alan