open TELEMAC-MASCARET The mathematically superior suite of solvers

Dear all,

I'm trying to run a simple parallel simulation where there are uncovered beds in the domain and I impose a tracer at the boundary domain. When I force the advection scheme for tracers =1 my simulation runs till the end but I have a considerable "mass" loss for the tracer. Therefore I changed the advection scheme for tracers =14 (for tidal flats), the simulation starts but after some time steps I get the following error:

(...)
runcode::main:
:
|runCode: Fail to run
|/opt/openmpi/intel/64/1.6.3/bin/mpiexec -wdir /projects/13_131_bevaarbozs/TELEMAC/Lippen_tr/Lippenbroek_tr_exp.cas_2014-07-17-12h18min18s -n 8 /projects/13_131_bevaarbozs/TELEMAC/Lippen_tr/Lippenbroek_tr_exp.cas_2014-07-17-12h18min18s/out_Lip_tracers5
|~~~~~~~~~~~~~~~~~~
|2
|forrtl: error (78): process killed (SIGTERM)
|Image PC Routine Line Source
|libmpi.so.1 00002AAAAADA101B Unknown Unknown Unknown
|libmpi.so.1 00002AAAAAF421D7 Unknown Unknown Unknown
|libmpi.so.1 00002AAAAAD64416 Unknown Unknown Unknown
|libmpi.so.1 00002AAAAAD8B0B8 Unknown Unknown Unknown
|libmpi_f77.so.1 00002AAAAB7FB705 Unknown Unknown Unknown
|out_Lip_tracers5 0000000000981388 Unknown Unknown Unknown
|out_Lip_tracers5 0000000000712E2D Unknown Unknown Unknown
|out_Lip_tracers5 000000000071534D Unknown Unknown Unknown
|out_Lip_tracers5 00000000006B065C Unknown Unknown Unknown
|out_Lip_tracers5 00000000004A0B61 Unknown Unknown Unknown
|out_Lip_tracers5 0000000000458C18 Unknown Unknown Unknown
|out_Lip_tracers5 00000000004BA82C Unknown Unknown Unknown
|out_Lip_tracers5 000000000043893C Unknown Unknown Unknown
|libc.so.6 00002AAAACF9FCDD Unknown Unknown Unknown
|out_Lip_tracers5 00000000004387C9 Unknown Unknown Unknown
(...)
|mpiexec has exited due to process rank 3 with PID 18989 on
|node eul02 exiting improperly. There are two reasons this could occur:
|
|1. this process did not call "init" before exiting, but others in
|the job did. This can cause a job to hang indefinitely while it waits
|for all processes to call "init". By rule, if one process calls "init",
|then ALL processes must call "init" prior to termination.
|
|2. this process called "init", but exited without calling "finalize".
|By rule, all processes that call "init" MUST call "finalize" prior to
|exiting or it will be considered an "abnormal termination"
|
|This may have caused other processes in the application to be
|terminated by signals sent by mpiexec (as reported here).

Does anyone have any idea why I'm getting this error?

Thank you in advance!

Kind regards,
Maria

Hello,

With tidal flats and this scheme you must have TREATMENT OF NEGATIVE DEPTHS = 2, could it be this ? (but normally you should get a message for this I think, not checked...).

Otherwise any crash could produce such a message, but there are more data (hopefully) in the log files in the temporary folder which is kept.

Regards,

JMH

Hello Jean-Michel,

I had already the keyword TREATMENT OF NEGATIVE DEPTHS = 2. I attach the log file and cas file regarding that run (output23.out and Lippenbroek_tr_exp.cas). I was now trying to run the simulation in serial mode and I got a different error:

CALCOT: PLANES

1 AND 2
INTERCROSS AT POINT 1110
LOWER POINT : 6.00000000000000
HIGHER POINT: 5.99993612246292
DIFFERENCE : -6.387753708203547E-005
DEPTH : -2.555101483284677E-004

A problem with crushed elements? If so would you know how to solve this?

thank you!

Regards,
Maria

Hello,

Do you see why you have somewhere a negative depth, which should not happen? Could it be in the initial conditions, or due to a source or a culvert? For example we could imagine pumping on a tidal flat that would not be secured.

If not I would need the whole case (and some time...).

Regards,

Jean-Michel

Dear Jean-Michel,

You are right. In the last months I have been working with the t3d_debsce subroutine to model flow through culverts in Telemac-3D. I use a pair of sources/sink terms that reproduce the flow coming from a part of domain to another part depending on the water levels in both areas. Until now I did not have tracers included and everything was working. Now when I introduce the tracers and I change the advection scheme for tracers=14 I get the error described above.

After your last post I deactivated the flow that goes through the culvert and I maintained the tracers with the advection scheme= 14. The model runs. So the problem is indeed in the source/sink terms that I'm modelling..The question is why do I get an error when I include tracers (for the advection scheme=14 and not for advection scheme for tracers=1)? Why do I get negative depths with tracers + sources/sinks? Do you have an idea?

If not, and if you have some time I could send you the changed sub-routine t3d_debsce.f+telemac3d.f and I would be really appreciated!

Kind regards,
Maria

Hello,

The scheme 14 really needs to rely on positive depths, it may be that your formulas of flows pumps on shallow or dry bed, in which case the flow should be stopped with some threshold value of water depth.

Regards,

JM

Dear Jean-Michel,

Thank you a lot for your help. I have included a threshold value for water depth in the culvert functionality, but I still get a really small negative depth (DEPTH=-4.005450776629687E-010) that I'm not able to avoid it...why do i get the error after the call of "correction_depth_3d.f'? This subroutine should filter all the negative depths, or not? (As I'm using the scheme for advection of tracers=14 I have the keyword TREATMENT FOR NEGATIVE DEPTHS=2). A way of bypassing this error is to clip the water depth to a minimum value of zero after the call of correction_depth_3d.f...but I really do not know if I'm "allowed" to do that.

Kind regards,
Maria

Hello,

For such small values the clipping is OK and will not show in the mass balance. However it means that something allows some truncation error that gives negative depths in correction_depth_3d.f, which is in fact in positive_depths.f, which could be due to some unusual source term like evaporation or something like that, but not in the main part of the algorithm. Anyway I would need your case to investigate this.

With best regards,

Jean-Michel

Dear Jean-Michel,

I really appreciate all the help you are giving me. Please find attached my test case.

Thank you!



Have a nice weekend,
Maria

Hello,

I'm running a simple test case where I deal with tidal flats and tracers (I attach the .cas file). In order to ensure a better mass conservation for tracers I chose the scheme for advection for tracers =14 together with the treatment of negative depths=2. When I run the test case in serial everything goes fine and I get a good mass conservation for water and tracers (please see listing output: output_ser). When I increase the number of processors, the water mass conservation remains the same but not the "mass" conservation of tracers that gets worse (please see listing output: output_par).

Any idea why the conservation of tracers gets worse in parallel? And if so, how I can I correct this?

Kind regards,
Maria