open TELEMAC-MASCARET The mathematically superior suite of solvers

Hello,

Always happy to be of assistance. Following this discovery, I am exploiting the full potential of the job submission routine within telemac and will post my findings when complete.

At the moment, I am looking into the "noderelease" function of HPC Pack, which is very useful to automatically recollect the results when the computation has completed. In order for this task to be submitted to the job scheduler, the command needs to be specified within the batch file ('<hpc_stdin>' line).

The only concern is to get the name and location (outside of <wdir> unfortunately) of the casFile required by the 'merge' option to work. What are the variables within the runcode.py that hold this information and possibly use aline to get?

Regards,
Costas

Hello,

<wdir> is the only directory available I afraid (not that we would not be able to make others available).

Also, within your STDIN script, you should already be placed where the root CAS file is, unless you did cd <wdir>, in which case you can simply do cd ..

Therefore runcode.py <codename> --merge casFile should work.

Let me know if you have problems.

Another option is to call on runSELAFIN.py to merge all files into one within <wdir> (but then you need to know what are the file name T2DRES etc ..., which does not make the code generic.

Sébastien.

Hello Sebastien,

I have managed to submit jobs and recollect results automatically in one go . However, two issues remain if the solution is to be generic:

Firstly, in the particular investigation, I am using a parallel case of Artemis (BTWI). When I use the runcode.py --merge command, I get the following message:This is overcome by using artemis.py --merge.

Secondly, I still have to define explicitly the file name of the .cas file within the <hpc_stdin> line in the systel.cfg.

To my understanding, both issues would require some modification of the runcode.py script.

Costas

Hello Sebastien,

In the runcode.py script, is there variable that holds the name of the .cas file currently in process? Maybe I could modify the STDIN file creation part of the runcode.py so as to specify it in my submission script.

Regards,
Costas

Hello,

After some more research, I found that it is not "casFile" (as initially thought) that holds the filename, but "casName" instead.

I inserted:after line 1070 in runcode.py and now I can get the .cas filename in my STDIN script.

Now, all that is left to make everything work, is the runcode.py-artemis.py --merge issue that I mentioned before.

Costas

OK --

try the following as your command:

as your merging command, within your STDIN, after your run command, and check what is written as a result in your local HPC_STDIN file.


Sébastien.

I tried and it is left as it is, i.e. <py_runcode>.

Costas

OK, problem solved. I can submit jobs to MS HPC Pack, recollect result files, all in one generic job submission command. Job submission does not require changes to runcode.py, automatic recollection does. I don't see why this recollection command can't be added in a similar fashion to other job scheduler scripts (if they have a similar function to HPC's node release task). So you may want to consider adding this functionality.

The command for recollection is:The changes needed to be made in runcode.py script are 2 lines after 1070:I haven't yet tried to submit multiple cas files in a single batch job, which might cause a problem with 'casName'.

HPC Pack provides numerous possibilities to meet the needs of the user. I will keep exploiting the HPC Pack commands to tailor the job submission script according to the runcode.py structure. I will be able to provide a more "mature" HPC configuration after I refine it in actual 'production' projects.

Regards,
Costas

Hello,

I am trying to submit to the HPC Pack queue, multiple jobs that are sequential, i.e. they are continued computation of the previous job. If I submit all the jobs at once, the working directories are created and populated before the previous computation is completed and the results files made available. Thus the job submission for the next computations fails.

Is there a way to overcome this problem? How to you address this issue with other job schedulers?

Best Regards,
Costas

Hi,

You should be able to build a dependencies between jobs with your job scheduler. Here is an example with slurm:This command will launc the jobB when the jobA finished properly.

Hope it helps.