Welcome, Guest
Username: Password: Remember me

TOPIC: HPC and recollection from queue

HPC and recollection from queue 9 years 1 month ago #18656

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello,

So far I have been recollecting the results from within the HPC queue by using the cas file outside the working directory. This means that I have to specify the PARALLEL PROCESSORS key in every cas file (although this value is derived from the job submission parameters, ncnode, nctile) just for gretel to read during the recollection command.

Is it possible to recollect within the queue, using the cas file within the working directory that includes the derived PARALLEL PROCESSORS key? That would make thing more generic and a lot easier.

Best Regards,
Costas
The administrator has disabled public write access.

HPC and recollection from queue 9 years 1 week ago #18858

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Hello,

If you use the --ncsize key (in addition to --nctile or--ncnode), it adds the keyword to the CAS file automatically.

Also, if you wish the merge to be completed on the queue, I recommend that you do everything on the queue, i.e. instead of running mpiexec on the queue, you atually run the python script. You can still prepare for the split ahead of running the job using -w mydirectory. This choice / feature is included in the python -- all you have to do is setup your configuration wit the appropriate parameters. I strongly recommend you have a look at the setup used in the configuration file systel.cis-hydra.cfg. It includes 5 type of launch, two of which running the python with the queue, for a parallel or even a scalar job (run on the queue).

Hope this helps,

Sébastien.
The administrator has disabled public write access.

HPC and recollection from queue 8 years 8 months ago #20170

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello Sebastien,

The solution to my problem was very simple :blush: . Adding --ncsize <ncsize> to the recollection (merge) command did the trick. <ncsize> is automatically derived from the <ncnode> and <nctile> parameters given in the mpiexec command.

The only thing left to master the HPC Pack queuing system, is to run the python script from the queue, which is in my todo list if the shortcoming of python and UNC paths in windows is overcome.

Best Regards,
Costas
The administrator has disabled public write access.
Moderators: borisb

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.