Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
Installation
Windows Version
Opentelemac and HPC Pack (MS-MPI)
Flat Threaded

TOPIC: Opentelemac and HPC Pack (MS-MPI)

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12025

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
In your last trial, try with this:
hpc_cmdexec: job submit /stdin:<wdir>\<hpc_stdin> <exename>

where <exename> should be replaced by the mpi command (becasue mpi is present in options)
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12026

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Again:
runcode::main:
:
   |runCode: Fail to run
   |job submit /stdin:\\path\twac_yfist.cas_2014-02-12-12h18min01s\HPC_STDIN <exename>
   |~~~~~~~~~~~~~~~~~~
   |The syntax of the command is incorrect.
   |~~~~~~~~~~~~~~~~~~
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12027

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
OK - sorry about this. This is puzzling.

Just in case, there is a difference, I am attaching the latest runcode.py -- can you try with this ?

What ever is wrong we will try to make sure it is fixed in the next release.

Thank you.
Sébastien.
Attachments:
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12031

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi
As I know, the python hpc script could modify the <HPC_STDIN> part of the systel config but it doesn't change anything in the command line except <HPC_STDIN>.
So it seems logical you have this error message in your case.

If you absolutely need to have the <exename> in your hpc_cmdexec, I think you should modify the runcode.py script to add this feature

Hope this helps
Christophe
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12032

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
I tried the latest script but it produced the same error as before. I will post if I manage to circumvent this problem. What I am thinking at the moment is to use the configuration file to 'create' or write a 'batch' file that would be executed in the hpc_cmdexec line to give the job submit command just as I would do within the directory.

The problem is that hpc_stdin does not have a .bat or .cmd extension and cannot be executed directly. I will look if I can overcome this problem through windows (rename?) or through the python script.

I will post if I find a solution. Thank you for your help.

Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12033

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
OK -- note that you can use the options --split and -x (to prepare the files) and --merge (to assemble after the job is completed).

You can also use (recommended) the option -w to set the directory to something fixed (as opposed to include the date/time)

Hope this helps.

Sébastien.
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12047

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello all,

I tried to fiddle with the runcode.py script but it's beyond my reach unless I take some python tutorials. :-) For the moment I have settled running jobs locally, which works fine. It would have been great to harness the capabilities of the Job Scheduler and Microsoft HPC Pack to distribute workload among the workstations of the cluster.

The good thing is that it is possible to compile opentelemac with mingw-64 and MS-MPI library. Development of MPICH2 has stopped for windows and Microsoft with MS-MPI has taken over. Having said that, I wouldn't recommend users to switch to MS-MPI unless they plan to set up a HPC Cluster by installing MS HPC Pack (which is free but requires a Server product to act as a headnode). Even then, it seems that telemac needs some adapting if it is to submit jobs seamlessly.

I would be happy to help, to the extends of my capabilities, with the development/testing of telemac to adapt to Microsoft HPC Pack.

Best Regards,
Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 7 months ago #12632

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello all,

After a brief introduction to python B), I was able to figure out the changes to the runcode.py script that would enable it to work with the job submit command of HPC Pack.

I inserted the following code after line 1050:
hpccmd = hpccmd.replace('<exename>',cfg['HPC']['exename'])

Now I can add the executable outside of the STDIN script, as required by MS HPC Pack, and the job submits successfully to the scheduler.

I hope this modification does not interfere with the rest functionality of the script and therefore you could add it to a future release. I will keep posting my findings regarding the compatibility on telemac and MS HPC Pack.

Regards,
Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 7 months ago #12643

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello,

After some more research I have found that I was looking in the wrong direction. It is much easier to keep the runcode.py as it is and use the hpc_stdin as an command prompt executable script. All that is required is to give it a script (.cmd) extension before execution:
hpc_cmdexec:  ren <wdir>\<hpc_stdin> <hpc_stdin>.cmd & <hpc_stdin>.cmd

The difference in functionality between Windows and Linux isn't that great after all (regarding job submission of course! B) ).

Regards,
Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 7 months ago #12644

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Hello,

I had started to look at your proposed changes and its implications -- I must admit, I like your latest offer much better !

This is briliant.

We will try to develop documentation around these script and HPC use in general.

As Président of the Consortium, thank you for helping us making the TELEMAC-MASCARET system a better tool for everyone.

Sébastien.
The administrator has disabled public write access.
Flat Threaded
Moderators: borisb
Forum
Installation
Windows Version
Opentelemac and HPC Pack (MS-MPI)

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.