Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
Installation
Windows Version
Opentelemac and HPC Pack (MS-MPI)
Flat Threaded

TOPIC: Opentelemac and HPC Pack (MS-MPI)

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11990

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
I tried adding the <wdir> but again it failed with a similar error:
'hpc_stdin' is not recognized as an internal or external command,
operable program or batch file

If I wanted to define a standard input file to the microsoft job scheduler I would have to type:
hpc_cmdexec:   job submit /stdin:<hpc_stdin>
However, when I try that, I get this message:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

_____________
runcode::main:
:
   |runCode: Fail to run
   |job submit /stdin:HPC_STDIN
   |~~~~~~~~~~~~~~~~~~
   |The Job command did not recognize a command line. Only a RunUntilCanceled job can be submitted without a command line.
   |~~~~~~~~~~~~~~~~~~
rununtilcanceled is an option for the job submit command.

To me it looks like the job is submitted to the Job Scheduler BUT the hpc_stdin file with all the parameters is not passed during the submit command. I think the problem is with the configuration file but I don't fully understand the HPC structure in order to troubleshoot. :(
Where can I read more about the usage/syntax of the .cfg file?

Regards,
Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11991

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Another thing I have notices in the configuration files you have posted is that the hpc_stdin: line has a script syntax. Does telemac expect a script/batch file to be associated with hpc_stdin and not a set of options/variables like in my configuration. In the temporary working directory there is a hpc_stdin file (with no extension) that contains everything I specify in the hpc_stdin: line. How does telemac calls this file?

Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11992

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Unfortunately, there is no documentation of this feature yet -- we are planning some for the next release. All I can say is to look at the python script itself.

However, from experience and what I see of your case, I believe the python script is correct and has created the file HPC_STDIN within the temporary directory (have a look at it).

So I believe the problem is with the job scheduler not understanding the content of the file HPC_STDIN, or not reading it properly.

To confirm what I am saying, cd yourself into the temporary directory, verify that the file HPC_STDIN is there, and run manually the scheduler command (without any python):
job submit /stdin:HPC_STDIN

Once this is solved, then you can try the standard python command.
Note: the HPC_STDIN files I showed you included a command at the beginning of the file:
#!/bin/bash
This helps defining the directives for the scheduler -- I am guessing you may need such directive but for your scheduler.

Maybe you can find somewhere on the web an example of such file that would work with your scheduler --

I think you are very close to make this work.

Hope this helps,
Sébastien.
The administrator has disabled public write access.
The following user(s) said Thank You: cyamin

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11994

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Dear Sebastien,
I will work it in the temporary directory and follow the python scripts. I also think that I am close, I am just missing a little something... Thank you for your help.

Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11993

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Try adding this /workdir:<wdir> to your command (is you have not done this already)
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #11995

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
I have checked /stdin: option in your job submit command -- it is not what we think it is ... but rather specify input files for the executable (in your case mpiexec).

So first try within the temporary directory:

job submit /workdir:<wdir> ...... and the full command

then has a hpc_cmdexec: command in your config:

cd <wdir>: job submit /workdir:<wdir> ...... and the full command

Hope this helps.
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12011

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello Sebastien,

I have tried executing the this command within the working directory:
job submit /workdir:\\path\twac_yfist.cas_2014-02-11-14h49min43s /stdin:hpc_stdin mpiexec.exe /cores 4 out_tomawac.exe
It submits the job successfully.

When I tried to change into the working directory within the configuration file (as you said) I got this error which is due to command prompt limitation with UNC paths.
CMD does not support UNC paths as current directories.

So far, I know I can submit jobs manually, the tricky part is to do it within the python script. I tried to studied the runcode.py script but I couldn't figure out how hpc_cmdexec works.
Moreover, when I use this command:
pc_cmdexec: job submit /stdin:<wdir>\<hpc_st[quote]The syntax of the command is incorrect.[/quote]din> <mpi_cmdexec>
I get:
runcode::main:
:
   |runCode: Fail to run
   |job submit /stdin:\\path\twac_yfist.cas_2014-02-11-15h33min40s\HPC_STDIN <mpi_cmdexec>
   |~~~~~~~~~~~~~~~~~~
   |The syntax of the command is incorrect.
   |~~~~~~~~~~~~~~~~~~
Any ideas? :(
Regards,
Costas
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12013

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
try this:
hpc_cmdexec: job submit /workdir:<wdir> /stdin:<hpc_stdin> <mpi_cmdexec>
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12014

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
I am sure I have tried it at some point... Here is the outcome:
runcode::main:
:
   |runCode: Fail to run
   |job submit /workdir:\\path\twac_yfist.cas_2014-02-11-16h05min44s /stdin:HPC_STDIN <mpi_cmdexec>
   |~~~~~~~~~~~~~~~~~~
   |The syntax of the command is incorrect.
   |~~~~~~~~~~~~~~~~~~
The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 10 years 9 months ago #12023

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
OK, after some researching, here where I think the problem lies:

The 'job submit' command expects the program executable to be defined in the same line but not within the standard input file (i.e hpc_stdin):
#
par_cmdexec:   <config>\partel.exe < PARTEL.PAR >> <partel.log>
#
#mpi_hosts:   
mpi_cmdexec: mpiexec.exe /wdir <wdir> /cores <ncsize> <exename>
hpc_stdin:   /numcores:<ncsize> /workdir:<wdir> /jobname:<jobname> /stdout:hpc_stdout.txt /stderr:hpc_stderr.txt <mpi_cmdexec>
#
hpc_cmdexec: job submit /stdin:<wdir>\<hpc_stdin>
'Job submit' detects that there is no executable present and exits without going through the hpc_stdin and gives this message:
runcode::main:
:
   |runCode: Fail to run
   |job submit /stdin:\\path\twac_yfist.cas_2014-02-12-10h31min27s\HPC_STDIN
   |~~~~~~~~~~~~~~~~~~
   |The Job command did not recognize a command line. Only a RunUntilCanceled job can be submitted without a command line.
   |~~~~~~~~~~~~~~~~~~

Now, if I want add the executable to the job submit command (hpc_cmdexec) like this:
par_cmdexec:   <config>\partel.exe < PARTEL.PAR >> <partel.log>
#
#mpi_hosts:   
mpi_cmdexec: mpiexec.exe /wdir <wdir> /cores <ncsize> <exename>
hpc_stdin:   /numcores:<ncsize> /workdir:<wdir> /jobname:<jobname> /stdout:hpc_stdout.txt /stderr:hpc_stderr.txt
#
hpc_cmdexec: job submit /stdin:<wdir>\<hpc_stdin> <mpi_cmdexec>
Then I get the following error:
runcode::main:
:
   |runCode: Fail to run
   |job submit /stdin:\\path\twac_yfist.cas_2014-02-12-11h47min35s\HPC_STDIN <mpi_cmdexec>
   |~~~~~~~~~~~~~~~~~~
   |The syntax of the command is incorrect.
   |~~~~~~~~~~~~~~~~~~
The problem seems to be related to the python script/configuration file, as I am able to submit the job from the temporary working directory.

Any help would be appreciated.
Costas
The administrator has disabled public write access.
Flat Threaded
Moderators: borisb
Forum
Installation
Windows Version
Opentelemac and HPC Pack (MS-MPI)

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.