TELEMAC-MASCARET Forum: Opentelemac and HPC Pack (MS-MPI) (6/6)

Opentelemac and HPC Pack (MS-MPI) 9 years 9 months ago #15788

sebourban OFFLINE Administrator Principal Scientist Posts: 814 Thank you received: 219	Can you show explain (show me the script files) how you submit your jobs ? Sébastien
	The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 9 months ago #15789

sebourban OFFLINE Administrator Principal Scientist Posts: 814 Thank you received: 219	... maybe of importance here : in our cases, runcode.py is the python submitting jobs on the queue ... w do not have any other scripts. ...
	The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 9 months ago #15790

cyamin
OFFLINE
openTELEMAC Guru
Posts: 997
Thank you received: 234

Here is the script I use to submit jobs to the queue:

hpc_stdin:	@echo off
			set telemac_conf_dir=\\atlas\Company\DataDisk\NET\PROGRAM_FILES\opentelemac\v7p0r0\configs
			set telemac_dir=\\atlas\Company\DataDisk\NET\PROGRAM_FILES\opentelemac\v7p0r0\scripts\Python27
			set python_dir=\\atlas\Company\DataDisk\NET\PROGRAM_FILES\WinPython-64bit-2.7.6.4\python-2.7.6.amd64
			for /f "usebackq tokens=4 delims=. " %%i in (`job new /jobtemplate:Telemac /jobname:<jobname> /numnodes:<ncnode> /corespernode:<nctile> /nodegroup:cnway`) do (
			job add %%i /name:"Code execution" /stdout:<wdir>\..\<jobname>_<time>_stdout.txt <mpi_cmdexec>
			job add %%i /name:"Recollection & Cleanup" /type:noderelease ^
			 pushd <wdir>\.. ^& ^
			 %python_dir%\Pythonw.exe %telemac_dir%\<codename>.py -c <cfgname> -w <wdir> --merge <casFileName> ^& ^
			 rd /s/q <wdir> ^& ^
			 cmd.exe <casFileName>.cmd
			job submit /id:%%i
			)
			exit
#
hpc_cmdexec:  ren <wdir>\<hpc_stdin> <hpc_stdin>.cmd & <hpc_stdin>.cmd

So far I was executing the runcode.py from my PC within the command prompt. I haven't tried executing the runcode.py from within the queue, if that is what you are suggesting.

Costas

The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 8 months ago #16311

cyamin OFFLINE openTELEMAC Guru Posts: 997 Thank you received: 234	Hello Sebastien, I am trying to see if I can submit jobs using python, using the <py_runcode> function. I have noticed that now the working directory is set where the cas file is and not the temp working directory. Hence, I cannot use <wdir> to get to hpc_stdin file that is placed in the temp. Why did the location of <wdir> changed? Regards, Costas
	The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 8 months ago #16335

sebourban
OFFLINE
Administrator
Principal Scientist
Posts: 814
Thank you received: 219

Hello,

You are the second person who tells me that -- so I have done the tests again today and confirmed that if I put it back to the way it used to be, not all possible combination of running TELEMAC on an HPC work. So, at this, I would rather try to understand why it fail on your system and maybe find a common ground.

My questions are:
Do you need to do pushd <wdir>\.. ?
Can you not do just this: pushd <wdir>

If you look into the systel.cis-hydra.cfg, you will find the HPC_STDIN as follows:

hpc_stdin: #!/bin/bash
   #PBS -S /bin/sh
   #PBS -o <sortiefile>
   #PBS -e <exename>.err
   #PBS -N <jobname>
   #PBS -l nodes=<nctile>:ppn=<ncnode>
   #PBS -q highp
   source /etc/profile.d/modules.sh
   module load gcc/4.7.2 openmpi/1.6.5/gcc/4.7.2 python/2.7.2
   PATH=$PATH:$HOME/bin:<root>/scripts/python27
   export PATH
   cd <wdir>
   <py_runcode>
   exit

reseting the current location to <widr> before calling on <py_runcode>

Does this helps ?
Sébastien.

The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 8 months ago #16346

cyamin
OFFLINE
openTELEMAC Guru
Posts: 997
Thank you received: 234

Hello Sebastien,

The problem in my case was that <wdir> was set one level up the temporary working directory (where the cas file is) while the HPC_STDIN file was put inside the temp work dir.

In my hpc_cmdexec line I am executing the HPC_STDIN batch file with this command:

hpc_cmdexec:  ren <wdir>\<hpc_stdin> <hpc_stdin>.cmd & <wdir>\<hpc_stdin>.cmd

After your post I did some more study and found out that, when using the python/hpc_runcode setup, <wdir> now points outside the temp directory and the current directory is inside the work temp dir.

So <wdir> is not required anymore in the hpc_runcode, and problem is solved:

hpc_runcode:  ren <hpc_stdin> <hpc_stdin>.cmd & <hpc_stdin>.cmd

Best Regards,
Costas

The administrator has disabled public write access.

Opentelemac and HPC Pack (MS-MPI) 9 years 8 months ago #16347

sebourban OFFLINE Administrator Principal Scientist Posts: 814 Thank you received: 219	Very good -- happy to hear that -- sorry for this little change. It will make things more generic for all the other possible configuration. Thanks, Sébastien.
	The administrator has disabled public write access.