Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
Installation
Windows Version
Error submitting job to HPC scheduler [python3-trunk]
Flat Threaded

TOPIC: Error submitting job to HPC scheduler [python3-trunk]

Error submitting job to HPC scheduler [python3-trunk] 5 years 7 months ago #33408

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello,

Further exploiting python3 script functionality, I encountered the following problem while submitting jobs using HPC functionality. Python27 scripts work fine.

Here is the output from the TOMAWAC test case friction:
STOP 0


Loading Options and Configurations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                _                                                        _       _               
               | |                                                      (_)     (_)              
  _   _  _ __  | | __ _ __    ___  __      __ _ __    _ __   ___ __   __ _  ___  _   ___   _ __  
 | | | || '_ \ | |/ /| '_ \  / _ \ \ \ /\ / /| '_ \  | '__| / _ \\ \ / /| |/ __|| | / _ \ | '_ \ 
 | |_| || | | ||   < | | | || (_) | \ V  V / | | | | | |   |  __/ \ V / | |\__ \| || (_) || | | |
  \__,_||_| |_||_|\_\|_| |_| \___/   \_/\_/  |_| |_| |_|    \___|  \_/  |_||___/|_| \___/ |_| |_|
                                                                                                 
                                                                                                 


Running your CAS file(s) for:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

gfmsmpi: 
    
    +> root:    \\hydra\opentelemac\trunk
    +> module: ad / api / artemis / bief
               damocles  / diffsel  / gaia  / gretel
               hermes  / identify_liq_bnd  / khione  / mascaret
               nestor  / parallel  / partel  / postel3d
               sisyphe  / special  / splitsel  / stbtel
               telemac2d  / telemac3d  / tomawac  / waqtel
               


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


... processing the steering file

... checking parallelisation
         copying: tom_friction.cas -> O:\trunk\examples\tomawac\bottom_friction\tom_friction.cas_2019-04-09-13h28min37s\WACCAS
         copying: tomawac.dico -> O:\trunk\examples\tomawac\bottom_friction\tom_friction.cas_2019-04-09-13h28min37s\WACDICO
         copying: geo_frot.slf -> O:\trunk\examples\tomawac\bottom_friction\tom_friction.cas_2019-04-09-13h28min37s\WACGEO
         copying: geo_frot.cli -> O:\trunk\examples\tomawac\bottom_friction\tom_friction.cas_2019-04-09-13h28min37s\WACCLI

... partitioning base files (geo, conlim, sections, zones and weirs)
     +> \\hydra\opentelemac\trunk\builds\gfmsmpi\bin\partel.exe < partel_WACGEO.par >> partel_WACGEO.log

... splitting / copying other input files

... checking the executable
Traceback (most recent call last):
  File "\\hydra\opentelemac\trunk\scripts\python3\tomawac.py", line 7, in <module>
    main('tomawac')
  File "\\hydra\opentelemac\trunk\scripts\python3\runcode.py", line 275, in main
    run_study(cas_file, code_name, options)
  File "\\hydra\opentelemac\trunk\scripts\python3\execution\run_cas.py", line 142, in run_study
    run_hpc_cas(my_study, options)
  File "\\hydra\opentelemac\trunk\scripts\python3\execution\run_cas.py", line 116, in run_hpc_cas
    my_study.compile_exe(options.bypass)
TypeError: compile_exe() takes 1 positional argument but 2 were given

Any help would be appreciated.

Best Regards,
Costas
The administrator has disabled public write access.

Error submitting job to HPC scheduler [python3-trunk] 5 years 7 months ago #33409

  • yugi
  • yugi's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 851
  • Thank you received: 244
Hi,

Thanks for your determination to test it :)
What command did you type ?
What kind of hpc configuration do you have ?
The one where the whole script is run on the node or the one were only the executable is run on the node ?
There are 10 types of people in the world: those who understand binary, and those who don't.
The administrator has disabled public write access.

Error submitting job to HPC scheduler [python3-trunk] 5 years 7 months ago #33410

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
My configuration is rather unique, that is I am using HPC Pack on a Windows Server. My old setup is tested and working for quite some time using python27 scripts.
My command was:
tomawac.py -f \\hydra\opentelemac\trunk\configs\systel_hpc.cfg -c gfmsmpi --ncnode=1 --nctile=6 --jobname=tom_friction tom_friction.cas
Job submission is carried out on my node/workstation, but recollection of results is carried out on the Head node (haven't been able to test it yet, because it requires some modification on the 'study.py\fill_hpc_stdin'). The submission script looks something like this (I hope you get the gist):
hpc_stdin:	@echo off
            set telemac_root=\\hydra\opentelemac\trunk
			set telemac_conf_dir=%telemac_root%\configs
			set telemac_dir=%telemac_root%\scripts\python3
			set python_dir=\\rackstation\DataDisk\NET\PROGRAM_FILES\WPy-3710\python-3.7.1.amd64
			for /f "usebackq tokens=6 delims=. " %%i in (`job new /faildependenttasks /jobtemplate:EPYC /jobname:<jobname> /nodegroup:EPYC`) do set jobid=%%i
			job add %jobid% /name:"Code execution" /stdout:<wdir>\..\<jobname>_<time>_stdout.txt <mpi_cmdexec>
			job add %jobid% /name:"Recollection (merging)" /type:NodeRelease ^
			 pushd <cdir> ^& ^
			 %python_dir%\Pythonw.exe %telemac_dir%\<codename>.py -f <cfgfile> -r %telemac_root% -c <configName> -w <wdir> --ncsize=<ncsize> --merge <casfile>
            job add %jobid% /name:"Cleanup" /type:NodeRelease pushd <cdir> & rd /s/q <wdir>
			job submit /id:%jobid%
			exit
#
hpc_cmdexec:  ren <wdir>\<hpc_stdin> <hpc_stdin>.cmd & <hpc_stdin>.cmd
The administrator has disabled public write access.

Error submitting job to HPC scheduler [python3-trunk] 5 years 7 months ago #33419

  • yugi
  • yugi's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 851
  • Thank you received: 244
Hi,

I just committed the correction.
There was indeed a syntax error and also the --merge option was not handled properly when using hpc_cmdexec.

Tell me if that solves your problem.
There are 10 types of people in the world: those who understand binary, and those who don't.
The administrator has disabled public write access.
The following user(s) said Thank You: cyamin

Error submitting job to HPC scheduler [python3-trunk] 5 years 7 months ago #33420

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Just did a quick test and indeed the problem is solved. Recollection also works, so functionality is restored. B)
The administrator has disabled public write access.
Flat Threaded
Moderators: borisb
Forum
Installation
Windows Version
Error submitting job to HPC scheduler [python3-trunk]

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.