Welcome, Guest
Username: Password: Remember me

TOPIC: Running on HPC with dependency between jobs

Running on HPC with dependency between jobs 1 year 1 month ago #43648

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3729
  • Thank you received: 1032
Hi
I've seen in the configuration and script the possibility to manage dependency between different jobs on hpc but cannot find explanation nor exemple on how to deal with this option...
What should we specify in the command line to specify a dependency to another job?
Regards
Christophe
The administrator has disabled public write access.

Running on HPC with dependency between jobs 1 year 1 month ago #43653

  • Renault
  • Renault's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 121
  • Thank you received: 33
Hi Christophe,

I'm not sure this will help, but when I am running Telemac jobs on Digital Alliance Canada's servers and have dependencies, I use something like:
sbatch --dependency:afterok $JOBNUMBER sbatch.sh

Their scheduler is Slurm, so mileage will vary.
Of course, I've probably misunderstood your question, because I'm assuming you're looking for something more automated.

André Renault
The administrator has disabled public write access.

Running on HPC with dependency between jobs 1 year 1 month ago #43655

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3729
  • Thank you received: 1032
Hi André

My question was probably not so clear :unsure:
I run my hpc computation directly from the command line by specifying the parameters.
For example: python telemac2d.py --ncsize 128 --queue XXX --jobname JJJ cas_file
This was in this particular context I was looking for the dependency option.
I saw the get_hpc_depend function in study.py but don't understant how it works and the associated changes needed in the config file...

Christophe
Christophe
The administrator has disabled public write access.

Running on HPC with dependency between jobs 1 year 1 month ago #43660

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
Looking at the current code, there is no way to specify a job id in command line.
run_hpc_full can accept an optionnal parameter "job_id" and set appropriate slurm parameter with hpc_depend: but runcode.py does not provide argument to pass the job id.

Seems quite easy to implement, I could look into it.
The administrator has disabled public write access.

Running on HPC with dependency between jobs 1 year 1 month ago #43663

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3729
  • Thank you received: 1032
Hi Nicolas
I've the same opinion But prefer to ask others users or developers to confirm...

Regards
Christophe
The administrator has disabled public write access.
Moderators: borisb

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.