Welcome, Guest
Username: Password: Remember me

TOPIC: Parallel mode - assignment of processors

Parallel mode - assignment of processors 13 years 5 days ago #3074

  • olslewfoot
  • olslewfoot's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 132
  • Thank you received: 3
Dear all

Is there a way to configure mpi_telmac.conf so that only one processor on the headnode is used with the bulk of the computaion on the compute nodes.
I list the headnode and compute node hosts in my mpi_telemac.conf with their respective processor numbers. In the steering file I list the number of processors totalled on the compute nodes.

Telemac also uses several of the processors on the headnode but I would prefer to only use one, so it is not slowed. When I change the value of processors in mpi_telemac.conf the simulation hangs.
Any suggestions would be appreciated.

Regards
John
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 13 years 5 days ago #3076

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Hello John,

You seem to have the right technique (your setup of the mpi_telemac.conf with one associated to your head node, and the list of the other nodes). Since you use this file, I presume you are using the Perl version.

A few pointer to check your system:
- in your systel.ini, make sure that you have the correct RUN_MPI (i.e. you do not want to see -localonly (or otherwise everything will be on your headnode)
- in your mpi_telemac.conf, sometimes it helps to have only one line per processor:
headnode 1
slave 1
slave 1
slave 1

- make sure you have a shared drive mapped (say letter T) on all computers and that you run from that T:> rather than from C:> or D:> ...

Hope this helps,
Sébastien.
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 13 years 4 days ago #3081

  • olslewfoot
  • olslewfoot's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 132
  • Thank you received: 3
Hi Sebastien
Thanks for your reply.
I do have my system set up as you suggest and RUN_MPI does not limit the run to -localonly.
One question - you suggest using one line per processor in the conf file. If I have 8 processors per host, do you mean listing the host 8 times with 1 processor each?
I currently list numbers of processors per host and the overall total at the head of the file equals the number on the slaves only. This matches the value used in the steering file.
I've attached an example of my current mpi_telemac.conf (renamed for transmission) which works fine, but it also uses half the processors on the headnode, which I'm trying to avoid. If I only list 1 processor on the headnode the system hangs.
Thanks
John
Attachments:
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 13 years 4 days ago #3093

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi
As i understand your request, the correct configuration of mpi_telemac.conf is:
deepblue.corp.cefas.co.uk  1
n17	8
n18	8
n19	8
n20	8
But you mention that the computation hangs so that's really strange.
Could you give us more inforamtion about the computation state?

regards
Christophe
The administrator has disabled public write access.
The following user(s) said Thank You: olslewfoot

Re: Parallel mode - assignment of processors 13 years 1 day ago #3099

  • olslewfoot
  • olslewfoot's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 132
  • Thank you received: 3
Hi Christophe

I think we had a network issue which made it hang. Using the configuarion below works fine now.
Thank-you.

John
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 12 years 11 months ago #3212

  • olslewfoot
  • olslewfoot's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 132
  • Thank you received: 3
Hi Christophe
When I have set the mpi_telemac file as shown in your post (with only 1 processor called on the headnode and 8 on each slave node), when the programme runs there are 7 processors taken by telemac on the headnode and 6 on each slave node.
Can you suggest why this is? I need to reduce the number of processors used on the headnode as when other users are working, the headnode is working at a much higher capacity. This would not occur if telemac used the 1 processer called?
Also, I am using mpdboot to establish an mpi ring with the hosts listed in mpd.hosts.
John
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 12 years 11 months ago #3213

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi John
This strance way of running for the telemac computation is not directly linked to Telemac itself (in my opinion).
It's probably due to the mpi ring managed by mpdboot, but i'm not too familiar with this to give you the exact answer.
Nevertheless, I'm curious to know if the order of hosts in mpi_telemac.conf could play a role in this problem.
Maybe you could try to move your head node at the end of the list to check the result.
Another possibility (as sebastien mentionned it) is to list the host with each time one processor.
n17    8
could be replace by
n17    1
n17    1
n17    1
n17    1
n17    1
n17    1
n17    1
n17    1

Hope this helps.
Christophe
The administrator has disabled public write access.

Re: Parallel mode - assignment of processors 12 years 11 months ago #3222

  • olslewfoot
  • olslewfoot's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 132
  • Thank you received: 3
Hi Christophe

I made those changes but the problem remained.

However - success :) ! I added the number of processors in the mpd.hosts file (used as default by mpirun) with a suffix.

headnode:1
slavenode1:8
slavenode2:8
etc...

This works with my original mpi_telemc.conf file.

I now have only one processor assigned and woking on the headnode.
Thanks for your help.
John
The administrator has disabled public write access.
Moderators: borisb

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.