Welcome, Guest
Username: Password: Remember me

TOPIC: Working parallel installation using python/gfortran in Windows

Working parallel installation using python/gfortran in Windows 11 years 6 months ago #8601

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello all,

Today I have managed to run some validation cases in parallel using gfortran in Windows :woohoo: . I share my configuration so that others can test/verify this.

MinGW-w64 gfortran in use (for windows x64 versions, builds 64bit binaries):
gcc-4.8.0-release

Pre-compiled METIS library for use with gfortran 64bit:
sourceforge.net/projects/mingw-w64/files...ies/scientist/metis/
METIS-5.0.2.7z

Configuration file:

File Attachment:

File Name: systel-gfortran.cfg.txt
File Size: 2 KB


I hope it works for others too.
Regards,
Costas
The administrator has disabled public write access.
The following user(s) said Thank You: konsonaut, pham

Working parallel installation using python/gfortran in Windows 11 years 6 months ago #8667

  • konsonaut
  • konsonaut's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 413
  • Thank you received: 144
Hello Costas,

thank you very much, I appreciate your work a lot!
This procedure could be a simple way for a Win/Gfortran/Parallel compilation.

I compiled Telemac on Win 7 using your settings and it worked fine (I was very happy :laugh: ), so it built all the objects and the executables.

However when launching a simulation, scalar or parallel, I get a popup with the error message:
"The procedure entry point ... could not be located in the dynamic link library libgfortran-3.dll"

Before the compilation I deleted all my existing Gfortran installations and environment variables and installed the Gfortran provided by you.

Apparently it is a Gfortran problem and maybe you have other Gfortran libraries concurrently on your pc?

I would be very glad for any hints.
Regards,
Clemens
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 6 months ago #8669

  • konsonaut
  • konsonaut's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 413
  • Thank you received: 144
Hello,

I could solve the problem.

I had some fortran libraries in my strawberry perl directory from former Telemac installations. I don't know why. And in the environment variable PATH these perl directories are listed before the new installed Gfortran directories. Deleting these entries or switching to the end of the path list solves the problem.

I will test the parallel mode for some case studies and give some feedback in the next time.

Best regards,
Clemens
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 6 months ago #8694

  • konsonaut
  • konsonaut's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 413
  • Thank you received: 144
Hello,

I tested two case studies on my Win7 64 bit local machine with Gfortran, 2*Intel Pentium D 3.00 GHz, in scalar and parallel mode (2 cores).
Spillway with 15000 elements and Malpasset Large validation case with 104000 elements.

In the parallel mode the computation times in both cases are approximately 40% lower. :laugh:

I used the GCC 4.7.1 compiler suite from tdm-gcc.tdragon.net/ which has a user friendly installer. During the installation one has to activate the button to install the Gfortran compiler additionally. However the link for the GCC suite provided by Cyamin works also.

Some hints which could help for a local machine configuration:
- don't mix different Gfortran installations
- give manually the relevant GCC bin directory to the environment variable PATH
- install as mpich2 as administrator
- don't worry if the command mpiexec -validate fails, for mpiexec use the flag -localonly in the systel.cfg configuration file.
- if you have Matlab or other parallel processing software with mpiexec etc. installed: place the mpich2/bin folder in the environment variable PATH before the e.g. the Matlab folders.

Find enclosed the configuration file.

Best regards,
Clemens
Attachments:
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8852

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hello,

After some testing/researching, I have found the following:

1. The '-fopenmp' flag does not affect the ability of telemac to run in parallel, so it can be omitted.

2. The optimization flag '-03' should be included in the linking command as well (not 100% sure, but it doesn't hurt to add it either).

3. If you really want to be more aggressive on the optimizations, you can use the '-0fast' flag and optimize the code for the architecture of your CPU using the '-march=cpu-type' and '-mtune=cpu-type' flags. However, pay attention to '-march' flag because it will render your code useless in a CPU of older architecture. More about the architecture optimizations can be read here. You can expect an increase in compilation time when using aggressive optimizations.

4. I am unable to distribute computations between different hosts. I have posted the issue here.

Here are the configuration I use at the moment:
cmd_obj:    x86_64-w64-mingw32-gfortran -march=corei7 -Ofast -c -ffixed-line-length-132 -fconvert=big-endian -frecord-marker=4 <mods> <incs> <f95name>
cmd_lib:    ar cru <libname> <objs>
cmd_exe:    x86_64-w64-mingw32-gfortran -march=corei7 -Ofast -fconvert=big-endian -frecord-marker=4 -v -o <exename> <objs> -Xlinker --start-group <libs>

Happy to hear your comments,
Costas
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8879

  • konsonaut
  • konsonaut's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 413
  • Thank you received: 144
Hello,

so, after some tests and configuration hints provided by cyamin:
optimizing the code has remarkable effects in the computation speed. I tested it on the large Malpasset validation case with 500 time steps.
My processor is a dual core Pentium D 925 3.00 GHz.
In the optimized configuration I used additionally the flags according to:
gcc.gnu.org/onlinedocs/gcc/i386-and-x86_002d64-Options.html

-march=nocona -Ofast

which is suitable for my processor.

Default configuration, parallel vs. scalar: 40% lower computation time using 2 cores
Optimized configuration, parallel vs. scalar: 50% lower computation time using 2 cores

Scalar mode: Optimized / Default = 0.67
Parallel mode: Optimized / Default = 0.56

which means that with the optimized configuration I have a speed up, compared to the default configuration, of up to 44%!
Enclosed you will find my systel file.

Thanks cyamin!

Best regards,
Clemens
Attachments:
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8935

  • OBA
  • OBA's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 19
  • Thank you received: 4
Hi all,

First, thanks a lot Clemens and Costas.

I’ve succeeded in compiling Telemac in parallel on windows 8 (64bits) with gfortran. I’ve encountered some difficulties with the two gfortran versions you’ve provided in your posts (#8601 and #8694). Then, I’ve found another version which works well with my configuration : www.equation.com/servlet/equation.cmd?fa=fortran (I’ve used the gcc-4.8.0-64.exe). It can be useful for others to try different gfortran.


As I have a dual-boot W8/Ubuntu-12.10 (both 64 bits) on my machine, I’ve made some tests with the two configurations : corei5 2,3GHz (2 cores and 4 logical processors) and 4 GB RAM.
Config 1 : Windows 8 64 bits/python 2.7/MPICH2 1.4.1/gfortran 4.8.0
Config 2 : Ubuntu 12.10 64 bits/python2.7/Openmpi 1.6.4/gfortran 4.8.0
The test is with one model of 68 200 nodes and 1 800 time steps.

Config 1 – W8 : scalar mode : 28 min 09 s
Parallel mode 2 proc : 15 min 25 s
Parallel mode 4 proc : 14 min 18 s

Config 2 – Ubuntu : scalar mode : 11 min 05 s
Parallel mode 2 proc : 8 min 16 s
Parallel mode 4 proc : 7 min 38 s

There’s a great difference between windows and linux. I’ve remarked than under linux, the processor are used at 100 % of their capacities. Under windows, the calculation only uses 10 % of the processors capacities in scalar mode, 20 % with 2 processors and 34 % with 4 proc ??? If someone has an explanation, I’m interesting…

I’ve tried to optimize the configuration under W8 by setting –march=corei7 (there’s nothing for corei5) in systel file, but there’s no significant gain in calculation’s time.

Hope this helps someone
Best regards.

Olivier
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8962

  • OBA
  • OBA's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 19
  • Thank you received: 4
Hi all,

just a complement to my previous post. I've succeeded to manage the limitation of my CPU's speed under Windows. Now the calculation's times under W8 are as follows :

Config 1 – W8 : scalar mode : 10 min 17 s
Parallel mode 2 proc : 7 min 27 s
Parallel mode 4 proc : 6 min 52 s

Best regards

Olivier
The administrator has disabled public write access.
The following user(s) said Thank You: cyamin

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8963

  • cyamin
  • cyamin's Avatar
  • OFFLINE
  • openTELEMAC Guru
  • Posts: 997
  • Thank you received: 234
Hi Olivier,

Thank you for your results. I was very interested to see if there is a large speed difference between Windows and Linux. Your results show that Windows are at least just as fast.

I have posted some benchmarks myself at these topics for the malpasset-large telemac2d validation case:
Malpasset case benchmark on several intel multicore CPUs
gfortran vs Intel fortran benchmarks (Windows x64)

You can compare your computation times with these if you like.

Costas
The administrator has disabled public write access.

Working parallel installation using python/gfortran in Windows 11 years 5 months ago #8972

  • Chris C.
  • Chris C.'s Avatar
Hello Olivier,

Could tell me how you managed to overpass the CPU limitations under Windows?

Thank you,
Christophe.
The administrator has disabled public write access.
Moderators: borisb

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.