Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
PRE/POST PROCESSORS
Scripts
Benchmark performance of ppUtils vs Selafin vs TelemacFile
Flat Threaded

TOPIC: Benchmark performance of ppUtils vs Selafin vs TelemacFile

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 4 days ago #44201

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
Hi,

One feedback.. and one question :)
I want to incorporate selafin as an xarray backend.
Within that scope I did a small performance test.

The idea would be to do:
import xarray as xr
ds = xr.open_dataset(file_3d, engine='selafin')

to read Selafin binaries directly using xarray (more info there)

So, I wanted to know the performance times for each python routine (that I knew of) that read selafin binaries.

The script is the following:
import xarray as xr
import matplotlib.pyplot as plt
import time
from data_manip.formats.selafin import Selafin 
from data_manip.extraction.telemac_file import TelemacFile
from .selafin_io_pp import ppSELAFIN

def main():
    def timer(func):
        def wrapper(*args, **kwargs):
            start = time.time()
            result = func(*args, **kwargs)
            end = time.time()
            print(f"Time taken by {func.__name__}: {end - start} seconds")
            return result
        return wrapper
    
    @timer
    def ppUtils(file_path):
        ppslf = ppSelafin(file_path)
        print(ppslf.meshx, ppslf.meshy)

    @timer
    def selafin(file_path):
        slf = Selafin(file_path)
        print(slf.meshx, slf.meshy)

    @timer
    def telemac(file_path):
        tel = TelemacFile(file_path)
        print(tel.meshx, tel.meshy)
    
    file_3d = 'hindcast/201301/input_wind_tmp.slf'
    ppUtils(file_3d)
    selafin(file_3d)
    telemac(file_3d)

This is what I get for a month of ERA5 data (u, v, p_msl) at 0.25° resolution (~9GB file size)

[ 0.    0.    0.   ... -0.25 -0.25 -0.25] [ 90.    89.75  89.5  ... -89.5  -89.75 -90.  ]
Time taken by ppUtils: 1.5250358581542969 seconds
[ 0.    0.    0.   ... -0.25 -0.25 -0.25] [ 90.    89.75  89.5  ... -89.5  -89.75 -90.  ]
Time taken by selafin: 0.41683053970336914 seconds
Warning: Using SerafinFile. It is recommended to compile Hermes api
[ 0.    0.    0.   ... -0.25 -0.25 -0.25] [ 90.    89.75  89.5  ... -89.5  -89.75 -90.  ]
Time taken by telemac: 79.03207039833069 seconds

ppUtils and selafin are quite similar in performance but TelemacFile reads all data directly in the RAM by default.

Is that really necessary?
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 4 days ago #44202

  • sebourban
  • sebourban's Avatar
  • NOW ONLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
hello Thomas,

Very nice use of xarray readers, and since I am on the forum : happy to see that my selafin class is still 3x faster than the second performer (hello Pat), and almost 200 times faster than TELEMAC (HERMES?) ;-)

Let me know of your progress anyway,

Best,
Sébastien.
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 4 days ago #44204

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi
This is not hermes as we see the message in the print...
I imagine this test is running on windows...

My personal experience (and my 2 cents) about this subject:
Dealing with quite large results to extract max in python, using hermes on linux on the master node of our cluster is clearly faster than using selafin_file on my PC...
Same python script but API is compiled and available on the cluster...
Christophe
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 4 days ago #44207

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
thanks both of you for your feedbacks.

I am on Linux Ubuntu 22.04.
For the performance test I used my conda environment:
mamba create -n opentelemac -c tomsail opentelemac
mamba activate opentelemac
github.com/tomsail/telemac-conda

which should work across Linux platforms out of the box. I have dynamically compiled TELEMAC where I have MPI, MUMPS, MED, GOTM but not HERMES.

Nicolas Godet has done the same for Windows:
mamba create -n opentelemac -c nicogodet opentelemac
mamba activate opentelemac
github.com/nicogodet/telemac-mascaret-feedstock
which is a simpler version (API but no MPI).

Anyway the important fact here is that TelemacFile is much slower because, when using serafin_file class, it load everything on the RAM (code is here), whereas the classes ppUtils.selafin_io_pp.py and selafin.py do not.

To read data at a certain time step, both classes have these functions: which load only the data required in the RAM.

My suggestion would be to implement the same "practice" in TelemacFile, because the class has much more useful wrapper functions (like interpolation on grid points, lines even spectrum etc..)

For now I chose the selafin class but I'd be keen to migrate to TelemacFile once that problem is solved.
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44217

  • Lux
  • Lux's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 96
  • Thank you received: 39
Dear Thomas,

Thank you for this interesting benchmark on parsing a Selafin file.

Would it be possible please for you to include PyTelTools in your benchmark ?
It an independant package, written I think in a more modern and maintainable way.
It does not load into memory the whole file and does not require Telemac API or HERMES.

You should code something like:
from pyteltools.slf import Serafin

...

    @timer
    def pyteltools(file_path):
        with Serafin.Read('hindcast/201301/input_wind_tmp.slf', 'en') as resin:
            resin.read_header()
            print(resin.header.x_stored, resin.header.y_stored)  # (or `print(resin.header.x, resin.header.y)` to take into account x and y shift)

Why not trying to read multiple specific frames (time records), because in what you show, you only read the header (x and y coordinates of nodes) which is quite simple.

Thank you in advance.

Best regards,
Luc
The administrator has disabled public write access.
The following user(s) said Thank You: tomsail

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44221

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
the winner is... selafin

Here is my benchmark on reading
1 - the header
2 - the 10th record of data in the file
import time
from data_manip.formats.selafin import Selafin 
from data_manip.extraction.telemac_file import TelemacFile
from xarray_selafin_backend import Serafin
from xarray_selafin_backend.selafin_io_pp import ppSELAFIN

def main():
    def timer(func):
        def wrapper(*args, **kwargs):
            start = time.time()
            result = func(*args, **kwargs)
            end = time.time()
            print(f"Time taken by {func.__name__}: {end - start} seconds")
            return result
        return wrapper
    # ppUtils
    @timer
    def ppUtils(f):
        ppslf = ppSELAFIN(f)
        # print(ppslf.x, ppslf.y)
        ppslf.readHeader()
        return ppslf
    @timer
    def ppUtils_access(i_record, cls):
        cls.readVariables(i_record)
    # selafin
    @timer
    def selafin(f):
        slf = Selafin(f)
        return slf
    @timer
    def selafin_access(i_record, cls):
        cls.get_series([i_record])
    # telemacFile (without Hermes)
    @timer
    def telemac(f):
        tel = TelemacFile(f)
        return tel
    @timer
    def telemac_access(i_record, cls):
        cls.get_data_value(cls.varnames[0], i_record)
    # PyTelTools
    @timer
    def pyTelTools(f):
        with Serafin.Read(f, 'en') as resin:
            resin.read_header()
    @timer
    def pyTelTools_access(i_record, f):
        with Serafin.Read(f, 'en') as resin:
            resin.read_header()
            resin.read_var_in_frame( i_record,resin.header.var_IDs[0])
    
    file_3d = '/home/tomsail/Documents/work/python/pyPoseidon/Tutorial/test/hindcast/201303/input_wind_tmp.slf'
    slf = ppUtils(file_3d)
    ppUtils_access(10,slf)
    slf = selafin(file_3d)
    selafin_access(10,slf)
    slf = telemac(file_3d)
    telemac_access(10,slf)
    pyTelTools(file_3d)
    pyTelTools_access(10,file_3d)

main()

the results:
Time taken by ppUtils: 1.4760596752166748 seconds
Time taken by ppUtils_access: 0.6306843757629395 seconds
Time taken by selafin: 0.4218120574951172 seconds
Time taken by selafin_access: 0.18802118301391602 seconds                      
Warning: Using SerafinFile. It is recommended to compile Hermes api
Time taken by telemac: 85.08106231689453 seconds
Time taken by telemac_access: 3.0279159545898438e-05 seconds
Time taken by pyTelTools: 0.6683106422424316 seconds
Time taken by pyTelTools_access: 0.7252557277679443 seconds

I haven't checked thoroughly the implementation in pyTelTools (@Luc you can confirm maybe if I am doing this right).

After comparing all routines, they all present pros/cons.
The Selafin Class has the benefit of being the fastest and it almost works out of the box using that only script. (I made my own version with the same wrappers as in ppUtils)
The administrator has disabled public write access.
The following user(s) said Thank You: nicogodet

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44223

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 157
  • Thank you received: 39
It would worth the try using TelemacFile with Hermes.

I try on my broken conda package with a T2D result (14,3Go, 480k nodes, 751 timesteps).

Time taken by telemac: 0.0008957386016845703 seconds
Time taken by telemac_access: 0.013008832931518555 seconds

Could you provide a link to your file so I can test it for you ?
The administrator has disabled public write access.
The following user(s) said Thank You: sebourban

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44224

  • sebourban
  • sebourban's Avatar
  • NOW ONLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Thank you Nicolas,

I am getting ready to accept defeat .. can you try the selafin (and the others) on your file as well ?
It would provide us with a more diverse benchmark.

Thank you again.
Sébastien.
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44228

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 157
  • Thank you received: 39
Here are the results :cheer:

Time taken by ppUtils: 2.026013135910034 seconds
Time taken by ppUtils_access: 2.699988842010498 seconds
Time taken by selafin: 0.39100193977355957 seconds
Time taken by selafin_access: 0.17937898635864258 seconds
Time taken by telemac: 0.003999233245849609 seconds
Time taken by telemac_access: 0.010999441146850586 seconds
Time taken by pyTelTools: 0.39600324630737305 seconds
Time taken by pyTelTools_access: 0.4320356845855713 seconds

Without Hermes, I don't have enough RAM.
Warning: Using SerafinFile. It is recommended to compile Hermes api
Traceback (most recent call last):
  File "D:\Documents\slf_bench\bench.py", line 89, in <module>
    main()
  File "D:\Documents\slf_bench\bench.py", line 81, in main
    slf = telemacHermes(file_3d)
  File "D:\Documents\slf_bench\bench.py", line 13, in wrapper
    result = func(*args, **kwargs)
  File "D:\Documents\slf_bench\bench.py", line 45, in telemacHermes
    tel = TelemacFile(f)
  File "D:\miniforge3\envs\opentelemac\Library\opentelemac\scripts\python3\data_manip\extraction\telemac_file.py", line 68, in __init__
    SerafinFile.__init__(self,
  File "D:\miniforge3\envs\opentelemac\Library\opentelemac\scripts\python3\data_manip\formats\serafin_file.py", line 134, in __init__
    self.__read()
  File "D:\miniforge3\envs\opentelemac\Library\opentelemac\scripts\python3\data_manip\formats\serafin_file.py", line 232, in __read
    self.__values = np.zeros((self.__ntimestep, self.__nvar, self.__npoin3),
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 27.3 GiB for an array with shape (751, 10, 487612) and data type float64
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 9 months 3 days ago #44229

  • sebourban
  • sebourban's Avatar
  • NOW ONLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
.. and you did that on Windows. Nice !
The administrator has disabled public write access.
Flat Threaded
Moderators: borisb
Forum
PRE/POST PROCESSORS
Scripts
Benchmark performance of ppUtils vs Selafin vs TelemacFile

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.