Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
PRE/POST PROCESSORS
Scripts
Benchmark performance of ppUtils vs Selafin vs TelemacFile
Flat Threaded

TOPIC: Benchmark performance of ppUtils vs Selafin vs TelemacFile

Benchmark performance of ppUtils vs Selafin vs TelemacFile 8 months 4 weeks ago #44276

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
Exactly Sebastien.

Ideally I would see Luc's implementations in the Selafin class routine. Also for the sovereignty of the xarray package (i.e. who maintains it).

But for now, I think we can go for pyTelTools Serafin backend, wrap the package and see how it performs.
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 8 months 4 weeks ago #44277

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 157
  • Thank you received: 39
As a quick comment, all packages should be available through pip for an easy deployment on all systems.
Maybe for a low proprity TODO list...
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 8 months 3 weeks ago #44297

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
Hi all,

xarray-selafin is on pipy, using pyTelTools backend.

to have it working in xarray, follow these steps:
Install:
pip install xarray-selafin

Read selafin:
import xarray as xr
ds = xr.open_dataset("input_file.slf", engine='selafin')
Write selafin:
ds.selafin.write('output_file.slf')

github.com/seareport/xarray-selafin

pypi.org/project/xarray-selafin/
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 8 months 3 weeks ago #44298

  • tomsail
  • tomsail's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 43
  • Thank you received: 17
example:
import xarray as xr

file  = "results202307.slf" #32GB
fileout = "world_20230801.slf" #238MB

ds = xr.open_dataset(file, engine="selafin")
ds.isel(time = -1).selafin.write(fileout) # 0.31511 seconds

Happy coding to all.

Thanks to Luc for the main contribution
The administrator has disabled public write access.
The following user(s) said Thank You: sebourban, Htun Pyae Sone

Benchmark performance of ppUtils vs Selafin vs TelemacFile 8 months 3 weeks ago #44306

  • sebourban
  • sebourban's Avatar
  • OFFLINE
  • Administrator
  • Principal Scientist
  • Posts: 814
  • Thank you received: 219
Brilliant !
The administrator has disabled public write access.

Benchmark performance of ppUtils vs Selafin vs TelemacFile 6 months 4 weeks ago #44653

  • borisb
  • borisb's Avatar
  • OFFLINE
  • Admin
  • Posts: 128
  • Thank you received: 64
Hello everyone,

As I'm currently working on mesh issues in TELEMAC Python scripts, I've taken the time to do some benchmarks, not only on mesh reading, but also on mesh writing and memory consumption when writing to another file, as I'm also trying to solve a problem that Nicolas Godet encountered when trying to merge several huge meshes.

I did my benchmark on f3d_malpasset-murd_p6.slf, which is a TELEMAC-3D results file of about 5 MB. Here are my results:

xarray with SELAFIN backend
Mesh reading: 0.05676 seconds
Mesh writing: 0.06217 seconds
Memory consumption when writing: 10.65 MB

TelemacFile with HERMES
Mesh reading: 0.00771 seconds
Mesh writing: 0.02675 seconds
Memory consumption when writing: 17.10 MB

PyTelTools
Mesh reading: 0.05070 seconds
Mesh writing: 0.05185 seconds
Memory consumption when writing: 5.79 MB

Here we can clearly see that TelemacFile with HERMES still outperforms the other solutions in terms of I/O performance. However, we can also see that TelemacFile is not optimised at all in terms of memory usage when we need to copy a SELAFIN file to another file, as it uses two memory buffers: one for the original file and one for the output file. PyTelTools does a better job here, as it is designed to copy frames in chunks.
The administrator has disabled public write access.
Flat Threaded
Moderators: borisb
Forum
PRE/POST PROCESSORS
Scripts
Benchmark performance of ppUtils vs Selafin vs TelemacFile

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.