Welcome, Guest
Username: Password: Remember me

TOPIC: BINARY ATMOSPHERIC DATA FILE

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45562

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Hello,

I tried the new method with the keyword BINARY ATMOSPHERIC DATA FILE to read pressure and wind data.

To date, I have these data under the netcdf format.
And I used the following command to convert the .nc file into .slf :
converter.py ecmwf2srf -f "2013-01-01" -s "2013-12-31" --bl "(42.,-12.)" --tr "(63.,13.)" --dataset win_pre_2013.nc --stream oper win_pre_2013.slf

Firstly, I get 2 errors when I run this command (scripts from v8p5r0).
The conversion finally succeed when I comment out some blocks in ecmwf.py :
- lines 446 to 449 for the error about x array
- lines 734 to 739 for the error about 't2m'

After this conversion :
- I get 1 error after using the command run_telfile.py alter –rename
(see picture attached);
- and much information is apparently missing when opening the resulting .slf file with Bluekenue.

So finally, I might have comment out the wrong lines.
How can I do please ?

Kind regards,
Audrey


error_run_telfile_hermes-api.png
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45563

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
What are the errors you tried to solve by commenting part of the script?
This is for sure the origin of the problem you finally encountered and which indicate a wrong selafin file...
Regards
Christophe
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45564

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Alright, thank you.

The 1rst error warns me about a longitudinal range of [348.0 , 13.0]
...while I wrote [-12.0 , 13.0] in the command.

After commenting lines 446 to 449, I have a 2nd error concerning 't2m'.
And the 't2m' data is not included in the ncdf file.

Please find attached both captures.

Kind regards,
Audrey
error_01.png


error_02.png
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45570

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi
Could you run the python script run_telfile with options "--scan --data" on the generated atmospheric file and copy the result?
Christophe
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45571

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Yes.

After running : run_telfile.py scan --data win_pre_2013.slf

I have :

Interpreting command line options
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Warning: Using SerafinFile. It is recommended to compile Hermes api
Traceback (most recent call last):
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/run_telfile.py", line 479, in <module>
main()
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/run_telfile.py", line 386, in main
scan(options.tel_file, options.bnd_file, options.data)
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/pretel/manip_telfile.py", line 27, in scan
res = TelemacFile(tel_file, bnd_file=bnd_file)
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/data_manip/extraction/telemac_file.py", line 68, in __init__
SerafinFile.__init__(self,
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/data_manip/formats/serafin_file.py", line 134, in __init__
self.__read()
File "/home/u0167980/Telemac/v8p5r0_01/scripts/python3/data_manip/formats/serafin_file.py", line 246, in __read
tmp = unpack(endian+str(self.__npoin3)+ftype,
struct.error: unpack requires a buffer of 34340 bytes
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45572

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Sounds logical.
In the selafin file, there is a difference between the number of points of the mesh and the length of some variables (probably X and Y)
Hard to say exactly where is the problem without data sources to debug...
I suspect that modification you made leads to a wrong masking operation...
If you're comfortable in python, try to print some values during the conversion (npoin2, npoin3, length of X and Y, length of variables array...)
Hope this helps
Christophe
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45573

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Alright. Thank you very much.

Actually my skills in python are very basic.
Is it possible that you show me on this example how to modify the script so that it works ?
After, I shall be able to manage other case by myself and show to the people.
If yes, how can I send you the data source ? it is about 500MB.

Kind regards,
Audrey
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45574

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3722
  • Thank you received: 1031
Hi
the modifications I propose are light...
Like you've commented some lines, you could add some print info like print('npoin2 ', npoin2) (with the variables you wanted to print).

For the last point, maybe you could try to limit netcdf file to few time-step as the problem is probably not related to the duration...
And then share it with any platform

Regards
Christophe
The administrator has disabled public write access.

BINARY ATMOSPHERIC DATA FILE 1 month 3 weeks ago #45575

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Alright, thank you very.

I'm having a look at it and give you the feedback.

Kind regards
The administrator has disabled public write access.
The following user(s) said Thank You: ah_sl

BINARY ATMOSPHERIC DATA FILE 1 month 2 weeks ago #45588

  • audrey.gno
  • audrey.gno's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 49
  • Thank you received: 2
Hello,
I had a look on some parameters and the npoin2 size seems ok with len(x)*len(y).
I have a doubt on some arrays in the block "Boundaries" : for instance I would make ipoin start from 1 and finish to npoin2 but not sure (?) Please find attached the script with my current checking on this block. And the data source : [url=data]https://kuleuven-my.sharepoint.com/:u:/g/personal/audrey_gineau_kuleuven_be/EYCvyDmNW0RLn7EsI3WES5YBPBS5SsnClxpbyNVDRye66w?e=mDO9z9[/url]
I run : converter.py ecmwf2srf -f "2013-01-01" -s "2013-12-31" --bl "(42.,-12.)" --tr "(63.,13.)" --dataset win_pre_2013.nc --stream oper win_pre_2013.slf
Kind regards,
Audrey

File Attachment:

File Name: ecmwf.py
File Size: 32 KB
The administrator has disabled public write access.
Moderators: pham

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.