Welcome, Guest
Username: Password: Remember me

TOPIC: BK 3.12.6 - empty Selafin variables (-999)

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41287

  • Renault
  • Renault's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 120
  • Thank you received: 33
Hi all,
I am trying to create Selafin objects for a grid convergence test and hitting snags all along the way.

Firstly, I tried to create meshes in 3.3.4 but for some reason, possibly due to the coordinate system (can't remember what it is off the top of my head), Y values in the .t3s file are truncated to the nearest metre. As my mesh resolution is much smaller than metres, this is quite problematic. So, I upgraded to 3.12.6 in order to hopefully circumvent this issue, and so far it works - no truncating issues.

The problem I am now facing is that whenever I create a variable in a new Selafin object, the attribute data is and remains -999 for every line of the data. The mesh XY values are correct, but the Value field is -999 and stubbornly refuses to stay changed. For instance, I might create a BOTTOM variable and base it off of a working .t3s file. When I run the 2D Interpolator, it says it's worked successfully - and indeed, if I go into the Properties, it shows the expected elevations (ranging between 216 and 222 m approximately). This step works for any other mesh outside a Selafin variable with no issues, permanently adding the new elevation. However, if I then save the Selafin, or go into the ColourScale pane and press "reset" to see the updated values, all the elevation data is wiped to -999 !
This issue extends to other variables, like BOTTOM FRICTION, in similar ways. If I impose a constant value of 0.035 for BOTTOM FRICTION, whenever I go to save or view the colour scale, all the data for the Value variable is wiped to -999. Honestly, I have no idea what to do or what can be causing this.
I have not encountered this issue in BK 3.3.4 before; under the same conditions, while the mesh is still truncated in the Y direction, the BOTTOM interpolation elevation values are saved and I am able to view them. No matter what I do, I can't seem to get a working Selafin object!..
BK's RAM usage is about 700 MB as I try to create a meaningful BOTTOM variable, so I doubt it has anything to do with a lack of space.
I have briefly looked at using pputils to write the BOTTOM / BOTTOM FRICTION variables to the Selafin, but I'm thinking there must be an easier way to do this.

Please let me know if I should submit a bug report on CHyMS and/or provide my files (I'd rather not post them publicly and the .xyz is fairly big), I will gladly do so if it can help. Otherwise, please let me know if you have any hints for a solution to this apparently new problem (again, the -999 problem hasn't happened to me in 3.3.4).

Kind regards,
André Renault
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41288

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
Hi André,

No need to share your private files. BK 3.12.6 is an alpha version, and so there are things being modified under-the-hood; I'm not surprised to see things that are different when compared with the released BK from about 8-10 years ago.

One thing we can try is for you to create a very simple example that mimics what you see. Then share those file(s) with me and the step(s) you are using and I'll have a look.

Kindly,
Alan
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41291

  • Renault
  • Renault's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 120
  • Thank you received: 33
Hi Alan,

Thanks for your quick reply. I put together some example data that will illustrate my issues. Here is what I did:
  1. Import or create .xyz file with data (I more-or-less randomly generated some values - dummyXYZ.xyz).
  2. Create a closed line (.i2s) to represent the boundary; add attribute (no way to not add an attribute) name = "blank"; units = "M" and default value = 0.
  3. Create a T3 Mesh Generator with default settings (Resample Outline = Y; Edge Growth Ratio = 1.2; Auto Smooth Mesh = Y; Resample Lines Only = N; Overwrite Mesh = Y; New Resampler = N, Default Edge Length = 1).
  4. Add the boundary.i2s as the T3 Mesh Generator outline and run the T3 mesh generator.
  5. Create a new 2D Interpolator and drag the .xyz onto the interpolator (something goes wrong at this step and the 2D interpolator won't accept the drag and drop; if I delete the header from the .xyz, it gives me a warning about lacking a header when loading the file, but the drag-and-drop onto the 2D interpolator works!).
  6. Use the Map Object to interpolate bathymetry onto the newly generated mesh. New Attribute characteristics are added (Object = "New Mesh"; :NoDataValue = -999 (this value cannot be changed); Attribute Count = 2 (this value cannot be changed); Name = "elev"; Units = "m"; Type = "double"; Default Value = 0; Value Count = 2887 (this value cannot be changed)).
  7. At this point, open the Properties of New Mesh. At first, the elev values appear to be saved, however when adjusting the ColourScale or looking at the attributes in Data, ALL the attributes of the mesh are reinitialized to -999 (NodeType, Density, elev)!!
  8. Retry the Map Object and this time, the elevation values are saved to the mesh, no reinitialization of values. Save the mesh with bathymetry data as newmesh.t3s.
  9. Create a new SELAFIN object and change no settings (Save as SELAFIND = N (I seem to have this bug whether or not it's a double); Save X/Y Origin = N).
  10. Right-click on the SELAFIN to Add Variable. Source Mesh = "New Mesh"; Name = "BOTTOM"; Units = "M"; Copy Node Values from Source = N; Default Node Value = 0.
  11. Check the properties of the BOTTOM variable. The only attribute is Value(float) and all values are -999.
  12. Run Map Object with the 2D Interpolator on the BOTTOM variable. Nothing has changed - all values are still -999. Repeat ad nauseaum.
  13. Save the SELAFIN as newslf.slf and again try to Map Object, to no avail.
  14. Run Map Object on the BOTTOM variable, this time with the newmesh.t3s file (which has bathymetry data), again to no avail.
  15. Save the BOTTOM mesh as its own mesh (separate from the SELAFIN) and open it in Blue Kenue. Run Map Object on the separate BOTTOM mesh and this time, values are successfully saved to the separate BOTTOM mesh, not to the SELAFIN BOTTOM.
  16. Squeeze stress ball.

A previous time, I had exactly the same problems defining BOTTOM FRICTION; the values would reset to -999 no matter what I did.
Please let me know if you want to reach out and talk directly - hopefully, the instructions and the provided files should be enough to reproduce my bug. I may reinstall Blue Kenue or try the alpha on a different machine, though this would be a last-ditch attempt.

Thanks again for your help, and hopefully this isn't too onerous a fix (if it is a bug - I could just be doing wrong, but it used to work in 3.3.4)! Again, please let me know if I should submit a CHyMS ticket for this glitch.

André Renault
Attachments:
The administrator has disabled public write access.
The following user(s) said Thank You: a.barton

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41294

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
Hi André,

Today I'm not near my computer, so I can have a look on Monday.

Thanks for the example data, detailed steps AND the extra commentary. If only everyone took the time to properly explain things like you have! Much appreciated! And don't squeeze that ball too hard, it might POP! :laugh:

In any case, sorry for the trouble.

Kindly,
Alan

As a side note, this -999 business is related to missing data. So, for example, min/max will not consider -999 as a part of the min/max calculations. And, over the last years, how this is being done under-the-hood has been radically changing. The paint may be the same colour, but the engine is being swapped out, part by part. So the current alpha version of Blue Kenue has some new parts, along side some old parts. Over time, the old parts will also be swapped out for newer parts. And while the old and new parts are in the same engine, there may be some hiccups. Stay tuned...
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41297

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
1. Import or create .xyz file with data (I more-or-less randomly generated some values - dummyXYZ.xyz).

Step1.png


All ok so far; the warning message is normal and I believe, self explanatory.

Marching on...
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41298

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
4. Add the boundary.i2s as the T3 Mesh Generator outline and run the T3 mesh generator.

Step4.png


A few debugging warnings were issued that I may need to check. Otherwise, the current attribute in the new mesh is the first attribute named "Value". So far so good.

Marching on...
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41300

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
5. ... if I delete the header from the .xyz, it gives me a warning about lacking a header when loading the file...
The warning is ok and the loading proceeds correctly. The issue that you've found is that the header is incorrect. Please delete the *.xyz files that contain a header as they are invalid. BK 3.12.7-alpha will contain a fix.

Note that BK 3.12.7-alpha will allow you to have a colour scale for X, Y or Z as in the following screen snapshot. This is because internally, X,Y and Z are considered non-removable attributes and externally, when written to the header in a file, they should be hidden since, externally, they are NOT considered attributes. Make sense? oh boy... anyway, it works! :silly:

Step5_fixForXYZHeader.png
The administrator has disabled public write access.
The following user(s) said Thank You: Renault

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41301

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
6. Use the Map Object to interpolate bathymetry onto the newly generated mesh.

The input and settings:
Step6_Input.png


The first output after a couple of little tweaks inside the code show that nodes in the mesh ARE updated according to the Z value but that the min/max stats in the properties dialog are NOT updated. Odd... will have to look deeper...

Step6_FirstInterpolationOutput.png


I'm off for the weekend. Enjoy!
The administrator has disabled public write access.
The following user(s) said Thank You: Renault

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41305

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
Stats fixed

Step6_SecondInterpolationOutput.png


Marching on...
The administrator has disabled public write access.

BK 3.12.6 - empty Selafin variables (-999) 2 years 1 month ago #41306

  • a.barton
  • a.barton's Avatar
  • OFFLINE
  • Moderator
  • Posts: 683
  • Thank you received: 178
8. ... Save the mesh with bathymetry data as newmesh.t3s.
Saving and loading the mesh WITH the bathymetry data mapped onto it is working as expected.

Step8_SaveMeshWithBathyAndReload.png


Marching on...
The administrator has disabled public write access.
Moderators: Serrer, a.barton

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.