Welcome, Guest
Username: Password: Remember me

TOPIC: API compilation fails with Numpy >= 1.24.0

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43257

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
here is the error :
Building the Python API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~> Wrap_api built
~> Compiling Modules api
Traceback (most recent call last):
File "D:\telemac-mascaret\scripts\python3\compilation\compil_tools.py", line 913, in compile_api_f2py
output = check_output(cmd, shell=True, stderr=STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'f2py --quiet -h D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\lib\api.pyf -m _api D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_handle_var_t
3d.f90 D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_handle_var_t2d.f90 D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_handle_var_sis.f90 D:\telemac-mascaret\builds\gnu.dyna
mic\wrap_api\src\api_handle_var_art.f90 D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_handle_var_wac.f90 D:\telemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_handle_error.f90 D:\t
elemac-mascaret\builds\gnu.dynamic\wrap_api\src\api_interface.f90 skip: get_boolean_t2d_d get_double_t2d_d get_integer_t2d_d get_string_t2d_d get_var_size_t2d_d set_boolean_t2d_d set_double_t
2d_d set_integer_t2d_d set_string_t2d_d get_double_array_t2d_d get_integer_array_t2d_d set_double_array_t2d_d set_integer_array_t2d_d get_boolean_sis_d get_double_sis_d get_integer_sis_d get_s
tring_sis_d get_var_size_sis_d set_boolean_sis_d set_double_sis_d set_integer_sis_d set_string_sis_d get_double_array_sis_d get_integer_array_sis_d set_double_array_sis_d set_integer_array_sis
_d get_boolean_t3d_d get_double_t3d_d get_integer_t3d_d get_string_t3d_d get_var_size_t3d_d set_boolean_t3d_d set_double_t3d_d set_integer_t3d_d set_string_t3d_d get_double_array_t3d_d get_int
eger_array_t3d_d set_double_array_t3d_d set_integer_array_t3d_d get_boolean_art_d get_double_art_d get_integer_art_d get_string_art_d get_var_size_art_d set_boolean_art_d set_double_art_d set_
integer_art_d set_string_art_d get_double_array_art_d get_integer_array_art_d set_double_array_art_d set_integer_array_art_d get_boolean_wac_d get_double_wac_d get_integer_wac_d get_string_wac
_d get_var_size_wac_d set_boolean_wac_d set_double_wac_d set_integer_wac_d set_string_wac_d get_double_array_wac_d get_integer_array_wac_d set_double_array_wac_d set_integer_array_wac_d :' re
turned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "D:\telemac-mascaret\scripts\python3\compile_telemac.py", line 142, in <module>
main()
File "D:\telemac-mascaret\scripts\python3\compile_telemac.py", line 134, in main
compile_api_files(silent=not options.verbose,
File "D:\telemac-mascaret\scripts\python3\compilation\compil_tools.py", line 1001, in compile_api_files
compile_api_f2py('api', api_dir, source_api, skip_source, ld_flags,
File "D:\telemac-mascaret\scripts\python3\compilation\compil_tools.py", line 915, in compile_api_f2py
raise TelemacException(\
utils.exceptions.TelemacException: Error during first part of f2py for api 1
Traceback (most recent call last):
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Scripts\f2py-script.py", line 10, in <module>
sys.exit(main())
^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\f2py2e.py", line 704, in main
run_main(sys.argv[1:])
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\f2py2e.py", line 441, in run_main
postlist = callcrackfortran(files, options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\f2py2e.py", line 342, in callcrackfortran
postlist = crackfortran.crackfortran(files)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\crackfortran.py", line 3320, in crackfortran
readfortrancode(files, crackline)
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\crackfortran.py", line 552, in readfortrancode
dowithline(finalline)
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\crackfortran.py", line 856, in crackline
analyzeline(m, pat[1], line)
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\crackfortran.py", line 1280, in analyzeline
for e in markoutercomma(ll).split('@,@'):
^^^^^^^^^^^^^^^^^^
File "C:\Users\godet\AppData\Local\mambaforge\envs\test-telemac\Lib\site-packages\numpy\f2py\crackfortran.py", line 893, in markoutercomma
assert not f, repr((f, line, l))
AssertionError: (1, ' pos_name = (', ' pos_name = (')

If I downgrade Numpy to 1.23.5, no error.
As numpy 1.23 will soon be EOL, could be nice to fix this for V8P5 release
endoflife.date/numpy

I'm currently building a conda package for windows and building on latest dependencies would be better.
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43262

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
Okay found other bug reports regarding numpy version.
Guess I need to stick with python 3.10 and numpy 1.21 for now

Weird cause I successfully compile with Py3.11 and Numpy 1.23 in another environnement...
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43268

  • borisb
  • borisb's Avatar
  • OFFLINE
  • Admin
  • Posts: 129
  • Thank you received: 67
Hello Nico,

This is indeed a known problem with the latest versions of F2PY, we've looked into it and it's not easy to solve. And as everything was working perfectly before NumPy 1.22.0, we're not even sure that the problem comes from the way the TELEMAC API is defined, this could very well be related to an issue in F2PY itself. However, this is a major problem and if we can't solve it on our own, we may have to replace F2PY with another system, in fact we're already looking at different alternatives.
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43359

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
With gitlab.pam-retd.fr/otm/telemac-mascaret/...ce7f7f51aea8d8e3fd30, I successfully compile using Python 3.11 and Numpy 1.25.2 !
But when running a validation case, I have an error
Validation < 1/17 > of <root>\examples\gaia\littoral-t2d-tom\vnv_tel2tom.py
geo_t2d_littoral.cli geo_tom_littoral.cli
_hermes.error is related to
which correspond to this line:
(telemac) D:\DEV\telemac-mascaret\examples\gaia\littoral-t2d-tom>python
Python 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import _hermes
>>> import sys
>>> H_hermes=sys.modules['_hermes']
>>> H_hermes.open_mesh('SERAFIN', 'geo_t2d_same.slf' , b'READ     ')
_hermes.error is related to
(telemac) D:\DEV\telemac-mascaret\examples\gaia\littoral-t2d-tom>python
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43360

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
By the way, typo in gitlab.pam-retd.fr/otm/telemac-mascaret/...?ref_type=heads#L163
self._errror --> self._error
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43366

  • borisb
  • borisb's Avatar
  • OFFLINE
  • Admin
  • Posts: 129
  • Thank you received: 67
Thanks for the report Nico. I've started working on a fix, but it's much trickier than expected. The problem started with F2PY >= 1.22.0: Fortran 90 files in fixed format are no longer processed correctly. At some point, probably next year, we'll switch all TELEMAC codebase to F90 free format, but in the meantime we have to rely on dirty tweaks to get F2PY working properly.

I have a preliminary working branch to fix problems with F2PY >= 1.22.0, but not all cases work, so there's still work to be done. This is because other changes have been made in F2PY to the way function signatures are generated. On top of that, there is an issue in F2PY itself: github.com/numpy/numpy/issues/24662. So, F2PY itself will need a patch at some point.
The administrator has disabled public write access.

API compilation fails with Numpy >= 1.24.0 1 year 3 months ago #43378

  • nicogodet
  • nicogodet's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 166
  • Thank you received: 40
Glad to see your are working on it !
The administrator has disabled public write access.

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.