Skip to content

Installation tweaks#32

Open
wdeconinck wants to merge 6 commits intoGridTools:masterfrom
wdeconinck:installation-tweaks
Open

Installation tweaks#32
wdeconinck wants to merge 6 commits intoGridTools:masterfrom
wdeconinck:installation-tweaks

Conversation

@wdeconinck
Copy link
Collaborator

@wdeconinck wdeconinck commented Jan 29, 2026

I have made time available this year to work on our plan to extend atlas4py and hand over maintenance.
As a first step I tried the new build-system, and noticed that I needed to tweak the installation a little bit.

  • Fix RPATH handling where atlas library is external
  • Faster compilation, removing unneeded parts
  • Reinstallation, e.g. when developing, no longer picks up a previously installed atlas4py-provided atlas/eckit library
  • Added more tests, and a few cosmetic changes
  • Bugfix in RectangularDomain construction
  • Changes to avoid using deprecated eckit functionality to be removed with eckit 2.0.0.

@wdeconinck wdeconinck mentioned this pull request Jan 29, 2026
@wdeconinck wdeconinck force-pushed the installation-tweaks branch 2 times, most recently from 597b52a to 6d74645 Compare February 4, 2026 14:44
@wdeconinck wdeconinck force-pushed the installation-tweaks branch 4 times, most recently from 6d1d6b1 to 1c7d1cb Compare February 9, 2026 09:34
@wdeconinck
Copy link
Collaborator Author

Thank you for merging #30 @tehrengruber , @havogt .
I have rebased this on latest master; It should be ready for review.

@wdeconinck wdeconinck force-pushed the installation-tweaks branch 2 times, most recently from b5f514e to 3281313 Compare February 11, 2026 15:23
@havogt
Copy link
Contributor

havogt commented Feb 12, 2026

I only did a brief walk through the code so far. One question popped up and I didn't follow the code to figure it out myself: how is the version managed? Is it taking always the version of atlas?

@wdeconinck
Copy link
Collaborator Author

wdeconinck commented Feb 12, 2026

I only did a brief walk through the code so far. One question popped up and I didn't follow the code to figure it out myself: how is the version managed? Is it taking always the version of atlas?

atlas4py version is not managed by the found atlas library, but that could be a strategy. I did not deviate from what was in place. I removed however duplicate places where the version was defined with only a single place remaining, in the pyproject.toml file.
This version is then defined as a CMake variable SKBUILD_PROJECT_VERSION_FULL by scikit-build-core. I then add that as a preprocessor definition into the _atlas4py.cpp __version__ attribute of the module. This means that the manually managed __version__ attribute is no longer needed.

There is still the extra variable ATLAS4PY_ATLAS_VERSION defined in pyproject.toml, which is only used to download atlas if atlas was not found. This could be introspected from the atlas4py version instead of course, if the format is clear.
With the use of externally installed atlas versions, the version of atlas4py and the actual atlas (commit hash, branch, or tag) used may differ, as then ATLAS4PY_ATLAS_VERSION variable is ignored. I added a CMake configuration warning however when the version is mismatched.

@wdeconinck
Copy link
Collaborator Author

Note, just rebased again on master.

@@ -1,5 +1,6 @@
black>=21.12b0
bump2version>=1.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bump2version>=1.0

find_package(ecbuild)
if(NOT ecbuild_FOUND)
FetchContent_Declare(
if(build_atlas)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this if-branch are fine with me, but my tendency is rather to simplify than to extend here. In case one want's to build atlas the motivation is likely: I want to use or develop atlas4py, don't bother me with installing atlas, just set me up. Being able to say build atlas for me, but please use this eckit version I provide appears a rather narrow use case that can easily be handled by just building everything manually.

if(build_atlas)
  # Build ecbuild
  set ( ecbuild_SOURCE_DIR ${CMAKE_BINARY_DIR}/_deps/ecbuild )
  message( STATUS "Downloading ecbuild version \"${ATLAS4PY_ECBUILD_VERSION}\" to ${ecbuild_SOURCE_DIR}" )
  FetchContent_Populate(
      ecbuild
      GIT_REPOSITORY https://github.com/ecmwf/ecbuild.git
      GIT_TAG        ${ATLAS4PY_ECBUILD_VERSION}
      SOURCE_DIR     ${ecbuild_SOURCE_DIR}
      QUIET
  )
  set( ecbuild_ROOT ${ecbuild_SOURCE_DIR} CACHE INTERNAL "Found ecbuild" )
  
  # Build eckit
  message( STATUS "Downloading and building eckit version \"${ATLAS4PY_ECKIT_VERSION}\"" )

  # Disable unused features for faster compilation
  set(ECKIT_ENABLE_TESTS     OFF)
  set(ECKIT_ENABLE_DOCS      OFF)
  set(ECKIT_ENABLE_PKGCONFIG OFF)
  set(ECKIT_ENABLE_ECKIT_GEO OFF)
  set(ECKIT_ENABLE_ECKIT_SQL OFF)
  set(ECKIT_ENABLE_ECKIT_CMD OFF)
  FetchContent_Declare(
      eckit
      GIT_REPOSITORY https://github.com/ecmwf/eckit.git
      GIT_TAG        ${ATLAS4PY_ECKIT_VERSION}
  )
  FetchContent_MakeAvailable(eckit)
  set( eckit_ROOT ${eckit_BINARY_DIR} CACHE INTERNAL "Found eckit" )

  # build atlas
  message( STATUS "Downloading and building atlas version \"${ATLAS4PY_ATLAS_VERSION}\"" )

  # Disable unused features for faster compilation
  set(ATLAS_ENABLE_TESTS     OFF)
  set(ATLAS_ENABLE_DOCS      OFF)
  set(ATLAS_ENABLE_PKGCONFIG OFF)
  set(ECKIT_ENABLE_ECKIT_GEO OFF)
  set(ECKIT_ENABLE_ECKIT_SQL OFF)
  set(ATLAS_ENABLE_ECKIT_CMD OFF)
  FetchContent_Declare(
      atlas
      GIT_REPOSITORY https://github.com/ecmwf/atlas.git
      GIT_TAG        ${ATLAS4PY_ATLAS_VERSION}
  )
  FetchContent_MakeAvailable(atlas)

PYBIND11_MODULE( _atlas4py, m ) {
m.def("_initialise", atlasInitialise)
.def("_finalise", atlas::finalise);
m.attr("version") = atlas::Library::instance().version();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
m.attr("version") = atlas::Library::instance().version();
m.attr("atlas_version") = atlas::Library::instance().version();

or just removing it altogether?

Suggested change
m.attr("version") = atlas::Library::instance().version();

else()
set(rpath_origin_install_libdir "$ORIGIN/${CMAKE_INSTALL_LIBDIR}")
endif()
set( CMAKE_INSTALL_RPATH "${rpath_origin_install_libdir}" ) # A semicolon-separated list specifying the RPATH to use in installed targets
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a motivation for the 3 additional options here?

atlas::initialise(argc, argv);
}

py::object toPyObject( eckit::Configuration const& v );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want a textbook implementation you could also use py::cast and a type caster for eckit::Configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments