Modifications to the DurationSubdivision plugin to reduce the memory usage#1
Merged
maxwhitemet merged 1 commit intomaxwhitemet:duration_subdivision_updatefrom Mar 10, 2026
Conversation
maxwhitemet
pushed a commit
that referenced
this pull request
Mar 11, 2026
* EPPT-2584: Fire Severity Index: Add a FineFuelMoistureContent class to IMPROVER (metoppv#2247) * Initial version of FFMC class * Full working test set for FFMC * Adding reminder for when I come back to this * Adding FFMC to the IMPROVER api * Review changes to FFMC code * Refactor to use a general FWI ABC * Fixing docstring that breaks sphinx * Changes from ABC refactor * Adding min/max feasible values for inputs and outputs in the ABC. This is then tested for the base class and FFMC * EPPT-2586: Fire Severity Index: Add a DroughtCode class to IMPROVER (metoppv#2251) * Adding working implementation of the DroughtCode class and associated tests * Adding DC to API * Corrections/improvements from review * Adding review suggestion of switching abstract _calculate method to a NotImplementedError * Docstring standardisation * EPPT-2585: Fire Severity Index: Add a DuffMoistureCode class to IMPROVER (metoppv#2250) * Adding working, documented implementation of the DuffMoistureCode class * Adding DMC to API * Changes from review * EPPT-2587: Fire Severity Index: Add a InitialSpreadIndex class to IMPROVER (metoppv#2252) * Adding working ISI class and test suite * Adding ISI to API * Changes from review * Docstring standardisation * EPPT-2589: Fire Severity Index: Add a CanadianForestFireWeatherIndex class to IMPROVER (metoppv#2254) * Adding working plugin and tests for CanadianForestFireWeatherIndex * Changes from review * EPPT-2590: Fire Severity Index: Add a FireSeverityIndex class to IMPROVER (metoppv#2255) * Adding a working FSI calculation and tests * Comments from the review * EPPT-2588: Fire Severity Index: Add a BuildUpIndex class to IMPROVER (metoppv#2253) * Adding working BUI implementation and test suite * Changes from review * EPPT-3031: Standardise FSI plugins (#1) * Removing duplicated tests by moving to base class * Removing redundant input_attribut_mapping tests from child classes * Standardising the tolerance of value comparisons in tests * Fixing type hints to stop Pylance complaining * Removing redundant tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses metoppv#2322, https://github.com/metoppv/mo-blue-team/issues/1013
Description
The DurationSubdivision plugin has been refactored to reduce the peak memory usage. This has primarily involved 1) avoiding having a full cube list containing the fidelity period cubes and 2) using concatenation, rather than merging to combine the target period cubes into a single cube. Following testing, as described in this comment, these changes seem to have been successful in reducing the peak memory usage by ~50% or more.
Testing: