Add EP cache versioning to avoid crashes from outdated caches after ORT updates#27660
Open
yiliangbetter wants to merge 1 commit intomicrosoft:mainfrom
Open
Add EP cache versioning to avoid crashes from outdated caches after ORT updates#27660yiliangbetter wants to merge 1 commit intomicrosoft:mainfrom
yiliangbetter wants to merge 1 commit intomicrosoft:mainfrom
Conversation
…T version When set to "1", execution provider cache directory options (CoreML ModelCacheDirectory, TensorRT trt_engine_cache_path, MIGraphX migraphx_model_cache_dir, etc.) are automatically suffixed with the ONNX Runtime version string. This invalidates caches when ORT is updated and avoids crashes from loading outdated EP caches. Also adds get_versioned_ep_cache_path() in Python for manual path versioning. Made-with: Cursor
1d69fdd to
33a90e8
Compare
|
@yiliangbetter Disclaimer: I never wrote a single line of C++ so my review is eventual bias.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds support for execution provider (EP) cache versioning so that cache directories are tied to the ONNX Runtime version. When enabled, known EP cache paths are automatically suffixed with the ORT version (e.g.
.caches→.caches/1.20.0). That invalidates old caches when ORT is upgraded and reduces crashes or bad behavior from loading incompatible cached artifacts.Changes:
session.ep_cache_use_ort_version(C++:kOrtSessionOptionsEpCacheUseOrtVersion). When set to"1", known EP cache directory options are suffixed with the ORT version. Set this before appending execution providers.ModelCacheDirectory; TensorRTtrt_engine_cache_pathandtrt_timing_cache_path; MIGraphXmigraphx_model_cache_dir; NvTensorRtRtxnv_runtime_cache_path.onnxruntime.get_versioned_ep_cache_path(base_path)for building versioned cache paths manually (e.g. for provider options).ep_cache_versioning.cc/.happly version suffixes using theORT_VERSIONmacro; wired intoSessionOptionsAppendExecutionProviderandInitializeSessionso both config options and provider options get versioned paths when the flag is on.InferenceSessiondocstring updated with a short “EP cache versioning” section and usage notes.test_session_options.ccfor config and provider path versioning (on/off, multiple EPs, empty paths, unknown providers/options, case insensitivity, no mutation of inputs).Motivation and Context
Many execution providers (CoreML, TensorRT, MIGraphX, etc.) compile and cache artifacts to speed up session creation. Those caches are tied to the ORT build; after an upgrade, loading them can cause crashes, wrong results, or ABI issues in detail please see (#27487)
This change:
How to use
Automatic (session option):