WIP instructions for running with S3.#232
WIP instructions for running with S3.#232dpsi wants to merge 1 commit intomlcommons:TF_ObjectStoragefrom
Conversation
|
MLCommons CLA bot: |
dpsi
force-pushed
the
darien-TF_ObjectStorage
branch
from
fea5e52 to
93d2426
Compare
…s3dlio) - Add URI-based storage handler with 3 library backends - Integrate s3dlio v0.9.40 native API (put_bytes, get_bytes, list) - Apply PR #232 fix for empty data_dir handling - Add comprehensive test suite with 3 validated implementations - Organize project structure (tests/, docs/, patches/) - Document MLP vs dpsi architectural comparison Changes preserved in patches/ directory for flexible integration approach. Test results: All 3 libraries working (s3torch: 30s, minio: 15s, s3dlio: 31s)
…s3dlio) - Add URI-based storage handler with 3 library backends - Integrate s3dlio v0.9.40 native API (put_bytes, get_bytes, list) - Apply PR mlcommons#232 fix for empty data_dir handling - Add comprehensive test suite with 3 validated implementations - Organize project structure (tests/, docs/, patches/) - Document MLP vs dpsi architectural comparison Changes preserved in patches/ directory for flexible integration approach. Test results: All 3 libraries working (s3torch: 30s, minio: 15s, s3dlio: 31s)
|
This PR could / should be closed, as all changes here are now included in my fork of this repository, which will be PR'd back into the main branch shortly. Leaving for TF leads to determine if this should be closed. |
2 participants
No description provided.