-
Notifications
You must be signed in to change notification settings - Fork 26
[TASK-243] Add subscribe_partition_buckets method and partitions supp… #273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@luoyuxia @leekeiabstraction PTAL 🙏 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds first-class support for subscribing to multiple (partition_id, bucket_id) pairs in one call (subscribe_partition_buckets) across the Rust core API and the C++/Python bindings, plus supporting examples and an integration test. It also fills a gap in the C++ bindings by exposing CreatePartition so partitioned-table workflows can be demonstrated end-to-end.
Changes:
- Add
subscribe_partition_bucketsto Rust scanners (record + batch) and expose it via C++/Python bindings. - Add
CreatePartitionto C++ bindings and wire it through the Rust CXX bridge. - Extend C++/Python examples and add an integration test validating batch partition subscription behavior.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/fluss/tests/integration/log_table.rs | Adds integration coverage for subscribe_partition_buckets on partitioned tables |
| crates/fluss/src/client/table/scanner.rs | Implements subscribe_partition_buckets and updates partitioned-table error messaging |
| bindings/python/src/table.rs | Exposes subscribe_partition_buckets on Python LogScanner and updates to_* docs/errors |
| bindings/python/fluss/init.pyi | Adds typing stub for subscribe_partition_buckets |
| bindings/python/example/example.py | Demonstrates subscribe_partition_buckets usage for partitioned table scans |
| bindings/cpp/src/table.cpp | Adds C++ wrapper call LogScanner::SubscribePartitionBuckets |
| bindings/cpp/src/lib.rs | Extends CXX bridge with partition subscriptions + create_partition plumbing |
| bindings/cpp/src/admin.cpp | Implements Admin::CreatePartition in the C++ binding |
| bindings/cpp/include/fluss.hpp | Adds PartitionBucketSubscription struct and declares new APIs |
| bindings/cpp/examples/example.cpp | Adds a partitioned-table example using CreatePartition and batch subscriptions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@fresh-borzoni hi, could you please rebase main branch? |
afda850 to
301c13c
Compare
|
@luoyuxia Rebased |
luoyuxia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fresh-borzoni Thanks for the pr. LGTM overall, Left minor comments.
301c13c to
209bb64
Compare
209bb64 to
f04eae3
Compare
|
@luoyuxia TY for the review. |
luoyuxia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fresh-borzoni Thanks for the pr. LGTM!
Summary
closes #243
subscribe_partition_bucketsmethod across Rust core, C++ and Python bindings to subscribe to multiple partition+bucket combinations at onceCreatePartitionto C++ bindings (was missing, already existed in Python)CreatePartition,SubscribePartitionandSubscribePartitionBucketsusagesubscribe_partition_bucketsexample to Python examplesubscribe_partition_buckets