Skip to content

Conversation

@fresh-borzoni
Copy link
Contributor

Summary

closes #263

  • Make write queueing synchronous by replacing tokio::sync::Mutex with parking_lot::Mutex in the core accumulator and writer client
  • upsert(), delete(), append() now queue writes synchronously (no await needed) and return a WriteResultHandle
  • WriteResultHandle supports both fire-and-forget (ignore the handle) and per-record ack (await handle.wait())
  • flush() remains async (real network I/O)
  • Apply same pattern to Python and C++ bindings

@fresh-borzoni
Copy link
Contributor Author

@luoyuxia @leekeiabstraction
Apreciate review here, PTAL 🙏

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes unnecessary async from the write queueing path by switching internal queue synchronization to parking_lot::Mutex, making upsert(), delete(), and append() queue synchronously while still allowing async acknowledgement via a returned handle/future; async behavior is retained for flush() since it performs network I/O.

Changes:

  • Replace async queueing (tokio::sync::Mutex) with synchronous locking (parking_lot::Mutex) in the core write accumulator and writer client.
  • Update Rust API call sites (tests/examples/docs) to remove the first await on queueing methods while keeping per-record ack via awaiting the returned handle.
  • Apply the same synchronous-queue + async-ack pattern to Python and C++ bindings (including a new Python WriteResultHandle).

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
crates/fluss/src/client/write/accumulator.rs Switch queue locks to parking_lot::Mutex and make queue operations synchronous (append/ready/drain/re_enqueue).
crates/fluss/src/client/write/writer_client.rs Make send() synchronous and remove awaits on accumulation.
crates/fluss/src/client/write/sender.rs Adjust sender to use synchronous accumulator APIs; keep async for network/metadata updates.
crates/fluss/src/client/write/mod.rs Update docs for new call pattern around WriteResultFuture.
crates/fluss/src/client/table/append.rs Make append/append_arrow_batch synchronous queueing returning WriteResultFuture.
crates/fluss/src/client/table/upsert.rs Make upsert/delete synchronous queueing returning WriteResultFuture.
crates/fluss/tests/integration/log_table.rs Remove await from queueing calls to match the new synchronous API.
crates/fluss/tests/integration/kv_table.rs Remove await from queueing calls and keep per-record ack via awaiting the returned handle.
crates/examples/src/example_table.rs Update example to use synchronous queueing + async flush.
crates/examples/src/example_kv_table.rs Update KV example for synchronous queueing + optional per-record ack.
crates/examples/src/example_partitioned_kv_table.rs Update partitioned KV example for synchronous queueing + optional per-record ack.
bindings/python/src/write_handle.rs Add Python WriteResultHandle with wait() for per-record ack.
bindings/python/src/upsert.rs Make Python upsert/delete synchronous and return WriteResultHandle; keep flush() async.
bindings/python/src/table.rs Make Python append APIs return WriteResultHandle; convert flush() to async/awaitable.
bindings/python/src/lib.rs Export/register WriteResultHandle.
bindings/python/example/example.py Update Python example to reflect synchronous queueing and awaiting flush()/handle.wait().
bindings/cpp/src/lib.rs Remove runtime blocking for append queueing; keep blocking for waiting/flush.
README.md Update Rust snippet for synchronous queueing and add flush.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@fresh-borzoni
Copy link
Contributor Author

fixed failing tests, should be fine now

@fresh-borzoni fresh-borzoni force-pushed the remove-async-queue-path branch from cefd4b7 to 0802788 Compare February 7, 2026 12:31
Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fresh-borzoni Thanks for the pr. LGTM!

@luoyuxia luoyuxia merged commit 26edb7a into apache:main Feb 7, 2026
32 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove unnecessary async from write queueing path

2 participants