Skip to content

Add partial object sync specification for protocol version 6+ #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JoaoDiasAbly wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add partial object sync specification for protocol version 6+ #413

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion textile/objects-features.textile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,13 @@ h3(#realtime-objects). RealtimeObjects
*** @(RTO5a3)@ If the sequence id matches the previously received sequence id, the client library should continue the sync process
*** @(RTO5a4)@ The objects sync sequence for that sequence identifier is considered complete once the cursor is empty; that is when the @channelSerial@ looks like @<sequence id>:@
*** @(RTO5a5)@ An @OBJECT_SYNC@ may also be sent with no @channelSerial@ attribute. In this case, the sync data is entirely contained within the @ProtocolMessage@
** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list
** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list. The behavior differs based on the protocol version in use:
*** @(RTO5b1)@ For protocol versions prior to 6, the server sends a complete @ObjectState@ for each object in a single @OBJECT_SYNC@ @ProtocolMessage@. Each @ObjectState@ received is stored directly in the @SyncObjectsPool@
Copy link
Collaborator

@lawrence-forooghian lawrence-forooghian Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec describes the behaviour for a single protocol version (that given by CSV2b); we don't need to maintain the description of the v5 behaviour.

*** @(RTO5b2)@ From protocol version 6 onwards, partial object sync is enabled. The server may split a large object across multiple @OBJECT_SYNC@ @ProtocolMessages@, each containing a partial @ObjectState@ for the same object. Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object. The client determines that sync messages relate to the same object by matching the @ObjectState.objectId@. When receiving partial @ObjectState@ messages for the same @objectId@:
Copy link
Collaborator

@lawrence-forooghian lawrence-forooghian Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object.

I think that this is a detail that clients don't need to concern themselves with, and which may confuse rather than enlighten (because it becomes unclear whether they're meant to do something as a result of receiving duplicate channelSerials). All they care about are two things:

  1. is this a new sync sequence?
  2. is this the end of the sync sequence?

— both of which are already addressed in the existing spec

**** @(RTO5b2a)@ If an @ObjectState@ with the given @objectId@ does not yet exist in the @SyncObjectsPool@, store the received @ObjectState@
Copy link
Collaborator

@lawrence-forooghian lawrence-forooghian Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that it makes sense to, in RTO5b2, talk about "when receiving partial ObjectState messages for the same objectId", and then in RTO5b2a describe what to do if that objectId doesn't exist yet; the child is not a sub-scenario of its parent.

**** @(RTO5b2b)@ If an @ObjectState@ with the given @objectId@ already exists in the @SyncObjectsPool@, merge the entries from the received @ObjectState@ into the existing one:
***** @(RTO5b2b1)@ For a map object, merge @ObjectState.map.entries@ from the received @ObjectState@ into the existing @ObjectState.map.entries@. Other fields on the @ObjectState@ envelope (such as @siteTimeserials@) and the map envelope (such as @semantics@) will be identical across all partial messages for the same object; the client may apply these from any message relating to that object
Copy link
Collaborator

@lawrence-forooghian lawrence-forooghian Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a map object

to be determined how, from the given ObjectState? presumably by checking whether ObjectState.map is present

Copy link
Collaborator

@lawrence-forooghian lawrence-forooghian Jan 29, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to describe which of the various ObjectMessages' serialTimestamp properties should be used if we need to tombstone the object when we apply the sync objects pool.

***** @(RTO5b2b2)@ Counter objects have a bounded size and will never be split across multiple sync messages
** @(RTO5c)@ When the objects sync has completed, the client library must perform the following actions in order:
*** @(RTO5c1)@ For each @ObjectState@ in the @SyncObjectsPool@ list:
**** @(RTO5c1a)@ If an object with @ObjectState.objectId@ exists in the internal @ObjectsPool@:
Expand Down
Loading