Skip to content

[DO NOT MERGE]: HSM Sync Engine#69

Open
dtkav wants to merge 151 commits intomainfrom
merge-hsm
Open

[DO NOT MERGE]: HSM Sync Engine#69
dtkav wants to merge 151 commits intomainfrom
merge-hsm

Conversation

@dtkav
Copy link
Member

@dtkav dtkav commented Feb 19, 2026

No description provided.

dtkav and others added 30 commits February 16, 2026 16:00
Adds RelayMetrics class that integrates with the obsidian-metrics plugin
when available, with graceful no-op fallback when disabled.
- Banner now automatically detects mobile Obsidian ≥1.11.0 and renders
  as a header button (short text) vs traditional banner (long text)
- Added BannerText type supporting string | { short, long }
- Removed setLoginIcon/clearLoginButton from LoggedOutView
- Removed setMergeButton/clearMergeButton from LiveView
- Removed font-weight bold and text-shadow from desktop banner
- Use Obsidian's native modal title via setTitle()
- Remove duplicate modal-content class nesting
- Use SlimSettingItem for inline toggle and button layout
- Add visible background to toggle track for dark backgrounds
- Fix folder name overflow with ellipsis truncation
- Prevent folder icons from shrinking
Initial foundation for device sync and recommended settings.
Adds test infrastructure for the new hierarchical state machine (HSM)
that will manage document synchronization between disk, local CRDT,
and remote CRDT.

Test harness includes:
- Type definitions for events, effects, and state
- Event factory functions for creating serializable test events
- Stub HSM implementation for test development
- Custom assertions (expectEffect, expectState, etc.)
- 23 passing tests covering core state transitions

Designed for future recording/replay support - all events are
plain serializable objects with snapshot capability.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add node-diff3.d.ts type declarations for portable module resolution
- Add required vaultId to MergeHSMConfig (convention: ${appId}-relay-doc-${guid})
- Add getVaultId callback to MergeManagerConfig and ShadowManagerConfig
- Fix PersistUpdatesEffect to use dbName/update instead of guid/updates
- Fix TimeProvider fallbacks to use DefaultTimeProvider instead of partial object
- Fix SerializableSnapshot structure in recording bridge
- Wire up MergeManager in main.ts with proper vaultId generator
- Update MergeManager tests with required getVaultId

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MergeHSM._path and _guid were private with no public accessors,
causing hsm.path to be undefined and falling back to the guid.

Closes BUG-009.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Document: Clean up existing HSM provider listeners before re-adding
  to prevent accumulation on repeated setup calls.
- HasProvider: Remove status listener once connected instead of relying
  on manual cleanup in destroy().
- LiveViews: Use releaseLock() instead of directly setting userLock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cherry-pick event subscription infrastructure from background_sync into
the YSweetProvider (CBOR-decoded messageEvent, subscribe/unsubscribe
protocol) and wire SharedFolder to forward document.updated events to
MergeManager.handleIdleRemoteUpdate() instead of the previous ydoc
update listener approach.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…sisted CRDT state

MergeManager was created without persistence callbacks, causing every
HSM to receive empty PERSISTENCE_LOADED events. This made every file
appear brand-new and triggered spurious merge conflicts on first edit.

Closes BUG-011.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… directly

RelayCanvasView.release() set canvas.userLock = false without notifying
MergeManager, so the HSM never received RELEASE_LOCK and stayed stuck
in active mode for canvas files. Added Canvas.releaseLock() matching
Document's pattern and wired it into RelayCanvasView.release().

Closes BUG-012.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
On first ACQUIRE_LOCK, if localDoc is empty (no prior CRDT data in
IndexedDB), read current disk contents via getDiskState and call
initializeLocalDoc to populate the CRDT. Also sends DISK_CHANGED so the
HSM tracks disk metadata. Without this, localDoc stayed empty despite
the editor showing file content, causing spurious merge conflicts.

Closes BUG-013.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Corrections 1-7 from specs/persistence-corrections.md:
- Fix Y.Text field name 'content' → 'contents' to match existing IndexedDB data
- Attach IndexeddbPersistence to localDoc via injectable CreatePersistence factory
- Destroy persistence in cleanupYDocs() on RELEASE_LOCK
- Gate active.entering → active.tracking on YDOCS_READY (persistence 'synced')
- Remove loadUpdates callback from MergeManager (persistence loads internally)
- Skip Document.ts IndexeddbPersistence when HSM active mode is enabled
- Retarget uploadDoc() to insert into localDoc via MergeHSM when HSM enabled

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When files are created in shared folders via uploadDoc, the LCA (Last
Common Ancestor) is now initialized to establish the baseline sync point.
This prevents merge conflicts with an empty base when editing newly
created files.

Changes:
- Add initializeLCA() method to MergeHSM for setting up sync baseline
- Call initializeLCA in both HSM active mode and standard upload paths
- Persist PERSIST_STATE effects to IndexedDB via saveMergeState

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track pending write operations and wait for them to complete in
destroy() before closing the database. This prevents data loss when
the persistence layer is torn down while writes are still in flight.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…k error

When clicking the merge conflict banner on a local-only shared folder
(no relayId), checkStale() would call backgroundSync.downloadItem()
which throws "Unable to decode S3RN" since there's no server to
download from.

Now checkStale() checks for relayId before attempting server download.
For local-only folders, it skips the download and just compares the
local CRDT with disk contents.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
dtkav and others added 30 commits February 17, 2026 20:53
Remove moduleNameMapper mocks for node-diff3 and y-indexeddb so tests
run against real implementations. Add a transform rule for src/*.js
files so ts-jest handles the ESM syntax in y-indexeddb.js.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Track OBSIDIAN_FILE_OPENED/OBSIDIAN_FILE_UNLOADED events to maintain
an isObsidianFileOpen flag on each HSM. DiskIntegration checks this
flag before executing WRITE_DISK effects, blocking writes when the
editor has the file open to prevent content duplication.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Simplify invokeForkReconcile to always use diff3 via remoteDoc,
removing the "remote unchanged" shortcut that bypassed three-way merge.
Reset providerSynced on fork creation and RELEASE_LOCK so reconciliation
waits for a fresh sync. Emit REQUEST_PROVIDER_SYNC when releasing lock
with an active fork.

Add patchLCAHash() to async-compute LCA hashes after reconciliation
instead of blocking on the hash computation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, all active documents skipped remote update injection.
Now the skip is gated on enableDirectRemoteUpdates, allowing the
enqueueDownload path (which fetches full server state) to work for
active documents when direct injection is disabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ingestDiskToLocalDoc action that applies pending disk contents to
localDoc. Change idle.synced DISK_CHANGED transition to re-enter
idle.localAhead with ingestion instead of going straight to diverged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The guard checked for sharedFolder.remote during state change
callbacks, which is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ication

Adds the OpCapture module (reversible CRDT op capture) and integrates it
end-to-end: persistence in IDB via y-indexeddb, test harness support, and
consumption during fork reconciliation. When both disk and remote edit the
same content, redundant disk ops are reversed; unique disk ops are dropped
(kept in CRDT). Also fixes diff3 tokenization to use split(/(\n)/) so
adjacent-line changes are handled independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a file was deleted or a shared folder was removed from settings,
the per-document y-indexeddb databases (and folder-level database)
were left behind as orphans in IndexedDB. This adds deleteDatabase
calls in deleteFile() and SharedFolders.delete() after the in-memory
objects are destroyed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes to the fork→diverged→idle-merge path:

1. clearForkKeepDiverged: clear pendingIdleUpdates (already evaluated
   by diff3) and restore pendingDiskContents from localDoc so the
   three-way merge sees the real disk content, not the LCA fallback.

2. invokeIdleThreeWayAutoMerge: read remoteDoc text directly instead
   of applying pendingIdleUpdates to localDoc via raw Y.applyUpdate.
   The raw CRDT merge causes interleaving corruption on conflicting
   edits. If remoteDoc isn't available yet, bail and wait for
   REMOTE_UPDATE to reenter idle.diverged.

3. clearForkAndUpdateLCA: clear pendingIdleUpdates for hygiene.

Also adds:
- MERGE_CONFLICT transition in active.tracking for fork conflicts
  detected during reconcileForkInActive
- OBSIDIAN_SAVE_FRONTMATTER / OBSIDIAN_METADATA_SYNC diagnostic
  events for correlating metadata editor hooks with drift events
- Regression test for the corruption scenario

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Path was stored at construction time and never updated on rename,
causing stale paths in logging and debug tools. Now MergeHSM takes
a getPath callback that computes the current path from the source
of truth (Document.path via SharedFolder's files map).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BackgroundSync.syncDocumentWebsocket only checked userLock before
disconnecting, missing cases where the MergeHSM was actively managing
the document. Also defers awareness cursor updates in RemoteSelections
to avoid re-entrant EditorView.update errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hout LCA

Two changes:
- Populate conflictData when fork-reconcile detects divergence, and add
  hasPreexistingConflict guard in active.entering.reconciling so the
  conflict banner shows when opening a diverged file.
- Allow invokeIdleRemoteAutoMerge to write to disk when there is no LCA
  and no file on disk (initial sync from a remote peer). Block writes
  when there is no LCA but a file exists (up-migration safety).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When Obsidian's auto-save flushes (DISK_CHANGED with dirty === false),
advance the LCA snapshot so it stays fresh during long editing sessions.
This improves conflict detection quality if the user goes offline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Svelte skipped re-rendering the Layers icon because view.tracking was
a getter on the same object reference. Pass tracking as a primitive
boolean prop so Svelte detects the change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When .staging-only marker file is present, npm run build prints a
warning and exits. Use npm run build:force to bypass, or use the
staging build for local development.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deduplicate HSM state change logs in LiveView (only log when statePath
actually changes) and remove per-render debug logs from MetadataRenderer,
PreviewRenderer, and ViewHookPlugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace monkey-patching in E2ERecordingBridge with push-based callbacks
from MergeHSM. The bridge is now a passive sink that receives transition
info (from/to state, event, effects) via MergeManager, eliminating
ObservableMap subscriptions that caused high-frequency Postie deliveries
on every keystroke.

Extract plugin-level debug surface into RelayDebugAPI (window.__relayDebug),
separating the per-folder write path (E2ERecordingBridge) from the
plugin-level read path (RelayDebugAPI).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move HSM state subscription from acquireLock callback into
setConnectionDot() so it is set up synchronously when the ViewActions
component is created. Call setConnectionDot() again after acquireLock
resolves to refresh the tracking state.

Strip editorViewRef from ACQUIRE_LOCK events in serializeEvent() —
the Obsidian View object has circular references that crash
JSON.stringify in the recording bridge.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract editorViewRef.dirty into a local variable for reliable
evaluation, and add crdtLog entries for LCA advance/skip to aid
debugging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Keep fork alive after fork-reconcile conflict (gates syncLocalToRemote,
  preserves OpCapture data for disk op reversal)
- resolveConflict action: reverse disk ops via OpCapture, merge remote
  CRDT into local, DMP resolved text, then sync with converged history
- Skip idle-merge when conflictData already set (prevent re-merge after
  fork-reconcile conflict)
- Remove DISPATCH_CM6 from resolve (y-codemirror binding handles it)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes for fork-based conflict detection:

1. reconcileForkInActive: read remoteDoc's actual state vector instead of
   the cached _remoteStateVector, which may be stale when updates arrive
   via provider sync rather than REMOTE_UPDATE events.

2. ensureRemoteDoc: skip seeding remoteDoc from localDoc when a fork
   exists, preventing the local disk edit from leaking to the server
   before conflict resolution.

3. connectForForkReconcile auto-disconnect: only tear down the idle
   provider when transitioning to another idle state, not when
   transitioning to active. acquireLock adopts the existing provider
   and ProviderIntegration.

Also: MergeManager handles REQUEST_PROVIDER_SYNC internally, poll()
replaces pollDiskState() with fork connection retry, ProviderIntegration
SYNC_TO_REMOTE gates on isActive() instead of getLocalDoc().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ViewHookPlugin and TextViewPlugin observed remoteDoc's ytext for UI
updates, causing MetadataRenderer to fire before localDoc was updated
and push stale frontmatter to the metadata visual editor. This created
a repeating drift loop where the metadata editor continuously reasserted
its stale state.

- ViewHookPlugin/TextViewPlugin: observe localYText instead of ytext
- differ actionLine/differencesView: write to getWritableDoc() (localDoc)
  instead of ydoc (remoteDoc)
- RemoteSelections: read cursor positions from localDoc with fallback,
  disable cursors when fork gate is blocking traffic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Edits typed before the HSM enters active.tracking were silently dropped,
causing merge conflicts with disk. CM6_CHANGE events are now accumulated
during idle and entering states and replayed on tracking entry.

Also gates editor plugins on hsm.awaitState instead of whenSynced/whenReady
to prevent crashes from accessing localDoc before HSM is active.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove makeCRDTUpdateLogger, setupCRDTLogging, describeOrigin, and
_remoteDocLogHandler. These fired on every Y.Doc update, decoding state
vectors each time, producing noisy logs with no diagnostic value.

One-time crdtLog calls in createYDocs and persistence synced are kept.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After a file rename, CM6Integration held the old vault-relative path in
expectedVaultPath. Its isEditorShowingExpectedFile() then returned false
for every subsequent editor update, silently dropping all editor→CRDT
changes. The localDoc stayed empty while the user kept typing.

Replace all path-based identity checks with GUID comparison:
- CM6Integration takes an EditorValidityCheck callback instead of a path
- HSMEditorPlugin provides the callback using document GUID comparison
- View-reuse detection compares document GUIDs, not TFile paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DifferencesView.onunload unconditionally called onResolve, so closing
the diff view with Ctrl+W sent RESOLVE to the HSM and silently dropped
the conflict. Track explicit resolution with a flag and call the new
onCancel callback on unresolved close, sending CANCEL to return the HSM
to active.conflict.bannerShown.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
handleIdleWriteDisk now creates files (and parent folders) that don't
yet exist on disk instead of silently returning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After idle-merge completes, update the HSM's disk hash/mtime to match
the new LCA so subsequent disk-read events don't trigger spurious diffs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant