refactor(rclone): remove feature flag and make rclone mounts core functionality#31
Merged
patchmemory merged 9 commits intomainfrom Feb 3, 2026
Merged
Conversation
- Enhanced Neo4jGraph.commit_scan() to write files/folders via Neo4jClient - Returns verification dict with counts (db_scan_exists, db_files, db_folders, db_verified) - Updated InMemoryGraph.commit_scan() to return matching dict structure - Added fallback to in-memory when SCIDK_GRAPH_BACKEND=neo4j but env incomplete - Added logging for backend selection at startup - Added comprehensive tests with mocked Neo4j driver Acceptance criteria met: ✅ With SCIDK_GRAPH_BACKEND=neo4j and env set, commit writes nodes/edges ✅ Post-commit verification returns counts ✅ Fallback to in-memory when env incomplete with logging ✅ Mocked-driver tests pass (4 new tests) ✅ /map schema parity holds (schema_triples signatures match) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add pytest auto-cleanup hook in tests/conftest.py (keeps 3 most recent sessions) - Add 'make clean-test-artifacts' target to manually clean pytest cache and sessions - Prevents test artifact accumulation (48MB+ of temp files and databases) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Extract nested functions and initialization logic from app.py into modular components to improve maintainability and testability. NEW MODULES CREATED: - scidk/core/channel_config.py: Channel-based feature flag defaults - scidk/core/interpreter_enablement.py: Interpreter enablement logic - scidk/core/neo4j_config.py: Neo4j params and graph backend creation - scidk/core/providers_init.py: Filesystem provider initialization - scidk/core/rclone_mounts_loader.py: Rclone mounts metadata loader - scidk/core/rclone_settings.py: Rclone interpretation settings - scidk/core/telemetry_loader.py: Telemetry data loader from SQLite - scidk/services/scan_index_service.py: Scan filesystem index builder CHANGES: - app.py: 625 → 161 lines (slim create_app with extracted logic) - Update imports in routes (api_neo4j.py, api_tasks.py) - Update test imports (test_neo4j_commit_flow.py, test_neo4j_commit_folders.py) TESTING: - All 148 tests passing (2 skipped E2E) - No functionality changes, pure refactoring - Backup created at scidk/app.py.pre-phase2-refactor BENEFITS: - Easier to understand create_app() flow - Each initialization module is independently testable - Reduced cognitive load when reading app.py - Functions can be reused in other contexts - Cleaner separation of concerns 🎯 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add /api/admin/cleanup-test-scans endpoint to remove accumulated test scans from the database that were showing up in the UI after test runs. PROBLEM: - 711 scans accumulated in database (681 were test scans) - Test scans with /tmp/, /nonexistent/, and scidk-e2e paths persisted - Cluttered UI when running scidk-serve after tests - Restarting server didn't help (scans in SQLite, not just memory) SOLUTION: 1. New cleanup endpoint: POST /api/admin/cleanup-test-scans - Removes scans from SQLite database - Clears from in-memory registry - Deletes related scan_items, scan_progress records - Returns counts and scan IDs 2. Automatic cleanup in tests/conftest.py - Runs before each test session - Removes test scans from database - Prevents accumulation over time - Gracefully handles missing tables/databases TESTING: - Tested cleanup endpoint: 681 test scans deleted successfully - Verified: 711 → 30 scans remaining (only real scans) - Confirmed tests still pass with auto-cleanup USAGE: # Manual cleanup via API curl -X POST http://127.0.0.1:5000/api/admin/cleanup-test-scans # Automatic cleanup # Runs automatically when pytest starts (via conftest.py) 🎯 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
PROBLEM: - Delete button in UI wasn't working properly - Scans were removed from memory but persisted in SQLite - Refreshing page showed "deleted" scans again ROOT CAUSE: - DELETE /scans/<scan_id> only removed from in-memory registry - When state.backend=sqlite, scans persist to database - UI reloads from database on page refresh FIX: - Update api_scan_delete() to also delete from SQLite - Remove scan from scans table - Cascade delete scan_items, scan_progress, scan_selection_rules - Gracefully handle missing tables (older schemas) TESTING: - All 148 tests passing - Delete now works correctly in both backends 🎯 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ctionality Rclone mounts are no longer behind a feature flag. Changes: - Removed _feature_rclone_mounts() function from app.py, api_providers.py, ui.py - Removed SCIDK_RCLONE_MOUNTS and SCIDK_FEATURE_RCLONE_MOUNTS checks - Updated providers_init.py to always enable rclone provider - Removed feature flag defaults from config.py channel settings - Updated settings.html to always show rclone mounts section - Updated README.md to remove feature flag instructions - Updated .env.* files with notes about rclone being core functionality - Regenerated feature-flags.md index Rclone mount manager UI and API endpoints are now always available. The rclone provider is automatically enabled for remote validation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the
SCIDK_RCLONE_MOUNTS/SCIDK_FEATURE_RCLONE_MOUNTSfeature flag and makes rclone mounts part of core functionality.Changes
Code Changes:
_feature_rclone_mounts()function and unconditionally enable rclone mounts_feature_rclone_mounts()functionrclone_mounts_feature=Trueto templateDocumentation Changes:
Impact
Files Changed
12 files changed: 23 insertions(+), 65 deletions(-)
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com