From 32871aa8df41f9bfa6d1204338ec023ff12749f1 Mon Sep 17 00:00:00 2001 From: Adam Patch Date: Tue, 3 Feb 2026 15:39:38 -0500 Subject: [PATCH 1/3] feat(ops): automate feature flags index generation and verification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements task:ops/mvp/feature-flags-index-automation: - CI now verifies dev/features/feature-flags.md is up-to-date - Added submodules: recursive checkout in CI for dev/ access - Created DEVELOPMENT.md with comprehensive documentation: - make flags-index usage - CI verification explanation - Optional pre-commit hook setup instructions - Feature flag best practices Implementation: - Updated .github/workflows/ci.yml with docs-check step - docs-check runs before pytest to catch stale documentation early - Makefile targets already existed (flags-index, docs-check) - Pre-commit hook example provided (optional for developers) Acceptance Criteria: ✅ Makefile has target: flags-index (already existed) ✅ CI/docs step verifies feature-flags.md is up-to-date ✅ Pre-commit snippet documented for optional local use Benefits: - Prevents stale feature flag documentation - Ensures environment variables are always documented - Provides clear workflow for contributors - Catches documentation drift in CI before merge Task: task:ops/mvp/feature-flags-index-automation DoD: ✅ docs, demo_steps 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .github/workflows/ci.yml | 5 ++ DEVELOPMENT.md | 133 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 138 insertions(+) create mode 100644 DEVELOPMENT.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index aad0726..e5f0345 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -12,6 +12,8 @@ jobs: steps: - name: Checkout uses: actions/checkout@v4 + with: + submodules: recursive - name: Set up Python uses: actions/setup-python@v5 with: @@ -20,6 +22,9 @@ jobs: run: | python -m pip install --upgrade pip pip install -e .[dev] + - name: Check feature flags index is up-to-date + run: | + make docs-check - name: Run pytest (exclude E2E) run: | python -m pytest -q -m "not e2e" diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000..c0baa21 --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,133 @@ +# Development Guide + +## Feature Flags Documentation + +SciDK uses environment variables for feature flags and configuration. The index of all feature flags is automatically generated and maintained in `dev/features/feature-flags.md`. + +### Keeping Feature Flags Index Up-to-Date + +#### Manual Update + +To regenerate the feature flags index: + +```bash +make flags-index +``` + +Or directly: + +```bash +python -m dev.tools.feature_flags_index --write +``` + +#### CI Verification + +The CI pipeline automatically checks that the feature flags index is up-to-date. If you add or modify environment variable usage in the code, you must regenerate the index before pushing: + +```bash +make flags-index +git add dev/features/feature-flags.md +git commit -m "docs: update feature flags index" +``` + +The CI check will fail with a diff if the index is stale. + +#### Pre-commit Hook (Optional) + +You can set up a git pre-commit hook to automatically regenerate the feature flags index on each commit. This ensures you never forget to update the documentation. + +**Setup:** + +1. Create or edit `.git/hooks/pre-commit`: + +```bash +#!/bin/sh +# Pre-commit hook: regenerate feature flags index if any Python files changed + +# Check if any Python files were modified +if git diff --cached --name-only | grep -q '\.py$'; then + echo "Python files changed, regenerating feature flags index..." + python -m dev.tools.feature_flags_index --write + + # Check if feature-flags.md was modified + if ! git diff --exit-code --quiet dev/features/feature-flags.md; then + echo "Feature flags index updated. Adding to commit..." + git add dev/features/feature-flags.md + fi +fi + +exit 0 +``` + +2. Make it executable: + +```bash +chmod +x .git/hooks/pre-commit +``` + +**How it works:** +- Runs before every commit +- If Python files were modified, regenerates the feature flags index +- If the index changed, automatically stages it for the commit +- Ensures your commits always include updated documentation + +**Note:** This hook is optional. The CI check will catch stale documentation even if you don't use the hook. + +### Adding New Feature Flags + +When adding new environment variables to the code: + +1. Use the standard pattern with `os.environ.get()` +2. Include default values where appropriate +3. Regenerate the index: `make flags-index` +4. Commit both the code and the updated index + +Example: + +```python +# Good: includes default +enabled = os.environ.get('SCIDK_MY_FEATURE', '0') == '1' + +# Also good: documents absence +api_key = os.environ.get('SCIDK_API_KEY') # None if not set +``` + +### Viewing the Feature Flags Index + +To view the current index without writing: + +```bash +python -m dev.tools.feature_flags_index +``` + +Or view the file directly: + +```bash +cat dev/features/feature-flags.md +``` + +## Testing + +### Running Tests + +```bash +# Unit tests only +make unit + +# Integration tests +make integration + +# E2E tests (requires running server) +BASE_URL=http://localhost:5000 make e2e + +# All tests +make check +``` + +### Test Organization + +- Unit tests: Fast, no external dependencies +- Integration tests: May use database, filesystem +- E2E tests: Full browser automation with Playwright + +See `Makefile` for more test targets including headed mode, parallel execution, and debugging options. From 4652061509cb92f7ab169ac5f6b0a589ffe460b2 Mon Sep 17 00:00:00 2001 From: Adam Patch Date: Tue, 3 Feb 2026 15:40:54 -0500 Subject: [PATCH 2/3] chore(dev): update submodule pointer for completed task --- dev | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev b/dev index dbc5766..ae891d1 160000 --- a/dev +++ b/dev @@ -1 +1 @@ -Subproject commit dbc5766f2dfd1b7ee23f18b71ebaa768e81c7571 +Subproject commit ae891d1abbac862f5985fcaa63162bd483436efd From 4672c961b86689c8f10ac7207cf9ed3be89e15bf Mon Sep 17 00:00:00 2001 From: Adam Patch Date: Tue, 3 Feb 2026 15:48:04 -0500 Subject: [PATCH 3/3] fix(ci): remove submodule checkout and docs-check step The dev/ submodule points to a private repository (scidk-dev) that is not accessible to GitHub Actions. This causes CI to fail when trying to clone the submodule. Changes: - Remove 'submodules: recursive' from checkout step - Remove 'Check feature flags index is up-to-date' step - Update DEVELOPMENT.md to note CI verification is currently disabled - CI verification will be re-enabled when submodule is public The docs-check can still be run locally via 'make docs-check'. Fixes CI failure on PR #30 --- .github/workflows/ci.yml | 5 ----- DEVELOPMENT.md | 12 +++++++++--- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e5f0345..aad0726 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -12,8 +12,6 @@ jobs: steps: - name: Checkout uses: actions/checkout@v4 - with: - submodules: recursive - name: Set up Python uses: actions/setup-python@v5 with: @@ -22,9 +20,6 @@ jobs: run: | python -m pip install --upgrade pip pip install -e .[dev] - - name: Check feature flags index is up-to-date - run: | - make docs-check - name: Run pytest (exclude E2E) run: | python -m pytest -q -m "not e2e" diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index c0baa21..d298662 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -22,7 +22,15 @@ python -m dev.tools.feature_flags_index --write #### CI Verification -The CI pipeline automatically checks that the feature flags index is up-to-date. If you add or modify environment variable usage in the code, you must regenerate the index before pushing: +**Note:** CI verification is currently disabled as the `dev/` directory is a private submodule not accessible to GitHub Actions. The verification step will be re-enabled when the submodule is made public or when feature flags documentation is moved to the main repository. + +For local development, you can verify the index is up-to-date: + +```bash +make docs-check +``` + +If you add or modify environment variable usage in the code, regenerate the index before pushing: ```bash make flags-index @@ -30,8 +38,6 @@ git add dev/features/feature-flags.md git commit -m "docs: update feature flags index" ``` -The CI check will fail with a diff if the index is stale. - #### Pre-commit Hook (Optional) You can set up a git pre-commit hook to automatically regenerate the feature flags index on each commit. This ensures you never forget to update the documentation.