feat: declarative skill sources with lockfile and curated directory

[mrfelton]

Summary

Adds Declarative Skill Sources — a new feature allowing users to declare remote GitHub skill repositories in rulesync.jsonc. During generate, these sources are automatically fetched, SHA-locked, and placed into .rulesync/skills/.curated/ for use alongside local skills.

Motivation

Currently, fetching skills from external repos requires manually running rulesync fetch for each source. This feature makes skill sourcing declarative and reproducible: declare once in config, and generate handles the rest — with lockfile-based determinism and local-skill precedence.

Changes

Core Implementation

• src/lib/sources.ts — Main orchestrator: resolves source entries, fetches remote skill directories into .curated/, handles local-skill precedence, duplicate detection, and error recovery
• src/lib/sources-lock.ts — Lockfile operations: read/write .rulesync/sources-lock.json with Zod schema validation, immutable set/get helpers
• src/lib/github-client.ts — New resolveRefToSha() method to resolve branch/tag/SHA to commit SHA via repos.getCommit()

Integration

• src/lib/generate.ts — Calls resolveAndFetchSources() before skill generation when sources is configured
• src/cli/commands/generate.ts — Wires --skip-sources and --update-sources CLI flags; validates mutual exclusivity
• src/cli/index.ts — Registers --skip-sources and --update-sources CLI options
• src/config/config.ts — Adds sources field to config schema with SourceEntry type
• src/config/config-resolver.ts — Resolves sources from config file (not CLI); type narrowed with Omit<ConfigParams, 'sources'>
• src/features/skills/skills-processor.ts — loadRulesyncDirs() now discovers curated skills in .curated/ subdirectory with local-precedence filtering
• src/cli/commands/gitignore.ts — Adds .rulesync/skills/.curated/ to gitignore entries
• src/constants/rulesync-paths.ts — New constants for curated skills dir and sources-lock path

Security

• Path traversal validation on skillDir.name (rejects .., /, \ characters)
• Path traversal validation on file paths within skill directories via checkPathTraversal()
• File size limits enforced via MAX_FILE_SIZE (10MB)

Tests (27 new tests)

• src/lib/sources.test.ts (13 tests) — empty sources, skip, update, local precedence, skill filter, duplicates, 404 handling, error recovery, stale pruning, path traversal, GitLab graceful handling
• src/lib/sources-lock.test.ts (12 tests) — empty lock, read/write, validation, immutability
• src/lib/generate.test.ts (4 tests) — sources integration with generate
• src/cli/commands/generate.test.ts (2 tests) — CLI flag wiring and mutual exclusivity

Documentation

• README.md — Full "Declarative Skill Sources" section with config examples, CLI flags, lockfile format, authentication, precedence rules
• skills/rulesync/declarative-skill-sources.md — Standalone skill doc
• skills/rulesync/SKILL.md — Updated table of contents
• config-schema.json — Added sources property to JSON schema

Config Example

{
  "targets": ["copilot", "claudecode"],
  "features": ["rules", "skills"],
  "sources": [
    { "source": "owner/skills-repo" },
    { "source": "org/repo", "skills": ["specific-skill"] },
    { "source": "org/repo@v1.0.0:custom/path" }
  ]
}

CLI

npx rulesync generate                    # Fetches sources using locked refs
npx rulesync generate --skip-sources     # Skip fetching (offline/CI cache)
npx rulesync generate --update-sources   # Force re-resolve all refs

Verification

• All 3724 tests pass
• TypeScript typecheck: 0 errors
• oxlint: 0 warnings
• eslint: 0 warnings

[dyoshikawa]

Thank you for this well-thought-out PR! The lockfile mechanism, .curated/ directory, and declarative source resolution are all solid engineering work.

After reviewing the changes, here's my thinking:

Concern: Complexity in the generate pipeline

The lockfile and .curated/ workflow is convenient, but it introduces significant complexity into the core generate command — which today is a straightforward, deterministic transformation from source files to output files. Embedding remote fetching, SHA resolution, lockfile management, and curated directory cleanup into generate changes its nature fundamentally.

I'm not yet confident that prompt files should be treated like node_modules — as untouchable, version-locked artifacts. The ecosystem is still evolving, and I want to keep the generate path simple and predictable.

Proposal: rulesync install as a new command

Rather than integrating this into generate, I'd like to propose extracting this functionality into a new rulesync install command. This would:

• Keep generate focused on local-only file transformation (no network I/O, no lockfile management)
• Keep fetch as-is — a one-shot, explicit pull from a remote repo
• Introduce install as the declarative, lockfile-based workflow for users who want reproducible, automated skill sourcing

The sources config field, lockfile, .curated/ directory, --skip-sources / --update-sources flags — all of this would live under rulesync install instead. Users who want the full workflow could run rulesync install && rulesync generate, or we could add a convenience flag later.

I'd accept this as an experimental feature under rulesync install. This gives us room to iterate on the UX and semantics without committing to it as part of the core generate contract.

Summary

• fetch: unchanged (one-shot, explicit)
• generate: unchanged (local-only, deterministic)
• install (new, experimental): declarative sources with lockfile + .curated/

Would you be open to restructuring the PR along these lines? The core implementation (sources resolution, lockfile, curated directory, GitHub client changes) can stay largely the same — it's mainly the integration point that shifts from generate to a new install command.

Thanks again for the contribution — the quality of the code and tests is excellent.

[mrfelton]

@dyoshikawa I pushed an alternative version to #1032.

[cm-dyoshikawa]

duplicate #1032