Open
Conversation
- New -g/--checkpoint <file> flag saves progress to a JSON file every N minutes (default 10, override with -w/--checkpointinterval). - On re-run with the same -g path, completed directories and computers are skipped automatically so the scan picks up where it left off. - CheckpointManager (thread-safe singleton) tracks scanned dirs and computers in ConcurrentDictionary sets; writes atomically via temp-file + rename to survive mid-write kills. - TreeWalker marks each directory entered; ShareFinder marks each computer after its shares are queued. - Final checkpoint is always written when the scan completes cleanly.
- Add Microsoft.NETFramework.ReferenceAssemblies to both projects to allow building net451 targets on Linux with dotnet SDK. - Add explicit <Reference HintPath> items backed by $(NuGetPackageRoot) so old-style .csproj PackageReferences resolve on Linux/dotnet SDK. - Manually import dnMerge.targets so the merged single-file exe is produced correctly (dnMerge embeds NLog, Nett, CommandLineParser and SnaffCore as compressed resources inside Snaffler.exe). - Include compiled Snaffler.exe in this branch for quick testing.
- Mark directory as scanned at end of WalkTree (after all file/subdir tasks are queued) instead of on entry. Prevents files being silently dropped when the process is killed between the entry mark and the actual file-task execution. Inspired by analysis of upstream PR SnaffCon#171. - CheckpointManager.TryLoad(): prune child-directory entries whose parent is already in the completed set. The parent being marked means WalkTree will skip it entirely, making any child entries unreachable and dead weight. Pruning keeps the in-memory set lean and now reports how many redundant entries were dropped at resume time.
- CheckpointManager: if a directory path is given (e.g. '-g .'), auto-
generate 'snaffler_checkpoint.json' inside it rather than trying to
use the directory itself as the file. Previously File.Exists('.') was
false so TryLoad was never called, and File.Copy to '.' silently
failed, leaving data forever stuck in the .tmp file.
- Promote checkpoint skip messages from Mq.Trace to Mq.Info so they
are visible in normal (non-verbose) output.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a checkpoint/resume system to Snaffler, enabling scan progress to be persisted to a JSON file and automatically resumed on subsequent runs. This is useful for long-running engagements against large environments where scans may be interrupted.
Changes:
- New
CheckpointManagersingleton andCheckpointDataPOCO that track scanned directories and computers usingConcurrentDictionary, with periodic atomic saves via aSystem.Timers.Timer. - CLI flags
-g/--checkpointand-w/--checkpointintervalwired into the existing argument parser andOptionsclass. - Skip logic integrated at three levels:
ShareDiscovery(computers),FileDiscovery(top-level paths), andTreeWalker.WalkTree(individual directories), with directories marked complete only after all sub-tasks are dispatched.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
SnaffCore/Checkpoint/CheckpointData.cs |
New [DataContract] POCO for serializable checkpoint state |
SnaffCore/Checkpoint/CheckpointManager.cs |
New singleton managing in-memory state, save/load, and deduplication |
SnaffCore/Config/Options.cs |
Added CheckpointFile and CheckpointIntervalMinutes properties |
Snaffler/Config.cs |
Added -g and -w CLI argument parsing |
SnaffCore/SnaffCon.cs |
Checkpoint initialization, timer lifecycle, and skip logic in discovery methods |
SnaffCore/ShareFind/ShareFinder.cs |
Marks computer as scanned after share discovery completes |
SnaffCore/TreeWalk/TreeWalker.cs |
Skip already-scanned directories and mark at end of WalkTree |
SnaffCore/SnaffCore.csproj |
Added System.Runtime.Serialization reference, checkpoint compile items, and cross-platform build workarounds |
Snaffler/Snaffler.csproj |
Added cross-platform build workarounds (HintPaths, dnMerge targets import) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- CheckpointData: clarify ScannedDirectories doc comment ('initiated' -> 'fully dispatched')
- SnaffCon: move IsComputerScanned check after DNS resolution to avoid IP vs hostname mismatch
- CheckpointManager: use File.Replace/File.Move for atomic checkpoint writes instead of File.Copy+Delete
- CheckpointManager: fix Initialize() doc comment (called inside SnaffCon ctor, not before)
- CheckpointManager: replace O(n²) deduplication loop with O(n log n) sort + single linear pass
- UltraSnaffCore.csproj: add System.Runtime.Serialization reference and Checkpoint compile items
- Config: validate checkpoint interval is >= 1 minute before accepting
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 11 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
…d paths - TreeWalker.WalkTree: call MarkDirectoryScanned before both early returns in the SCCM ContentLib block so those directories are checkpointed and not re-scanned on resume. - CheckpointManager.TryLoad: clear _scannedDirectories and _scannedComputers in the catch block so a partial load failure truly resets to fresh state instead of silently skipping entries.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 11 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- TreeWalker.WalkTree: downgrade checkpoint skip message from Mq.Info to Mq.Trace to avoid flooding log output during resume, consistent with other per-item skip messages in the codebase. - Config.cs: remove 'n' from the unused-letters comment; n is already taken by compTargetArg.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checkpoint / Resume System
Adds the ability to save scan progress to a checkpoint file and resume from where Snaffler left off — useful for long-running engagements against large environments where the scan may be interrupted.
New Flags
-g <path>/--checkpoint <path>snaffler_checkpoint.jsoninside it). Enables checkpointing. If the file already exists, automatically resumes without a separate flag.-w <minutes>/--checkpointinterval <minutes>Usage
First run — save progress every 5 minutes into the current directory:
This creates
./snaffler_checkpoint.json. If the scan is interrupted (Ctrl-C, network drop, VM reboot), the file retains everything completed so far.Resume — run the exact same command again:
Snaffler detects the existing checkpoint file and prints:
Only the remaining targets are processed — nothing already scanned is repeated.
How It Works
CheckpointManagersingleton holds twoConcurrentDictionarysets: scanned directories and scanned computers.System.Threading.Timerfires every N minutes and atomically writes the state to disk via a temp-file + rename, preventing corruption if killed mid-write.ShareDiscovery— entire computers are skipped if already scanned.FileDiscovery— top-level path targets are skipped.TreeWalker.WalkTree— individual directories are skipped recursively.WalkTree(after all file tasks are dispatched), not on entry — preventing silent data loss if the process is killed between marking and execution.\\SRV\SHAREis in the set,\\SRV\SHARE\subdiris removed since the parent skip makes it unreachable anyway.DataContractJsonSerializerfromSystem.Runtime.Serialization.Benefits
CheckpointManageris only instantiated when-gis supplied.Files Changed
SnaffCore/Checkpoint/CheckpointData.cs[DataContract])SnaffCore/Checkpoint/CheckpointManager.csSnaffCore/Config/Options.csCheckpointFile,CheckpointIntervalMinutesSnaffler/Config.cs-g/--checkpointand-w/--checkpointintervalCLI flagsSnaffCore/SnaffCon.csSnaffCore/ShareFind/ShareFinder.csGetComputerSharescompletesSnaffCore/TreeWalk/TreeWalker.csWalkTreeSnaffCore/SnaffCore.csprojSystem.Runtime.Serializationreference, new Compile items