-
Notifications
You must be signed in to change notification settings - Fork 46
Open
Description
Context
Raised in PR #464 review.
Problem
max_retries in EvalMetadata and the retry loop in _process_one_mp has confusing off-by-one semantics:
max_retries=0→ 1 total attempt (no retries)max_retries=2→ 3 total attempts (1 initial + 2 retries)
The loop uses while retry_count <= max_retries, so the total number of attempts is always max_retries + 1. This creates ambiguity in log messages like "failed after {max_retries} retries" and is easy to misread.
Suggestion
Rename max_retries → max_attempts across EvalMetadata, _process_one_mp, _execute_single_attempt, and the CLI argument, then adjust the loop:
for attempt in range(max_attempts):
...This makes the semantics obvious: max_attempts=1 means one try, max_attempts=3 means three tries.
Scope: EvalMetadata.max_retries, the retry loop in evaluation.py, log messages, CLI args, and any callers that set the value.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels