Handle deprecated models detection

[kashifpk]

Description

Implement detection of deprecated models when fetching from provider APIs. When a previously stored model is no longer returned by the API, flag it as deprecated and notify the user without auto-removing it.

Requirements

Detection Logic

When fetching models from a provider API:

1. Get list of currently stored model IDs for that provider
2. Get list of fetched model IDs from API
3. Find models in storage but not in API response → deprecated
4. Return deprecated list in fetch response

Response Format

POST /models/fetch response should include:

{
  "status": "success",
  "provider": "openai",
  "models_added": 15,
  "models_updated": 3,
  "deprecated": [
    {
      "id": "gpt-4-turbo-preview",
      "display_name": "GPT-4 Turbo Preview",
      "reason": "No longer available in provider API"
    }
  ]
}

Storage Handling

Do NOT auto-remove deprecated models. Instead:

• Keep them in storage
• Optionally add deprecated: true flag
• Let frontend notify user
• User decides whether to remove

Frontend Notification (for reference)

When deprecated models detected:

⚠️ Model Update Available

The following models are no longer available:
- gpt-4-turbo-preview (OpenAI)
- claude-2.1 (Anthropic)

These models may stop working. Would you like to:
[Remove Deprecated Models] [Keep Them] [Learn More]

Implementation Steps

1. Add detection to fetch logic:

async def fetch_models(provider_id: str) -> dict:
    config = load_models_config()
    provider = registry.get(provider_id)
    
    # Get currently stored model IDs
    stored_ids = set(config.providers[provider_id].models.keys())
    
    # Fetch from API
    fetched_models = await provider.fetch_models()
    fetched_ids = {m.id for m in fetched_models}
    
    # Detect deprecated
    deprecated_ids = stored_ids - fetched_ids
    deprecated = [
        {
            "id": model_id,
            "display_name": config.providers[provider_id].models[model_id].display_name,
            "reason": "No longer available in provider API"
        }
        for model_id in deprecated_ids
    ]
    
    # Update storage (don't remove deprecated)
    # ... add/update logic ...
    
    return {
        "status": "success",
        "provider": provider_id,
        "models_added": len(added),
        "models_updated": len(updated),
        "deprecated": deprecated
    }

2. Optional: Add deprecated flag to schema:

class Model(BaseModel):
    id: str
    display_name: str
    # ... other fields ...
    deprecated: bool = False
    deprecated_at: Optional[str] = None

Tasks

• Implement deprecated detection in fetch logic
• Add deprecated models to fetch response
• Optionally add deprecated flag to model schema
• Handle edge cases (provider never fetched before)
• Don't remove deprecated models automatically
• Write unit tests for detection logic
• Test with mocked API responses (simulate model removal)

Edge Cases

• First fetch: No stored models yet → no deprecated
• All models deprecated: Rare but possible (provider migration)
• Manual models: Should never be marked deprecated (only API-sourced)
• Re-added models: If deprecated model returns in future fetch, unmark

Acceptance Criteria

• Deprecated models correctly detected when missing from API
• Fetch response includes deprecated list
• Deprecated models NOT removed automatically
• Manual models never marked as deprecated
• Unit tests cover all edge cases
• Works for both OpenAI and OpenRouter providers

[kashifpk]

Partially implemented: Backend detects deprecated models when fetching from API (returns list in response). Frontend notification UI not yet implemented.