[LEADS-25] Support new MCP header

config/system.yaml

@@ -41,11 +41,21 @@ api:
   # API input configuration
   provider: "openai"                   # LLM provider for queries
   model: "gpt-4o-mini"                 # Model to use for queries
-  no_tools: null                       # Whether to bypass tools and MCP servers (optional)
+  no_tools: false                       # Whether to bypass tools and MCP servers (optional)
   system_prompt: null                  # System prompt (default None)
 
   cache_dir: ".caches/api_cache"  # Directory with lightspeed-stack cache
   cache_enabled: true                  # Is lightspeed-stack cache enabled?
+
+  # MCP Server Authentication Configuration
+  mcp_headers:
+    enabled: true                      # Enable MCP headers functionality
+    servers:                          # MCP server configurations
+      filesystem-tools:
+        auth_type: bearer              # Authentication type: only bearer is supported
+        env_var: API_KEY              # Environment variable containing the token/key
+
+  # Legacy authentication (fallback when mcp_headers.enabled is false)
   # Authentication via API_KEY environment variable only for MCP server
 
   # Retry configuration for 429 Too Many Requests API errors

docs/configuration.md

@@ -107,8 +107,6 @@ embedding:
 This section configures the inference API for generating the responses. It can be any Lightspeed-Core compatible API.
 Note that it can be easily integrated with other APIs with a minimal change.
 
-Authentication via `API_KEY` environment variable only for MCP server.
-
 | Setting (api.) | Default | Description |
 |----------------|---------|-------------|
 | enabled | `"true"` |  Enable/disable API calls |
@@ -121,6 +119,30 @@ Authentication via `API_KEY` environment variable only for MCP server.
 | system_prompt | `null` | Custom system prompt (optional) |
 | cache_dir | `".caches/api_cache"` | Directory with cached API responses |
 | cache_enabled | `true` | Is API cache enabled? |
+| mcp_headers | `null` | MCP headers configuration for authentication (see below) |
+| num_retries | `3` | Maximum number of retry attempts for API calls on 429 errors |
+
+### MCP Server Authentication
+
+The framework supports two methods for MCP server authentication:
+
+#### 1. New MCP Headers Configuration (Recommended)
+The `mcp_headers` configuration provides a flexible way to configure authentication for individual MCP servers:
+
+| Setting (api.mcp_headers.) | Default | Description |
+|----------------------------|---------|-------------|
+| enabled | `true` | Enable/disable MCP headers functionality |
+| servers | `{}` | Dictionary of MCP server configurations |
+
+For each server in `servers`, you can configure:
+
+| Setting (api.mcp_headers.servers.<server_name>.) | Default | Description |
+|---------------------------------------------------|---------|-------------|
+| auth_type | `"bearer"` | Authentication type (currently only "bearer" is supported) |
+| env_var | required | Environment variable containing the authentication token |
+
+#### 2. Legacy Authentication (Fallback)
+When `mcp_headers.enabled` is `false`, the system falls back to using the `API_KEY` environment variable for all MCP server authentication.
 
 ### API Modes
 
@@ -137,6 +159,8 @@ Authentication via `API_KEY` environment variable only for MCP server.
 - **Reproducible results**: Same response data used across runs
 ### Example
 
+#### Example Configuration
+
 ```yaml
 api:
   enabled: true
@@ -150,8 +174,37 @@ api:
   system_prompt: null
   cache_dir: ".caches/api_cache"
   cache_enabled: true
+  num_retries: 3
+
+  # MCP Server Authentication Configuration
+  mcp_headers:
+    enabled: true                      # Enable MCP headers functionality
+    servers:                          # MCP server configurations
+      filesystem-tools:
+        auth_type: bearer              # Authentication type: only bearer is supported
+        env_var: API_KEY              # Environment variable containing the token/key
+      another-mcp-server:
+        auth_type: bearer
+        env_var: ANOTHER_API_KEY      # Use a different environment variable
+```
+
+#### Lightspeed Stack API Compatibility
+
+**Important Note for lightspeed-stack API users**: To use the MCP headers functionality with the lightspeed-stack API, you need to modify the `llama_stack_api/openai_responses.py` file in your lightspeed-stack installation:
+
+In the `OpenAIResponsesToolMCP` class, change the `authorization` parameter's `exclude` field from `True` to `False`:
+
+```python
+# In llama_stack_api/openai_responses.py
+class OpenAIResponsesToolMCP:
+    authorization: Optional[str] = Field(
+        default=None,
+        exclude=False  # Change this from True to False
+    )
 ```
 
+This change allows the authorization headers to be properly passed through to MCP servers.
+
 ## Metrics
 Metrics are enabled globally (as described below) or within the input data for each individual conversation or individual turn (question/answer pair). To enable a metrics globally you need to set `default` meta data attribute to `true`
 

src/lightspeed_evaluation/core/api/client.py

@@ -91,14 +91,62 @@ def _setup_client(self) -> None:
             )
             self.client.headers.update({"Content-Type": "application/json"})
 
-            # Use API_KEY environment variable for authentication
-            api_key = os.getenv("API_KEY")
-            if api_key and self.client:
-                self.client.headers.update({"Authorization": f"Bearer {api_key}"})
+            # Set up MCP headers based on configuration
+            mcp_headers_dict = self._build_mcp_headers()
+            if mcp_headers_dict:
+                self.client.headers.update(
+                    {"MCP-HEADERS": json.dumps(mcp_headers_dict)}
+                )
 
         except Exception as e:
             raise APIError(f"Failed to setup API client: {e}") from e
 
+    def _build_mcp_headers(self) -> dict[str, dict[str, str]]:
+        """Build MCP headers based on configuration.
+
+        Returns:
+            Dictionary of MCP server headers, or empty dict if none configured.
+        """
+        # Use new MCP headers configuration if available and enabled
+        if (
+            self.config.mcp_headers
+            and self.config.mcp_headers.enabled
+            and self.config.mcp_headers.servers
+        ):
+
+            mcp_headers = {}
+            for server_name, server_config in self.config.mcp_headers.servers.items():
+                # Get token from environment variable
+                token = os.getenv(server_config.env_var)
+                if not token:
+                    logger.warning(
+                        "Environment variable '%s' not found "
+                        "for MCP server '%s'. Skipping authentication.",
+                        server_config.env_var,
+                        server_name,
+                    )
+                    continue
+
+                # Set up bearer auth headers
+                header_name = server_config.header_name or "Authorization"
+                header_value = f"Bearer {token}"
+
+                mcp_headers[server_name] = {header_name: header_value}
+
+            if not mcp_headers:
+                raise APIError(
+                    "MCP headers are enabled, but no valid server credentials were resolved. "
+                    "Check mcp_headers.servers and required environment variables."
+                )
+            return mcp_headers
+
+        # Fallback to legacy API_KEY behavior for backward compatibility
+        api_key = os.getenv("API_KEY")
+        if api_key:
+            return {"filesystem-tools": {"Authorization": f"Bearer {api_key}"}}
+
+        return {}
+
     def query(
         self,
         query: str,

src/lightspeed_evaluation/core/models/system.py

@@ -160,6 +160,49 @@ def _validate_provider(cls, v: str) -> str:
         return v
 
 
+class MCPServerConfig(BaseModel):
+    """Configuration for a single MCP server authentication."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    auth_type: str = Field(
+        default="bearer",
+        description="Authentication type: only bearer is supported",
+    )
+    env_var: str = Field(
+        ...,
+        min_length=1,
+        description="Environment variable containing the token/key",
+    )
+    header_name: Optional[str] = Field(
+        default=None,
+        description="Custom header name (optional, defaults based on auth_type)",
+    )
+
+    @field_validator("auth_type")
+    @classmethod
+    def validate_auth_type(cls, v: str) -> str:
+        """Validate auth_type is supported."""
+        if v != "bearer":
+            raise ValueError("auth_type must be 'bearer'")
+        return v
+
+
+class MCPHeadersConfig(BaseModel):
+    """Configuration for MCP headers functionality."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    enabled: bool = Field(
+        default=True,
+        description="Enable MCP headers functionality",
+    )
+    servers: dict[str, MCPServerConfig] = Field(
+        default_factory=dict,
+        description="MCP server configurations",
+    )
+
+
 class APIConfig(BaseModel):
     """API configuration for dynamic data generation."""
 
@@ -196,6 +239,9 @@ class APIConfig(BaseModel):
     cache_enabled: bool = Field(
         default=True, description="Is caching of lightspeed-stack queries enabled?"
     )
+    mcp_headers: Optional[MCPHeadersConfig] = Field(
+        default=None, description="MCP headers configuration for authentication"
+    )
     num_retries: int = Field(
         default=DEFAULT_API_NUM_RETRIES,
         ge=0,