Skip to content

sdk: centralize programmatic settings schema#2361

Draft
neubig wants to merge 7 commits intomainfrom
openhands/issue-2228-sdk-settings-schema
Draft

sdk: centralize programmatic settings schema#2361
neubig wants to merge 7 commits intomainfrom
openhands/issue-2228-sdk-settings-schema

Conversation

@neubig
Copy link
Contributor

@neubig neubig commented Mar 8, 2026

Summary

  • centralize a structured settings schema in the SDK via AgentSettings and export_settings_schema() for downstream clients
  • expose the full current LLM settings surface in the exported schema instead of a curated five-field subset
  • keep SDK-owned metadata focused on neutral config semantics like sections, types, defaults, required-ness, secrets, choices, dependencies, and prominence
  • remove the lossy from_agent() / apply_to_agent() / to_agent() round-trip path so this PR stays focused on schema export rather than materializing a default-only SDK config API

Checklist

  • If the PR is changing/adding functionality, are there tests to reflect this?
  • If there is an example, have you run the example to make sure that it works?
  • If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
  • If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
  • Is the github CI passing?

Testing

  • uv run pre-commit run --files AGENTS.md openhands-sdk/openhands/sdk/__init__.py openhands-sdk/openhands/sdk/llm/llm.py openhands-sdk/openhands/sdk/settings.py openhands-sdk/openhands/sdk/settings_metadata.py tests/sdk/test_settings.py
  • uv run pytest tests/sdk/test_settings.py -q

Fixes

Fixes #2228


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:ae37726-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-ae37726-python \
  ghcr.io/openhands/agent-server:ae37726-python

All tags pushed for this build

ghcr.io/openhands/agent-server:ae37726-golang-amd64
ghcr.io/openhands/agent-server:ae37726-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:ae37726-golang-arm64
ghcr.io/openhands/agent-server:ae37726-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:ae37726-java-amd64
ghcr.io/openhands/agent-server:ae37726-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:ae37726-java-arm64
ghcr.io/openhands/agent-server:ae37726-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:ae37726-python-amd64
ghcr.io/openhands/agent-server:ae37726-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:ae37726-python-arm64
ghcr.io/openhands/agent-server:ae37726-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:ae37726-golang
ghcr.io/openhands/agent-server:ae37726-java
ghcr.io/openhands/agent-server:ae37726-python

About Multi-Architecture Support

  • Each variant tag (e.g., ae37726-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., ae37726-python-amd64) are also available if needed

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 8, 2026

API breakage checks (Griffe)

Result: Passed

Action log

@github-actions
Copy link
Contributor

github-actions bot commented Mar 8, 2026

Agent server REST API breakage checks (OpenAPI)

Result: Failed

Log excerpt (first 1000 characters)
{"asctime": "2026-03-12 14:55:30,168", "levelname": "WARNING", "name": "openhands.agent_server.config", "filename": "config.py", "lineno": 173, "message": "\u26a0\ufe0f OH_SECRET_KEY was not defined. Secrets will not be persisted between restarts."}
::error title=openhands-agent-server REST API::Breaking REST API change detected without MINOR version bump (1.13.0 -> 1.13.0).

Breaking REST API changes detected compared to baseline release:
- the 'file' request property type/format changed from 'string'/'' to 'string'/'binary'
/home/runner/work/software-agent-sdk/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/async_client_cleanup.py:66: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()

Action log

@github-actions
Copy link
Contributor

github-actions bot commented Mar 8, 2026

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-sdk/openhands/sdk
   __init__.py23291%90–91
   settings.py1961890%195, 199, 210, 219, 223, 274, 279, 358, 368, 370, 375, 393, 406, 408, 410, 412, 416, 419
openhands-sdk/openhands/sdk/llm
   llm.py4807783%440, 493, 714, 820, 822–823, 851, 897, 908–910, 914–918, 926–928, 938–940, 943–944, 948, 950–951, 953, 1151–1152, 1349–1350, 1359, 1372, 1374–1379, 1381–1398, 1401–1405, 1407–1408, 1414–1423, 1474, 1476
TOTAL20148582171% 

Copy link
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taste Rating: 🟢 Good taste

Analysis:

This PR solves a real problem - centralizing settings schema as a single source of truth for downstream clients. The data structure choice (Pydantic model with field metadata) is pragmatic and follows established patterns.

Data Structure: Clean. Settings live in one place, metadata is co-located with field definitions, and the schema export is straightforward.

Simplicity: The conversion logic handles real use cases (create vs update, custom factories) without over-engineering. Test coverage validates real behavior - agent round-tripping works, factories integrate correctly, schema export produces the right structure.

Backward Compatibility: Purely additive - no existing APIs changed.

Verdict: ✅ Worth merging

Key Insight: By making SDKSettings the declarative source of truth with exportable schema, this eliminates the need for clients to reverse-engineer agent configuration or maintain duplicate settings definitions.

Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig marked this pull request as draft March 9, 2026 03:07
Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig marked this pull request as ready for review March 9, 2026 18:04
Copy link
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taste Rating: 🟡 Acceptable - Solid engineering approach solving a real problem.

Data Structure: Clean. Settings metadata lives alongside field definitions via Pydantic annotations, avoiding the anti-pattern of duplicate config models that drift from the source of truth.

Verdict: ✅ Worth merging. The core design is sound - annotating canonical fields eliminates special cases and maintains a single source of truth. Minor documentation fix suggested below.

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi — OpenHands-GPT-5.4 here. I read the PR, the review threads, the related issues (#2228, #2274, #1451), and I also checked the current usage patterns in OpenHands/OpenHands-CLI before leaving this review.

I agree with the short-term motivation: programmatic settings should come from the SDK instead of being hand-curated separately in each client. I also think replacing the duplicate mini-LLM settings subset with the canonical LLM fields was the right local correction.

That said, I don't think this PR lands on the right boundary for the SDK yet.

  1. The new AgentSettings round-trip is lossy. from_agent() / apply_to_agent() read like a general SDK config/edit/rebuild path, but the implementation only preserves a curated default story (LLMSummarizingCondenser, APIBasedCritic, and an annotated subset of LLM fields). In an extensible SDK, an official settings/materialization path should not silently drop alternative built-in or downstream implementations.

  2. The exported schema mixes semantic config with client presentation policy. widget, advanced, placeholder, slash_command, and similar hints are first-party UX decisions, not core SDK semantics. Exporting them from SDK models hardcodes one client opinion into every client and makes future divergence painful.

  3. More broadly, #2274 is right that LLM already carries too many responsibilities. This PR adds more meaning onto the current LLM class instead of separating configuration from runtime/materialization.

My preference would be either:

  • narrow this into an explicitly first-party client adapter (instead of a generic SDK settings abstraction), or
  • redesign around a canonical immutable config/profile object with neutral schema export, and keep presentation decisions in CLI/GUI.

So: useful motivation, but I think the current abstraction is risky for an extensible SDK.

@neubig neubig marked this pull request as draft March 12, 2026 14:43
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prepare settings for ingestion to programmatic settings menu in CLI and GUI

4 participants