common : rework gpt-oss parser

[aldehir]

Rework the gpt-oss parser.

• Tighten up the grammar, gpt-oss is very good at following its own Harmony spec.
• Allow any sequence of analysis/preamble.
• Clean up the trigger rules, gpt-oss may sometimes invoke a builtin function if not constrained when emitting the tool namespace.
• Include fix from common : fix gpt-oss Jinja error with content and thinking on tool-call messages #19704.
• Fix response_format not being enforced.
• Remove parallel call logic, gpt-oss is trained to only emit single tool calls.
• Removed support for reasoning-format = none. It makes no sense for gpt-oss. Users can choose to ignore reasoning_content.
• Remove invalid test cases.

fixes #20344

[pwilkin]

Final is a keyword, does this work correctly? I'd change it anyway just in case.

[aldehir]

It does, which is why I kept it. But I'll change it. Most syntax highlighters are not semantic aware so it'll highlight it as a keyword.

[pwilkin]

I'll trust you on this one :)

[pwilkin]

Just two issues:
-> support for reasoning_format: none should stay, since some people want just raw template output to further process with their own custom parsers / apps (I think SillyTavern does stuff like that) and not outputting the reasoning in content breaks this
-> since you're saying this fixes structured output, better add a test for structured output :)

[aldehir]

I'll add the logic back in, but it truly makes no sense. For one, the official template will throw an exception if it sees any harmony tags in the message. Ultimately, this model is incapable of not reasoning. Even when constrained, it will leak reasoning traces inside the final response if it can.

That said, if clients depend on this then I guess 🤷‍♂️

[pwilkin]

@aldehir reasoning_format: none is not enable_thinking: false, it just means you don't process the reasoning tags and you spill them all in the content.

[aldehir]

@aldehir reasoning_format: none is not enable_thinking: false, it just means you don't process the reasoning tags and you spill them all in the content.

I'm fully aware of the difference. See point 2. The responsibility is pushed to the client to strip the tags. Currently there is a hack to remove the exception, but then the model will in-context learn and start to emit bad harmony output, thus breaking parsing.

I've been down this road when I implemented the original parsing.

Edit: I can see my phrasing conflated the two. Ignore the second part.

[pwilkin]

@aldehir I know this is generally non-feasible, but there exists a small but vocal group of people who use their own parsing tools for whatever reasons and they really like to get the raw unprocessed contents :)

But I just realized you can not change it and instead approve my #20289 to satisfy them :)

[aldehir]

But I just realized you can not change it and instead approve my #20289 to satisfy them :)

Yes, I rather just give them the whole harmony output. I've seen complaints about "missing tokens" too, it's never ending!