[BUG] Security Plugin: verify_hostnames: false setting in openid_connect_idp is ignored in OpenSearch 3.5.0 — regression from 2.19.3

[RamHaridas]

Describe the bug

When configuring OpenID Connect authentication in the security plugin's config.yml, the openid_connect_idp.verify_hostnames: false setting is not honored in OpenSearch 3.5.0. Hostname verification is still enforced by the Java TLS stack, causing SSLHandshakeException when the IdP certificate does not have a Subject Alternative Name (SAN) matching the IdP hostname.

This is a regression from OpenSearch 2.19.3 where the same configuration works correctly.

Error log from OpenSearch:

org.opensearch.security.auth.http.jwt.keybyoidc.AuthenticatorUnavailableException:
  Error while getting https://ckey-ckey.fpm-ckey.svc.cluster.local:8443/access/realms/fpm-realm/.well-known/openid-configuration:
  javax.net.ssl.SSLHandshakeException: (certificate_unknown)
  No subject alternative DNS name matching ckey-ckey.fpm-ckey.svc.cluster.local found.

    at o.o.s.a.h.j.keybyoidc.KeySetRetriever.getJwksUri(KeySetRetriever.java:245)
    at o.o.s.a.h.j.keybyoidc.KeySetRetriever.get(KeySetRetriever.java:108)
    at o.o.s.a.h.j.keybyoidc.SelfRefreshingKeySet$1.run(SelfRefreshingKeySet.java:213)

Caused by: javax.net.ssl.SSLHandshakeException: (certificate_unknown)
  No subject alternative DNS name matching ckey-ckey.fpm-ckey.svc.cluster.local found.
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(Unknown Source)
    at org.apache.hc.client5.http.ssl.AbstractClientTlsStrategy.executeHandshake(AbstractClientTlsStrategy.java:245)
    at org.apache.hc.client5.http.ssl.DefaultClientTlsStrategy.upgrade(DefaultClientTlsStrategy.java:48)
    at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.upgradeToTls(DefaultHttpClientConnectionOperator.java:263)
    at o.o.s.a.h.j.keybyoidc.KeySetRetriever.getJwksUri(KeySetRetriever.java:218)

Related component

Plugins

To Reproduce

1. Deploy a Keycloak (or any OIDC IdP) with a self-signed TLS certificate that has no SAN (only CN):

   Subject: CN=ckey.com
   Issuer:  CN=ckey.com
   

   IdP accessible at: ckey-ckey.fpm-ckey.svc.cluster.local:8443

2. Configure the security plugin config.yml with OpenID Connect and hostname verification disabled:

   authc:
     openid_auth_domain:
       http_enabled: true
       transport_enabled: false
       order: 1
       http_authenticator:
         type: openid
         challenge: false
         config:
           subject_key: preferred_username
           roles_key: roles
           openid_connect_url: "https://ckey-ckey.fpm-ckey.svc.cluster.local:8443/access/realms/fpm-realm/.well-known/openid-configuration"
           openid_connect_idp:
             enable_ssl: true
             verify_hostnames: false
             trust_all: false
             pemtrustedcas_filepath: "/etc/opensearch/config/certs/keycloakRootCaPem"
       authentication_backend:
         type: noop

3. Obtain a valid JWT token from the IdP and send a request to OpenSearch:

   curl -k https://opensearch:9200/_cat/indices?v \
     -H "Authorization: Bearer <valid_jwt_token>"

4. Observe HTTP 401 with body Authentication finally failed.

Expected behavior

OpenSearch should skip hostname verification when verify_hostnames: false is set, validate the JWT successfully, and return HTTP 200. This is the behavior in OpenSearch 2.19.3.

Additional Details

Plugins
opensearch-security

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: Rocky 8
• Version: 3.5.0

Additional context given by AI
OpenSearch 3.5.0 migrated from Apache HttpClient 4.x to 5.x. In HttpClient 5, hostname verification is performed by the JVM via SSLParameters.setEndpointIdentificationAlgorithm("HTTPS") inside DefaultClientTlsStrategy. The KeySetRetriever does not set this to null when verify_hostnames: false, so the JVM enforces hostname checking regardless of the plugin setting.

[pawelw1]

I can confirm that the verify_hostnames option in the OpenID configuration is always set to true. Security plugin ignores these settings.
According to my test, the latest working version was 2.19.4. The issue started in 3.0.0

This issue was also reported in OpenSearch forum.
https://forum.opensearch.org/t/opensearch-3-5-ignores-verify-hostnames-false-in-openid-config-and-fails-with-san-hostname-validation-error/27893/4

[cwperks]

Sounds similar to #5647