ldaptive default ThreadLocalTLSSocketFactory mixes hostnames with multiple server configs

Our setup has 4 different subject sources. Three of these correspond to a single LDAP source in grouper-loader, and one corresponds to an AD source. Both servers use ldaps:// connections. Occasionally, a loader job results in a org.ldaptive.provider.ConnectionException error with message:

javax.naming.CommunicationException: devaddc0.addev.unc.edu:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Hostname '[ldap-dev.isis.unc.edu]' does not match the hostname in the server's certificate 'CN=devaddc0.addev.unc.edu, OU=Information Technology Services, O=University of North Carolina at Chapel Hill, STREET=153A Country Club Road, L=Chapel HIll, ST=NC, OID.2.5.4.17=27514, C=US']

Server "ldap-dev.isis.unc.edu" refers to the LDAP server's loader config (uncLdap), while "devaddc0.addev.unc.edu" is the AD server's config (uncAD). The query that failed was using the uncAD source. Getting an error that the SSL handshake expected the other server's hostname is not easily explained.

This can be reproduced in a sample database with a java debugger. The sample job I use is a SQL_GROUP_LIST job with a readers column.

1) Break on edu/internet2/middleware/grouper/app/loader/GrouperLoader.java:1572:

if (isSqlLoader)

{ GrouperLoaderJob.runJob(hib3GrouperLoaderLog, group, grouperSession); }

2) With gsh hooked up to a Java debugger, run GrouperLoader.runJobOnceForGroup(session, group). The breakpoint will hit, but let it continue

3) Run the job again. When the breakpoint hits, close the open ldaps network connections for both hosts. This simulates the pooled ldap connections getting reset after being inactive.

4) Continue the run. The same error and stacktrace are sent to the log file.

What I have traced so far is that when the ldaptive SslSocketFactory is ThreadLocalTLSSocketFactory (and this is the default factory), it initializes with a hostname validator based on the host in the url when it was first used. As long as the same thread with the same url, this works fine. But when the thread gets reused with a different url, it retains the same SSL validator with the old host. This explains why the error mentions expecting the old host name, when there is nothing in the socket connection itself referring to that name.

If I manually set the socket factory to TLSSocketFactory instead, this error seems to go away. But I could not find any documentation on either TLSSocketFactory or ThreadLocalTLSSocketFactory anywhere, so I don't know if this is the best approach.

My local fix:

grouper-laoder.properties:
ldap.UncLdap.sslSocketFactory = org.ldaptive.ssl.TLSSocketFactory
ldap.UncAD.sslSocketFactory = org.ldaptive.ssl.TLSSocketFactory

grouper/src/grouper/edu/internet2/middleware/grouper/ldap/ldaptive/LdaptiveSessionImpl.java:

{{@@ -176,8 +176,20 @@ public class LdaptiveSessionImpl implements LdapSession

{ LOG.info("Setting gssApiConfig"); binder.setBindSaslConfig(gssApiConfig); }

-

• // handle ssl socket factory
  +
  + // If there is >1 ldap connect url, threadlocal tls won't work reliable
  + String sslSocketFactory = GrouperLoaderConfig.retrieveConfig().propertyValueString("ldap." + ldapServerId + ".sslSocketFactory");
  + if (sslSocketFactory != null)
  Unknown macro: {+ try { + Class<SSLSocketFactory> theClass = GrouperUtil.forName(sslSocketFactory); + ((JndiProviderConfig) connectionFactory.getProvider().getProviderConfig()).setSslSocketFactory(theClass.newInstance()); + } catch (Exception e) { + LOG.error("Failed to set ldap." + ldapServerId + ".sslSocketFactory to " + sslSocketFactory, e); + + }+ }

  +
  + // handle ssl socket factory (also overrides sslSocketFactory property)
  String cafile = GrouperLoaderConfig.retrieveConfig().propertyValueString("ldap." + ldapServerId + ".pemCaFile");
  String certfile = GrouperLoaderConfig.retrieveConfig().propertyValueString("ldap." + ldapServerId + ".pemCertFile");
  String keyfile = GrouperLoaderConfig.retrieveConfig().propertyValueString("ldap." + ldapServerId + ".pemKeyFile");
  }}