Get MissingRequiredPropertyException on GetTokenRequest

[mhlnk]

Java API client version

7.17.4

Java version

jdk1.8.0_333

Elasticsearch Version

7.17.4

Problem description

I am running Elasticsearch on a Red Hat Linux with Platinum license. A custom application is connecting to Elasticsearch using native realm for authentication. The username and password are passed for authentication. Upon successful authentication an access token is passed back to the client. The client uses that token with every request to make sure the request came from authenticated user. This process works using Elasticsearch Java API 7.11.2, RestHighLevelClient. Upon upgrading to 7.17.4 ElasticsearchClient, a MissingRequiredPropertyException is thrown: Missing required property GetTokenResponse.refreshToken , when generating a token request. If I surround the GetTokenRequest/GetTokenResponse block of code with ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true);, the token is generated, but then Too many open files exception is thrown.

Below are the snippets of the code to reproduce the error:
Authentication

public String authenticate(String userName, String password) throws AuthenticationException, IOException {
		
  String token = null;
  ElasticsearchClientFactory esClientFactory = new ElasticsearchClientFactory(esProperties, profiles);
  try {
    ElasticsearchClient esClient = esClientFactory.createEsClient(userName, password);

    AuthenticateResponse authResponse = esClient.security().authenticate();

    GetTokenRequest tokenRequest = new GetTokenRequest.Builder()
					.grantType(AccessTokenGrantType.ClientCredentials)
					.build();
    GetTokenResponse tokenResponse = esClient.security().getToken(tokenRequest);

    token = tokenResponse.accessToken();

  } catch (CertificateException | NoSuchAlgorithmException | 
              UnrecoverableKeyException | KeyStoreException | KeyManagementException e) {
	e.printStackTrace();
	throw new AuthenticationException("Unauthorized user!");
  } catch (IOException e) {
	e.printStackTrace();
	throw new IOException("Unable to Login!");
  } finally {
	// Close Elasticsearch client
	esClientFactory.destroy();
  }
  return token;
}

Creating ES Client

public ElasticsearchClient createEsClient(String userName, String password)
			throws CertificateException, UnrecoverableKeyException, NoSuchAlgorithmException,
				   KeyStoreException, KeyManagementException, IOException {
  final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
  credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));

  final SSLContext sslContext = buildSSLContext();

  return buildEsClient(null, sslContext, "https", credentialsProvider);
}

Building the Client

private ElasticsearchClient buildEsClient(Header[] header,
	                                          SSLContext sslContext,
	                                          String scheme,
	                                          final CredentialsProvider credProvider) {
  RestClientBuilder httpClientBuilder = RestClient.builder(new HttpHost(hostId, portId, scheme));

  if (header != null && header.length > 0) {
    httpClientBuilder.setDefaultHeaders(header);
  }

  httpClientBuilder
	.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
		@Override
		public HttpAsyncClientBuilder customizeHttpClient (HttpAsyncClientBuilder httpClientBuilder) {
			if (credProvider != null) {
				httpClientBuilder.setDefaultCredentialsProvider(credProvider);
			}

			if (sslContext != null) {
				httpClientBuilder
					.setSSLContext(sslContext)
					.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
			}
			return httpClientBuilder;
		}
	});

       RestClient restClient = httpClientBuilder.build();

	// Create the transport with a Jackson mapper
	ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper(objMapper));

	// Create the API client
	return new ElasticsearchClient(transport);
}

[swallez]

Thanks for the very detailed report. Indeed the refresh token field isn't always present. This is an issue in the API specification used to produce the Java code. Once the API specification is fixed, we'll update the Java client code to resolve this issue.

Regarding the Too many open files, this is is most certainly unrelated to disabling required properties check. A more likely cause can be that a new Elasticsearch client is created for every request and incorrectly closed (you have to call client._transport().close()). What is the content of esClientFactory.destroy()?

[mhlnk]

Thanks for looking into the issue! Regarding the “Too many open files”, I was indeed creating a new Elasticsearch client on every request. As such, connections were created faster than they could be cleaned up, resulting in too many file descriptors being open. I have since added a manager to manage client connections to Elasticsearch, such that one client connection is created per user. If the session is inactive for a period of time or the user logs out, that connection is closed. This fixed the problem. Thanks again! From: Sylvain Wallez ***@***.***> Sent: Tuesday, August 23, 2022 12:41 PM To: elastic/elasticsearch-java ***@***.***> Cc: Margarita Hiett ***@***.***>; Author ***@***.***> Subject: Re: [elastic/elasticsearch-java] Get MissingRequiredPropertyException on GetTokenRequest (Issue #332) Thanks for the very detailed report. Indeed the refresh token field isn't always present. This is an issue in the API specification<https://github.com/elastic/elasticsearch-specification> used to produce the Java code. Once the API specification is fixed, we'll update the Java client code to resolve this issue. Regarding the Too many open files, this is is most certainly unrelated to disabling required properties check. A more likely cause can be that a new Elasticsearch client is created for every request and incorrectly closed (you have to call client._transport().close()). What is the content of esClientFactory.destroy()? — Reply to this email directly, view it on GitHub<#332 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARJGA3MMPRV2T5Y3IWR2TWDV2T5JPANCNFSM5ZUQBHUQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>