Introduce OpenSearch integration

[reta]

• You have read the Spring Data contribution guidelines.
• There is a ticket in the bug tracker for the project in our [issue tracker](https://github.
  com/spring-projects/spring-data-elasticsearch/issues). Add the issue number to the Closes #issue-number line below
• You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
• You submit test cases (unit or integration tests) that back your changes.
• You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

Since https://github.com/spring-projects/spring-data-elasticsearch/pull/2196/files got merged, we have an opportunity to introduce clean OpenSearch integration (based on HighLevelRestClient at least) along with Elasticsearch one. This pull request draft aims to kick of the discussion if such a path forward makes sense for maintainers or not. Upon positive feedback, the remaining issues (test cases, documentation, ...) are going to be added right away.

@sothawo what do you think about it? CC @brijos @dblock

Closes #1918

[sothawo]

That's the way it should go I think. Two big things to do:

Configuration

The current code for Elasticsearch uses the same configuration class org.springframework.data.elasticsearch.client.ClientConfiguration for both the old code using RHLC (and the custom reactive code) and the new code using the Elasticsearch client (imperative and reactive); so this class is not in the elc or erhlc packages. This configuration class uses org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback that comes from the low level Elasticsearch client library, which is and always has been Apache 2 licensed. Does Opensearch use this same low level client or does it use an opensearch-renamed-copy of this as well? If the latter, then the opensearch integration needs a copy of the configuration class as well somewhere.

Integration tests

I already prepared last year the integration of Opensearch into the integration test setup insofar as there is a property in pom.xml (mvn.integration-test-opensearch.goal) which should be set to integration-test to add OS integration tests. In addition to this property,

• the integration tests using Elasticsearch (*ELCIntegrationTests and *ERHLCIntegrationTests) should get an additional annotation @EnabeldIfSystemProperty(named="sde.integration-test.environment" matches="elasticsearch"))
• every test should get a new copy *ORHLCIntegrationTests which uses the new Opensearch configuration and is annotated with @EnabeldIfSystemProperty(named="sde.integration-test.environment" matches="opensearch")

And src/test/resources/testcontainers-opensearch.properties needs to be set up to define and configure the correct opensearch docker image to be used by TestContainers.

I did not now try this out now, just writing what I assume should be working.

[reta]

@sothawo thanks a lot for the feedback and insights.

This configuration class uses org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback that comes from the low level Elasticsearch client library, which is and always has been Apache 2 licensed. Does Opensearch use this same low level client or does it use an opensearch-renamed-copy of this as well?

Yes, it uses the same underlying HttpAsyncClientBuilder but the callback is repackaged under org.opensearch.client.RestClientBuilder.HttpClientConfigCallback, I will take care of that, thank you.

I already prepared last year the integration of Opensearch into the integration test setup insofar as there is a property in pom.xml (mvn.integration-test-opensearch.goal) which should be set to integration-test to add OS integration tests.

Yes, saw that as well, will continue the work you have started and make the changes to ensure both test suites are working, thank you.

[reta]

Making it public to be used by org.springframework.data.elasticsearch.client.orhlc.DefaultClientConfiguration

[reta]

@sothawo I believe Opesearch's RestHighLevelClient support is complete (beside the documentation, working on it now), may I ask you to take a look please? Thank you.

[sothawo]

Hi, I will have a look these days.
For your info: I will be in a call probably next week with a Product Manager for OpenSearch from Amazon and with the Spring Data project lead from VMware, talking about where to host and maintain the OS integration and how it might be handled in the future. So as I said, I will a a thorough look at your work, but will wait with a merge until after that call.

[reta]

So as I said, I will a a thorough look at your work, but will wait with a merge until after that call.

Sure, thanks a lot!

[ralberts]

Excited to see this progressing!

[sothawo]

looks alright for me as a start on the new repository.

I just changed the InetSocketAddressParser visiblity in main (#2227) so that when you rebase on main all changes from your side would then be in the new orhlc package or in the test setup.

[Mr-Smith-13]

I was wondering if this PR is merged soon because we would highly appreciate to be able to use the OpenSearch client to connect to our OpenSearch cluster.
Is there already a planned date / is it clear in which version this change wuill be included?

[sothawo]

This PR won't be merged. Please read the comment on the issue in #1918 (comment) and opensearch-project/opensearch-clients#28.

After the repository is created by Opensearch the code from this PR will be migrated over there

[reta]

Closing in favor of opensearch-project/spring-data-opensearch#1 as per discussions with Spring Data team