Prepare for the "next generation" clients

[sothawo]

Spring Data Elasticsearch needs to be changed in the near future to support new clients:

imperative code:

• Elasticsearch is working on a new client that will be the successor of the RestHighLevelClient as the RestHighLevelClient uses libraries from the Elasticsearch core that are not licensed under the Apache 2 license anymore.
• Opensearch is a fork based on Elasticsearch 7.10.2 and should be supported as well. We need a client that is based on the 7.10.2 libraries. Edit 03.09.20201: the client can use the now available OpenSearch fork of the Elasticsearch client libraries.
• it should be possible to use external client implementations that adhere to a to-be-defined API. This would for example allow Opensearch to implement their own implementation.

reactive code:

The reactive code uses code (classes) from the Elasticsearch libraries and code copied from the pre 7.11 codebase which was Apache licensed. The transport layer is implemented using Spring's WebClient.

• reactive implementation must be adapted to use the types provided by the new Elasticsearch client library, the existing copied code is not necessary anymore then
• the existing implementation is provided on the pre-7.11 level.

The currently supported clients, the RestHighLevelClient and the TransportClient should not be used anymore when the new Elasticsearch client is available due to potential licensing problems.

The purpose of this ticket is to prepare the internal package structure and architectural changes to enable use of the new clients as well as to document the needed changes

[janhoy]

Is there any movement on this last 6 months? Are we closer to a goal of having opensearch as one of many clients supported? Or are we perhaps farther away, and more likely to see someone else start a spring-data-opensearch integration?

[sothawo]

Currently the focus is on adding the new Elasticsearch client to Spring Data Elasticsearch as an alternative to the existing RHLC as this has been deprecated in Elasticsearch 7.16.

Restructuring the code in order to have a clean structure in which to add additional client implementations has to wait as this means breaking changes in package structures, which we don't do on minor version updates.

So the next version 4.4 will have the new Elasticsearch client available as an alternative for users to try it out, breaking changes can be introduced with version 5.0 which is planned in alignment with Spring Data 3, Spring Boot 3 and Spring 6.

[sothawo]

this is finished with #2196