Remove org.elasticsearch dependencies from API classes.

Original Pull Request #1913
Closes #1884
Closes #1885

Author: sothawo

Diff for: src/main/asciidoc/reference/elasticsearch-clients.adoc

@@ -3,13 +3,14 @@
 
 This chapter illustrates configuration and usage of supported Elasticsearch client implementations.
 
-Spring Data Elasticsearch operates upon an Elasticsearch client that is connected to a single Elasticsearch node or a cluster. Although the Elasticsearch Client can be used to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of <<elasticsearch.operations>> and <<elasticsearch.repositories>>.
+Spring Data Elasticsearch operates upon an Elasticsearch client that is connected to a single Elasticsearch node or a cluster.
+Although the Elasticsearch Client can be used to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of <<elasticsearch.operations>> and <<elasticsearch.repositories>>.
 
 [[elasticsearch.clients.transport]]
 == Transport Client
 
-WARNING: The `TransportClient` is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. (https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[see the Elasticsearch documentation]). Spring Data Elasticsearch will support the `TransportClient` as long as it is available in the used
-Elasticsearch <<elasticsearch.versions,version>> but has deprecated the classes using it since version 4.0.
+WARNING: The `TransportClient` is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. (https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html[see the Elasticsearch documentation]).
+Spring Data Elasticsearch will support the `TransportClient` as long as it is available in the used Elasticsearch <<elasticsearch.versions,version>> but has deprecated the classes using it since version 4.0.
 
 We strongly recommend to use the <<elasticsearch.clients.rest>> instead of the `TransportClient`.
 
@@ -46,6 +47,7 @@ IndexRequest request = new IndexRequest("spring-data")
 
 IndexResponse response = client.index(request);
 ----
+
 <.> The `TransportClient` must be configured with the cluster name.
 <.> The host and port to connect the client to.
 <.> the RefreshPolicy must be set in the `ElasticsearchTemplate` (override `refreshPolicy()` to not use the default)
@@ -54,8 +56,7 @@ IndexResponse response = client.index(request);
 [[elasticsearch.clients.rest]]
 == High Level REST Client
 
-The Java High Level REST Client is the default client of Elasticsearch, it provides a straight forward replacement for the `TransportClient` as it accepts and returns
-the very same request/response objects and therefore depends on the Elasticsearch core project.
+The Java High Level REST Client is the default client of Elasticsearch, it provides a straight forward replacement for the `TransportClient` as it accepts and returns the very same request/response objects and therefore depends on the Elasticsearch core project.
 Asynchronous calls are operated upon a client managed thread pool and require a callback to be notified when the request is done.
 
 .High Level REST Client
@@ -93,6 +94,7 @@ IndexRequest request = new IndexRequest("spring-data")
 
 IndexResponse response = highLevelClient.index(request,RequestOptions.DEFAULT);
 ----
+
 <1> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL.
 <2> Create the RestHighLevelClient.
 <3> It is also possible to obtain the `lowLevelRest()` client.
@@ -131,6 +133,7 @@ Mono<IndexResponse> response = client.index(request ->
     .source(singletonMap("feature", "reactive-client"));
 );
 ----
+
 <.> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL.
 ====
 
@@ -162,39 +165,44 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder()
     headers.add("currentTime", LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));
     return headers;
   })
-  .withWebClientConfigurer(webClient -> {                               <.>
-    //...
-    return webClient;
-  })
-  .withHttpClientConfigurer(clientBuilder -> {                          <.>
-      //...
+  .withClientConfigurer(                                                <.>
+    (ReactiveRestClients.WebClientConfigurationCallback) webClient -> {
+  	  // ...
+      return webClient;
+  	})
+  .withClientConfigurer(                                                <.>
+    (RestClients.RestClientConfigurationCallback) clientBuilder -> {
+  	  // ...
       return clientBuilder;
-  })
+  	})
   . // ... other options
   .build();
 
 ----
+
 <.> Define default headers, if they need to be customized
 <.> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL.
 <.> Optionally enable SSL.
 <.> Optionally set a proxy.
 <.> Optionally set a path prefix, mostly used when different clusters a behind some reverse proxy.
-<.> Set the connection timeout. Default is 10 sec.
-<.> Set the socket timeout. Default is 5 sec.
+<.> Set the connection timeout.
+Default is 10 sec.
+<.> Set the socket timeout.
+Default is 5 sec.
 <.> Optionally set headers.
 <.> Add basic authentication.
 <.> A `Supplier<Header>` function can be specified which is called every time before a request is sent to Elasticsearch - here, as an example, the current time is written in a header.
 <.> for reactive setup a function configuring the `WebClient`
 <.> for non-reactive setup a function configuring the REST client
 ====
 
-IMPORTANT: Adding a Header supplier as shown in above example allows to inject headers that may change over the time, like authentication JWT tokens. If this is used in the reactive setup, the supplier function *must not* block!
+IMPORTANT: Adding a Header supplier as shown in above example allows to inject headers that may change over the time, like authentication JWT tokens.
+If this is used in the reactive setup, the supplier function *must not* block!
 
 [[elasticsearch.clients.logging]]
 == Client Logging
 
-To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level needs
-to be turned on as outlined in the snippet below.
+To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level needs to be turned on as outlined in the snippet below.
 
 .Enable transport layer logging
 [source,xml]

Diff for: src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc

@@ -3,12 +3,53 @@
 
 This section describes breaking changes from version 4.2.x to 4.3.x and how removed features can be replaced by new introduced features.
 
+[NOTE]
+====
+Elasticsearch is working on a new Client that will replace the `RestHighLevelClient` because the `RestHighLevelClient` uses code from Elasticsearch core libraries which are not Apache 2 licensed anymore.
+Spring Data Elasticsearch is preparing for this change as well.
+This means that internally the implementations for the `*Operations` interfaces need to change - which should be no problem if users program against the interfaces like `ElasticsearchOperations` or `ReactiveElasticsearchOperations`.
+If you are using the implementation classes like `ElasticsearchRestTemplate` directly, you will need to adapt to these changes.
+
+Spring Data Elasticsearch also removes or replaces the use of classes from the `org.elasticsearch` packages in it's API classes and methods, only using them in the implementation where the access to Elasticsearch is implemented.
+For the user that means, that some enum classes that were used are replaced by enums that live in `org.springframework.data.elasticsearch` with the same values, these are internally mapped onto the Elasticsearch ones.
+
+Places where classes are used that cannot easily be replaced, this usage is marked as deprecated, we are working on replacements.
+
+Check the sections on <<elasticsearch-migration-guide-4.2-4.3.deprecations>> and <<elasticsearch-migration-guide-4.2-4.3.breaking-changes>> for further details.
+====
+
 [[elasticsearch-migration-guide-4.2-4.3.deprecations]]
 == Deprecations
 
 [[elasticsearch-migration-guide-4.2-4.3.breaking-changes]]
 == Breaking Changes
 
+=== Removal of `org.elasticsearch` classes from the API.
+
+* In the `org.springframework.data.elasticsearch.annotations.CompletionContext` annotation the property `type()` has changed from `org.elasticsearch.search.suggest.completion.context.ContextMapping.Type` to `org.springframework.data.elasticsearch.annotations.CompletionContext.ContextMappingType`, the available enum values are the same.
+* In the `org.springframework.data.elasticsearch.annotations.Document` annotation the `versionType()` property has changed to `org.springframework.data.elasticsearch.annotations.Document.VersionType`, the available enum values are the same.
+* In the `org.springframework.data.elasticsearch.core.query.Query` interface the `searchType()` property has changed to `org.springframework.data.elasticsearch.core.query.Query.SearchType`, the available enum values are the same.
+* In the `org.springframework.data.elasticsearch.core.query.Query` interface the return value of  `timeout()` was changed to `java.time.Duration`.
+
 === Handling of field and sourceFilter properties of Query
 
-Up to version 4.2 the `fields` property of a `Query` was interpreted and added to the include list of the `sourceFilter`. This was not correct, as these are different things for Elasticsearch. This has been corrected. As a consequence code might not work anymore that relies on using `fields` to specify which fields should be returned from the document's `_source' and should be changed to use the `sourceFilter`.
+Up to version 4.2 the `fields` property of a `Query` was interpreted and added to the include list of the `sourceFilter`.
+This was not correct, as these are different things for Elasticsearch.
+This has been corrected.
+As a consequence code might not work anymore that relies on using `fields` to specify which fields should be returned from the document's `_source' and should be changed to use the `sourceFilter`.
+
+=== search_type default value
+
+The default value for the `search_type` in Elasticsearch is `query_then_fetch`.
+This now is also set as default value in the `Query` implementations, it was previously set to `dfs_query_then_fetch`.
+
+=== BulkOptions changes
+
+Some properties of the `org.springframework.data.elasticsearch.core.query.BulkOptions` class have changed their type:
+
+* the type of the `timeout` property has been changed to `java.time.Duration`.
+* the type of the`refreshPolicy` property has been changed to `org.springframework.data.elasticsearch.core.RefreshPolicy`.
+
+=== IndicesOptions change
+
+Spring Data Elasticsearch now uses `org.springframework.data.elasticsearch.core.query.IndicesOptions` instead of `org.elasticsearch.action.support.IndicesOptions`.

Diff for: src/main/java/org/springframework/data/elasticsearch/annotations/CompletionContext.java

@@ -1,3 +1,18 @@
+/*
+ * Copyright 2019-2021 the original author or authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
 package org.springframework.data.elasticsearch.annotations;
 
 import java.lang.annotation.Documented;
@@ -7,12 +22,11 @@
 import java.lang.annotation.RetentionPolicy;
 import java.lang.annotation.Target;
 
-import org.elasticsearch.search.suggest.completion.context.ContextMapping;
-
 /**
  * Based on reference doc - https://www.elastic.co/guide/en/elasticsearch/reference/current/suggester-context.html
  *
  * @author Robert Gruendler
+ * @author Peter-Josef Meisch
  */
 @Retention(RetentionPolicy.RUNTIME)
 @Target(ElementType.FIELD)
@@ -22,9 +36,16 @@
 
 	String name();
 
-	ContextMapping.Type type();
+	ContextMappingType type();
 
 	String precision() default "";
 
 	String path() default "";
+
+	/**
+	 * @since 4.3
+	 */
+	enum ContextMappingType {
+		CATEGORY, GEO
+	}
 }

Diff for: src/main/java/org/springframework/data/elasticsearch/annotations/Document.java

@@ -21,7 +21,6 @@
 import java.lang.annotation.RetentionPolicy;
 import java.lang.annotation.Target;
 
-import org.elasticsearch.index.VersionType;
 import org.springframework.data.annotation.Persistent;
 
 /**
@@ -116,9 +115,15 @@
 
 	/**
 	 * Controls how Elasticsearch dynamically adds fields to the document.
-	 * 
+	 *
 	 * @since 4.3
 	 */
 	Dynamic dynamic() default Dynamic.INHERIT;
 
+	/**
+	 * @since 4.3
+	 */
+	enum VersionType {
+		INTERNAL, EXTERNAL, EXTERNAL_GTE
+	}
 }

Diff for: src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java

@@ -27,6 +27,7 @@
 import javax.net.ssl.SSLContext;
 
 import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback;
+import org.springframework.data.elasticsearch.client.reactive.ReactiveRestClients;
 import org.springframework.http.HttpHeaders;
 import org.springframework.lang.Nullable;
 import org.springframework.web.reactive.function.client.WebClient;
@@ -120,16 +121,16 @@ static ClientConfiguration create(InetSocketAddress socketAddress) {
 	boolean useSsl();
 
 	/**
-	 * Returns the {@link SSLContext} to use. Can be {@link Optional#empty()} if unconfigured.
+	 * Returns the {@link SSLContext} to use. Can be {@link Optional#empty()} if not configured.
 	 *
-	 * @return the {@link SSLContext} to use. Can be {@link Optional#empty()} if unconfigured.
+	 * @return the {@link SSLContext} to use. Can be {@link Optional#empty()} if not configured.
 	 */
 	Optional<SSLContext> getSslContext();
 
 	/**
-	 * Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured.
+	 * Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured.
 	 *
-	 * @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured.
+	 * @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured.
 	 */
 	Optional<HostnameVerifier> getHostNameVerifier();
 
@@ -152,7 +153,7 @@ static ClientConfiguration create(InetSocketAddress socketAddress) {
 
 	/**
 	 * Returns the path prefix that should be prepended to HTTP(s) requests for Elasticsearch behind a proxy.
-	 * 
+	 *
 	 * @return the path prefix.
 	 * @since 4.0
 	 */
@@ -161,7 +162,7 @@ static ClientConfiguration create(InetSocketAddress socketAddress) {
 
 	/**
 	 * returns an optionally set proxy in the form host:port
-	 * 
+	 *
 	 * @return the optional proxy
 	 * @since 4.0
 	 */
@@ -173,11 +174,19 @@ static ClientConfiguration create(InetSocketAddress socketAddress) {
 	Function<WebClient, WebClient> getWebClientConfigurer();
 
 	/**
-	 * @return the client configuration callback.
+	 * @return the Rest Client configuration callback.
 	 * @since 4.2
+	 * @deprecated since 4.3 use {@link #getClientConfigurer()}
 	 */
+	@Deprecated
 	HttpClientConfigCallback getHttpClientConfigurer();
 
+	/**
+	 * @return the client configuration callback
+	 * @since 4.3
+	 */
+	<T> ClientConfigurationCallback<T> getClientConfigurer();
+
 	/**
 	 * @return the supplier for custom headers.
 	 */
@@ -274,7 +283,7 @@ interface TerminalClientConfigurationBuilder {
 		TerminalClientConfigurationBuilder withDefaultHeaders(HttpHeaders defaultHeaders);
 
 		/**
-		 * Configure the {@literal milliseconds} for the connect timeout.
+		 * Configure the {@literal milliseconds} for the connect-timeout.
 		 *
 		 * @param millis the timeout to use.
 		 * @return the {@link TerminalClientConfigurationBuilder}
@@ -327,7 +336,7 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) {
 
 		/**
 		 * Configure the path prefix that will be prepended to any HTTP(s) requests
-		 * 
+		 *
 		 * @param pathPrefix the pathPrefix.
 		 * @return the {@link TerminalClientConfigurationBuilder}
 		 * @since 4.0
@@ -342,21 +351,36 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) {
 
 		/**
 		 * set customization hook in case of a reactive configuration
-		 * 
+		 *
 		 * @param webClientConfigurer function to configure the WebClient
 		 * @return the {@link TerminalClientConfigurationBuilder}.
+		 * @deprecated since 4.3, use {@link #withClientConfigurer(ClientConfigurationCallback)} with
+		 *             {@link ReactiveRestClients.WebClientConfigurationCallback}
 		 */
+		@Deprecated
 		TerminalClientConfigurationBuilder withWebClientConfigurer(Function<WebClient, WebClient> webClientConfigurer);
 
 		/**
 		 * Register a {HttpClientConfigCallback} to configure the non-reactive REST client.
-		 * 
+		 *
 		 * @param httpClientConfigurer configuration callback, must not be null.
 		 * @return the {@link TerminalClientConfigurationBuilder}.
 		 * @since 4.2
+		 * @deprecated since 4.3, use {@link #withClientConfigurer(ClientConfigurationCallback)} with
+		 *             {@link RestClients.RestClientConfigurationCallback}
 		 */
+		@Deprecated
 		TerminalClientConfigurationBuilder withHttpClientConfigurer(HttpClientConfigCallback httpClientConfigurer);
 
+		/**
+		 * Register a {@link ClientConfigurationCallback} to configure the client.
+		 *
+		 * @param clientConfigurer configuration callback, must not be {@literal null}.
+		 * @return the {@link TerminalClientConfigurationBuilder}.
+		 * @since 4.3
+		 */
+		TerminalClientConfigurationBuilder withClientConfigurer(ClientConfigurationCallback<?> clientConfigurer);
+
 		/**
 		 * set a supplier for custom headers. This is invoked for every HTTP request to Elasticsearch to retrieve headers
 		 * that should be sent with the request. A common use case is passing in authentication headers that may change.
@@ -377,4 +401,15 @@ default TerminalClientConfigurationBuilder withSocketTimeout(long millis) {
 		 */
 		ClientConfiguration build();
 	}
+
+	/**
+	 * Callback to be executed to configure a client.
+	 *
+	 * @param <T> the type of the client configuration class.
+	 * @since 4.3
+	 */
+	@FunctionalInterface
+	interface ClientConfigurationCallback<T> {
+		T configure(T clientConfigurer);
+	}
 }