Remove Elasticsearch classes from suggest response data.

Original Pull Request #1940 
Closes #1302

Author: sothawo

Diff for: src/main/asciidoc/reference/elasticsearch-migration-guide-4.2-4.3.adoc

@@ -21,6 +21,14 @@ Check the sections on <<elasticsearch-migration-guide-4.2-4.3.deprecations>> and
 [[elasticsearch-migration-guide-4.2-4.3.deprecations]]
 == Deprecations
 
+=== suggest methods
+
+In `SearchOperations`, and so in `ElasticsearchOperations` as well, the `suggest` methods taking a `org.elasticsearch.search.suggest.SuggestBuilder` as argument and returning a `org.elasticsearch.action.search.SearchResponse` have been deprecated.
+Use `SearchHits<T> search(Query query, Class<T> clazz)` instead, passing in a `NativeSearchQuery` which can contain a `SuggestBuilder` and read the suggest results from the returned `SearchHit<T>`.
+
+In `ReactiveSearchOperations` the new `suggest` methods return a `Mono<org.springframework.data.elasticsearch.core.suggest.response.Suggest>` now.
+Here as well the old methods are deprecated.
+
 [[elasticsearch-migration-guide-4.2-4.3.breaking-changes]]
 == Breaking Changes
 
@@ -59,3 +67,7 @@ Some properties of the `org.springframework.data.elasticsearch.core.query.BulkOp
 === IndicesOptions change
 
 Spring Data Elasticsearch now uses `org.springframework.data.elasticsearch.core.query.IndicesOptions` instead of `org.elasticsearch.action.support.IndicesOptions`.
+
+=== Completion classes
+
+The classes from the package `org.springframework.data.elasticsearch.core.completion` have been moved to `org.springframework.data.elasticsearch.core.suggest`.

Diff for: src/main/asciidoc/reference/elasticsearch-operations.adoc

@@ -20,11 +20,11 @@ The default implementations of the interfaces offer:
 [NOTE]
 ====
 .Index management and automatic creation of indices and mappings.
+The `IndexOperations` interface and the provided implementation which can be obtained from an `ElasticsearchOperations` instance - for example with a call to `operations.indexOps(clazz)`- give the user the ability to create indices, put mappings or store template and alias information in the Elasticsearch cluster.
+Details of the index that will be created can be set by using the `@Setting` annotation, refer to <<elasticsearc.misc.index.settings>> for further information.
 
-The `IndexOperations` interface and the provided implementation which can be obtained from an `ElasticsearchOperations` instance - for example with a call to `operations.indexOps(clazz)`- give the user the ability to create indices, put mappings or store template and alias information in the Elasticsearch cluster. Details of the index that will be created
-can be set by using the `@Setting` annotation, refer to <<elasticsearc.misc.index.settings>> for further information.
-
-**None of these operations are done automatically** by the implementations of `IndexOperations` or `ElasticsearchOperations`. It is the user's responsibility to call the methods.
+**None of these operations are done automatically** by the implementations of `IndexOperations` or `ElasticsearchOperations`.
+It is the user's responsibility to call the methods.
 
 There is support for automatic creation of indices and writing the mappings when using Spring Data Elasticsearch repositories, see <<elasticsearch.repositories.autocreation>>
 
@@ -58,6 +58,7 @@ public class TransportClientConfig extends ElasticsearchConfigurationSupport {
   }
 }
 ----
+
 <1> Setting up the <<elasticsearch.clients.transport>>.
 Deprecated as of version 4.0.
 <2> Creating the `ElasticsearchTemplate` bean, offering both names, _elasticsearchOperations_ and _elasticsearchTemplate_.
@@ -82,6 +83,7 @@ public class RestClientConfig extends AbstractElasticsearchConfiguration {
   // no special bean creation needed                       <2>
 }
 ----
+
 <1> Setting up the <<elasticsearch.clients.rest>>.
 <2> The base class `AbstractElasticsearchConfiguration` already provides the `elasticsearchTemplate` bean.
 ====
@@ -127,6 +129,7 @@ public class TestController {
 }
 
 ----
+
 <1> Let Spring inject the provided `ElasticsearchOperations` bean in the constructor.
 <2> Store some entity in the Elasticsearch cluster.
 <3> Retrieve the entity with a query by id.
@@ -164,6 +167,7 @@ Contains the following information:
 * Maximum score
 * A list of `SearchHit<T>` objects
 * Returned aggregations
+* Returned suggest results
 
 .SearchPage<T>
 Defines a Spring Data `Page` that contains a `SearchHits<T>` element and can be used for paging access using repository methods.
@@ -182,12 +186,12 @@ Almost all of the methods defined in the `SearchOperations` and `ReactiveSearchO
 [[elasticsearch.operations.criteriaquery]]
 === CriteriaQuery
 
-`CriteriaQuery` based queries allow the creation of queries to search for data without knowing the syntax or basics of Elasticsearch queries. They allow the user to build queries by simply chaining and combining `Criteria` objects that specifiy the criteria the searched documents must fulfill.
+`CriteriaQuery` based queries allow the creation of queries to search for data without knowing the syntax or basics of Elasticsearch queries.
+They allow the user to build queries by simply chaining and combining `Criteria` objects that specifiy the criteria the searched documents must fulfill.
 
 NOTE: when talking about AND or OR when combining criteria keep in mind, that in Elasticsearch AND are converted to a **must** condition and OR to a **should**
 
-`Criteria` and their usage are best explained by example
-(let's assume we have a `Book` entity with a `price` property):
+`Criteria` and their usage are best explained by example (let's assume we have a `Book` entity with a `price` property):
 
 .Get books with a given price
 ====
@@ -211,19 +215,21 @@ Query query = new CriteriaQuery(criteria);
 
 When chaining `Criteria`, by default a AND logic is used:
 
-.Get all persons with first name _James_  and last name _Miller_:
+.Get all persons with first name _James_ and last name _Miller_:
 ====
 [source,java]
 ----
 Criteria criteria = new Criteria("lastname").is("Miller") <1>
   .and("firstname").is("James")                           <2>
 Query query = new CriteriaQuery(criteria);
 ----
+
 <1> the first `Criteria`
 <2> the and() creates a new `Criteria` and chaines it to the first one.
 ====
 
-If you want to create nested queries, you need to use subqueries for this. Let's assume we want to find all persons with a last name of _Miller_ and a first name of either _Jack_ or _John_:
+If you want to create nested queries, you need to use subqueries for this.
+Let's assume we want to find all persons with a last name of _Miller_ and a first name of either _Jack_ or _John_:
 
 .Nested subqueries
 ====
@@ -236,6 +242,7 @@ Criteria miller = new Criteria("lastName").is("Miller")  <.>
   );
 Query query = new CriteriaQuery(criteria);
 ----
+
 <.> create a first `Criteria` for the last name
 <.> this is combined with AND to a subCriteria
 <.> This sub Criteria is an OR combination for the first name _John_
@@ -281,5 +288,3 @@ Query query = new NativeSearchQueryBuilder()
 SearchHits<Person> searchHits = operations.search(query, Person.class);
 ----
 ====
-
-

Diff for: src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchRestTransportTemplate.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2021-2021 the original author or authors.
+ * Copyright 2021 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -32,6 +32,7 @@
 import org.elasticsearch.action.search.SearchResponse;
 import org.elasticsearch.index.query.MoreLikeThisQueryBuilder;
 import org.elasticsearch.index.query.QueryBuilders;
+import org.elasticsearch.search.suggest.SuggestBuilder;
 import org.springframework.data.elasticsearch.BulkFailureException;
 import org.springframework.data.elasticsearch.core.document.SearchDocumentResponse;
 import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates;
@@ -103,11 +104,12 @@ public <T> List<SearchHits<T>> multiSearch(List<? extends Query> queries, Class<
 
 		MultiSearchResponse.Item[] items = getMultiSearchResult(request);
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchHits<T>> callback = new ReadSearchDocumentResponseCallback<>(clazz, index);
 		List<SearchHits<T>> res = new ArrayList<>(queries.size());
 		int c = 0;
 		for (Query query : queries) {
-			res.add(callback.doWith(SearchDocumentResponse.from(items[c++].getResponse())));
+			res.add(callback.doWith(SearchDocumentResponse.from(items[c++].getResponse(), documentCallback::doWith)));
 		}
 		return res;
 	}
@@ -134,11 +136,13 @@ public List<SearchHits<?>> multiSearch(List<? extends Query> queries, List<Class
 		for (Query query : queries) {
 			Class entityClass = it1.next();
 
+			IndexCoordinates index = getIndexCoordinatesFor(entityClass);
+			ReadDocumentCallback<?> documentCallback = new ReadDocumentCallback<>(elasticsearchConverter, entityClass, index);
 			SearchDocumentResponseCallback<SearchHits<?>> callback = new ReadSearchDocumentResponseCallback<>(entityClass,
-					getIndexCoordinatesFor(entityClass));
+					index);
 
 			SearchResponse response = items[c++].getResponse();
-			res.add(callback.doWith(SearchDocumentResponse.from(response)));
+			res.add(callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith)));
 		}
 		return res;
 	}
@@ -166,11 +170,12 @@ public List<SearchHits<?>> multiSearch(List<? extends Query> queries, List<Class
 		for (Query query : queries) {
 			Class entityClass = it1.next();
 
+			ReadDocumentCallback<?> documentCallback = new ReadDocumentCallback<>(elasticsearchConverter, entityClass, index);
 			SearchDocumentResponseCallback<SearchHits<?>> callback = new ReadSearchDocumentResponseCallback<>(entityClass,
 					index);
 
 			SearchResponse response = items[c++].getResponse();
-			res.add(callback.doWith(SearchDocumentResponse.from(response)));
+			res.add(callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith)));
 		}
 		return res;
 	}
@@ -204,5 +209,11 @@ protected String getRuntimeLibraryVersion() {
 		return Version.CURRENT.toString();
 	}
 
+	@Override
+	@Deprecated
+	public SearchResponse suggest(SuggestBuilder suggestion, Class<?> clazz) {
+		return suggest(suggestion, getIndexCoordinatesFor(clazz));
+	}
+
 	// endregion
 }

Diff for: src/main/java/org/springframework/data/elasticsearch/core/AbstractElasticsearchTemplate.java

@@ -22,8 +22,6 @@
 import java.util.List;
 import java.util.stream.Collectors;
 
-import org.elasticsearch.action.search.SearchResponse;
-import org.elasticsearch.search.suggest.SuggestBuilder;
 import org.springframework.beans.BeansException;
 import org.springframework.context.ApplicationContext;
 import org.springframework.context.ApplicationContextAware;
@@ -432,11 +430,6 @@ protected void searchScrollClear(String scrollId) {
 	 */
 	abstract protected void searchScrollClear(List<String> scrollIds);
 
-	@Override
-	public SearchResponse suggest(SuggestBuilder suggestion, Class<?> clazz) {
-		return suggest(suggestion, getIndexCoordinatesFor(clazz));
-	}
-
 	// endregion
 
 	// region Helper methods
@@ -758,6 +751,7 @@ public ReadSearchDocumentResponseCallback(Class<T> type, IndexCoordinates index)
 			this.type = type;
 		}
 
+		@NonNull
 		@Override
 		public SearchHits<T> doWith(SearchDocumentResponse response) {
 			List<T> entities = response.getSearchDocuments().stream().map(delegate::doWith).collect(Collectors.toList());
@@ -778,6 +772,7 @@ public ReadSearchScrollDocumentResponseCallback(Class<T> type, IndexCoordinates
 			this.type = type;
 		}
 
+		@NonNull
 		@Override
 		public SearchScrollHits<T> doWith(SearchDocumentResponse response) {
 			List<T> entities = response.getSearchDocuments().stream().map(delegate::doWith).collect(Collectors.toList());

Diff for: src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplate.java

@@ -317,8 +317,10 @@ public <T> SearchHits<T> search(Query query, Class<T> clazz, IndexCoordinates in
 		SearchRequest searchRequest = requestFactory.searchRequest(query, clazz, index);
 		SearchResponse response = execute(client -> client.search(searchRequest, RequestOptions.DEFAULT));
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchHits<T>> callback = new ReadSearchDocumentResponseCallback<>(clazz, index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override
@@ -332,9 +334,10 @@ public <T> SearchScrollHits<T> searchScrollStart(long scrollTimeInMillis, Query
 
 		SearchResponse response = execute(client -> client.search(searchRequest, RequestOptions.DEFAULT));
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchScrollHits<T>> callback = new ReadSearchScrollDocumentResponseCallback<>(clazz,
 				index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override
@@ -346,9 +349,10 @@ public <T> SearchScrollHits<T> searchScrollContinue(@Nullable String scrollId, l
 
 		SearchResponse response = execute(client -> client.scroll(request, RequestOptions.DEFAULT));
 
-		SearchDocumentResponseCallback<SearchScrollHits<T>> callback = //
-				new ReadSearchScrollDocumentResponseCallback<>(clazz, index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
+		SearchDocumentResponseCallback<SearchScrollHits<T>> callback = new ReadSearchScrollDocumentResponseCallback<>(clazz,
+				index);
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override

Diff for: src/main/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplate.java

@@ -353,8 +353,9 @@ public <T> SearchHits<T> search(Query query, Class<T> clazz, IndexCoordinates in
 		SearchRequestBuilder searchRequestBuilder = requestFactory.searchRequestBuilder(client, query, clazz, index);
 		SearchResponse response = getSearchResponse(searchRequestBuilder);
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchHits<T>> callback = new ReadSearchDocumentResponseCallback<>(clazz, index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override
@@ -369,9 +370,10 @@ public <T> SearchScrollHits<T> searchScrollStart(long scrollTimeInMillis, Query
 
 		SearchResponse response = getSearchResponseWithTimeout(action);
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchScrollHits<T>> callback = new ReadSearchScrollDocumentResponseCallback<>(clazz,
 				index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override
@@ -385,9 +387,10 @@ public <T> SearchScrollHits<T> searchScrollContinue(@Nullable String scrollId, l
 
 		SearchResponse response = getSearchResponseWithTimeout(action);
 
+		ReadDocumentCallback<T> documentCallback = new ReadDocumentCallback<T>(elasticsearchConverter, clazz, index);
 		SearchDocumentResponseCallback<SearchScrollHits<T>> callback = new ReadSearchScrollDocumentResponseCallback<>(clazz,
 				index);
-		return callback.doWith(SearchDocumentResponse.from(response));
+		return callback.doWith(SearchDocumentResponse.from(response, documentCallback::doWith));
 	}
 
 	@Override