Error deserializing co.elastic.clients.elastic search._types.mapping.SourceField: Unknown field

[Enomine88]

Java API client version

<dependency>       <groupId>co.elastic.clients</groupId>       <artifactId>elasticsearch-java</artifactId>       <version>8.4.0</version>     </dependency>

Java version

1.8.17

Elasticsearch Version

IDK - the current Elasticsearch Cloud Version hosted by Elastic itself

Problem description

Trying to add a schema (index mapping) to an index.

  <dependencies>
  	<dependency>
      <groupId>co.elastic.clients</groupId>
      <artifactId>elasticsearch-java</artifactId>
      <version>8.4.0</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.12.3</version>
    </dependency>

    <dependency>
      <groupId>jakarta.json</groupId>
      <artifactId>jakarta.json-api</artifactId>
      <version>2.0.1</version>
    </dependency>

    <dependency>
      <groupId>org.elasticsearch</groupId>
   	  <artifactId>elasticsearch-x-content</artifactId>
      <version>8.1.0</version>
	</dependency>

	<dependency>
      <groupId>org.elasticsearch.client</groupId>
      <artifactId>transport</artifactId>
      <version>7.17.6</version>
	</dependency>
  </dependencies>

Code:

		XContentBuilder xcb = XContentFactory.jsonBuilder();
		xcb.startObject();
		{
			xcb.startObject("properties");
			{
				xcb.startObject("Rechnungsdatum");
				{
					xcb.field("type", "date");
				}
				xcb.endObject();
			}
			xcb.endObject();
		}
		xcb.endObject();
		String json = Strings.toString(xcb);
		System.out.println(json);
		SourceField sf = new SourceField.Builder().withJson(new StringReader(json)).build();
		PutMappingRequest pmr2 = new PutMappingRequest.Builder().index("rechnungen4").source(sf).build();
		client.indices().putMapping(pmr2);

Gives:

{"properties":{"Rechnungsdatum":{"type":"date"}}}
Exception in thread "main" co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elastic
search._types.mapping.SourceField: Unknown field 'properties' (JSON path: properties) (line no=1, column no=14, offset=1
3)
	at co.elastic.clients.json.ObjectDeserializer.parseUnknownField(ObjectDeserializer.java:221)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:178)
	at co.elastic.clients.util.WithJsonObjectBuilderBase.withJson(WithJsonObjectBuilderBase.java:54)
	at co.elastic.clients.json.WithJson.withJson(WithJson.java:57)

I am not so much into this all that i could prove it is an issue / bug but i really can't help myself. Can't see what i am doing wrong.
Could be related to #392 and #361 but i don't know.

Any suggestions how to generate JSON by Java Code (not typing JSON into the Code nor in a textfile) and bring that JSON to an execution in the ElasticsearchClient? Please give any hint what could be a smart way to do so. Is there an alternative to XContentBuilder?

Thanks - Enomine

[l-trotta]

Hello, thank you for the report! It's not really clear to me what you're trying to do, I'm going to assume you want to add a mapping for the Rechnungsdatum field, so here's how I'd do it with the java client, without using json:

        esClient.indices().putMapping(pm -> pm
            .index("rechnungen4")
            .properties("Rechnungsdatum", pr -> pr
                .date(d -> d)));

Ideally one should be able to write almost every request just by leveraging the methods of the java client, and only use json for very specific cases the java client doesn't yet support. Still not sure this was the problem though, so let me know :)

[yazabara]

The same issue with 'normalizer'

java.util.concurrent.CompletionException: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.mapping.TextProperty: Unknown field 'normalizer' (JSON path: properties.name.normalizer) (in object at line no=1, column no=85, offset=84)
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run$$$capture(CompletableFuture.java:1807)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java)
	at io.micrometer.context.ContextSnapshot.lambda$wrap$0(ContextSnapshot.java:78)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:842)
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.mapping.TextProperty: Unknown field 'normalizer' (JSON path: properties.name.normalizer) (in object at line no=1, column no=85, offset=84)

[Enomine88]

@yazabara

The same issue with 'normalizer'

Please ALWAYS provide example code!

@l-trotta As i worked on my Master-thesis these days, i wanted to try ALL WAYS how to interact with elasticsearch. Thus i tried "not common" ways.

@yazabara My master-thesis contains a section where 4 alternatives to communicate with Elasticsearch are shown by code and explained. This is section 5.6 with the title "Ein Schema hinzufügen". I will paste my master-thesis into this post. You can translate my explanation-texts of my master-thesis with chatGPT into your favorite language.

2022.11.29_Masterarbeit_Vereinheitlichung_und_Zentralisierung_von_Schnittstellenlogdateien.pdf

EDIT: And if you want to know how to get a connection to Elasticsearch then view the other prior sections.
EDIT2: Page 39 (47) contains the version for ContentBuilder that actually worked for me. I have no clue today, 2 years what the issue was, but if you take ALL my screenshots in my masterthesis you should get a working example.

Thanks - Enomine

[yazabara]

@Enomine88

I've just added annotation param 'normalizer'
Example:
@Field(type = FieldType.Text, fielddata = true, normalizer = "my_normalizer")

org.springframework.boot:3.0.2. ('org.springframework.boot:spring-boot-starter-data-elasticsearch')

The document definition is:

@Getter
@Setter
@ToString(callSuper = true)
@Setting(settingPath = "elasticsearch/custom-es.json")
@Document(indexName = "#{@environment.getProperty('com.work.elasticsearch.indexes.my.document')}", createIndex = false)
public class MyDocument extends MyParent {

   ...
   @Field(type = FieldType.Text, fielddata = true, normalizer = "my_normalizer")
   private String name;
}

To create an index I'm using special method (because of blue/green strategy implementation)

private final ElasticsearchClient elasticsearchClient;
private final ElasticsearchOperations operations;

....
String json = operations.indexOps(aClass).createMapping().toJson();

CreateIndexRequest createIndexRequest = CreateIndexRequest
    .of(builder -> builder
        .index(indexName)
        .settings(settings -> settings(aClass, settings))
        .mappings(mappings -> mappings.withJson(new StringReader(json)))
    );
log.debug("Elasticsearch index {} created ", elasticsearchClient.indices().create(createIndexRequest));
...

Without normalizer annotation param it works perfectly. (Also with analyzer definition - fine)

[l-trotta]

@Enomine88 thank you so much for your interest in Elasticsearch, if you have any other question feel free to reopen this issue or create a new one (we promise we'll be faster this time!)

@yazabara unfortunately we cannot help you with that particular issue, since you're using spring data elasticsearch and not the java client directly, I suggest you open an issue on their repository if the problem persists.