Impossible to set index.mapping.nested_objects_limit and index.requests.cache.enable

[marcreichman]

Java API client version

7.17.4

Java version

17.0.3.1

Elasticsearch Version

7.17.0

Problem description

Attached is a mapping JSON file which correctly creates an index using HLRC, and fails using the newer Java client. A unit test class is provided to show what works and what does not, with both clients.

The json file (fails_with_new_client.json) gives an example of an ES mapping and settings that can be read directly by the old high level client to create an index, but when we attempt to use the new client, we get unknown field exceptions.

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.indices.IndexSettings: Unknown field 'index.mapping.nested_objects.limit' (JSON path: ['index.mapping.nested_objects.limit']) (line no=1, column no=38, offset=37)

and

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.indices.IndexSettings: Unknown field 'requests' (JSON path: index.requests) (line no=1, column no=22, offset=21)

We provide unit tests demonstrating this failure and also showing that these fields cannot be used to create IndexSettings.

We've managed to get the desired functionality by separating out the mapping from the settings, and using separate builders to set what's possible, and then finally, once the index is created, using a low level rest call to set the remaining settings. This is, as you might expect, not satisfactory. These workarounds keep us from moving to the new client in production.

Having the HLRC be deprecated while the new client misses functionality seems like a peculiar decision
ESUnknownFieldExceptionTests.zip
.

[swallez]

Thanks for the very detailed report. This is an issue in the API specification used to produce the Java code: mappings should be mapping.

The fix will be included in the next 8.x and 7.17.x releases.

[KnowledgeGarden]

Timing on next release?

[j0u13]

Hi,
Being stuck by this issue in the migration of our project to the latest ES version I recently updated to 8.3.2 but I'm still facing this issue of the Java API serialising the settings with 'mappings' instead of 'mapping'. Is there somethings I missed or has the merge not fully worked for this issue?
The setting we're trying to use unsuccessfully is index.mapping.total_fields.limit.
The system still complains about mappings that should be mapping...