Elasticsearch Java API Client unable to put otherwise valid index template due to falsely-required fields

[tomas0svk]

Elasticsearch Version

8.5.1

Installed Plugins

No response

Java Version

OpenJDK 64-Bit Server VM 18.9 build 11.0.16+8

OS Version

5.15.0-0.bpo.3-amd64 elastic/elasticsearch#1 SMP Debian 5.15.15-2~bpo11+1 (2022-02-03) x86_64 GNU/Linux

Problem Description

I am trying to put an index template definition using Java API client's PutIndexTemplateRequest.Builder.withJson(). Template itself (see below) seems correct, if I use kibana's devtools, response says "acknowledged": true.
However API client fails during request serialization on Missing required property 'PatternTokenizer.flags' or Missing required property 'PatternTokenizer.group', despite groups is optional parameter with default -1.

If I dive into the client code, PatternTokenizer,java requires these two parameters to be set
this.flags = ApiTypeHelper.requireNonNull(builder.flags, this, "flags"); this.group = ApiTypeHelper.requireNonNull(builder.group, this, "group"); this.pattern = ApiTypeHelper.requireNonNull(builder.pattern, this, "pattern");

Steps to Reproduce

Code:
String theTemplateName = FilenameUtils.getBaseName(aTemplateUrl.getFile()); String theTemplateName = FilenameUtils.getBaseName(aTemplateUrl.getFile()); PutIndexTemplateResponse theResponse = elasticClient .indices() .putIndexTemplate(new PutIndexTemplateRequest.Builder().name(theTemplateName).withJson(theTemplateJson).build());

Pattern:
{ "index_patterns": [ "e-logs-*" ], "template": { "settings": { "analysis": { "tokenizer": { "stacktrace_tokenizer": { "type": "pattern", "pattern": "[\\s:\\[\\](),]+" }, "package_tokenizer": { "type": "pattern", "pattern": "[.]" } }, "analyzer": { "stacktrace_analyzer": { "tokenizer": "stacktrace_tokenizer" }, "package_analyzer": { "tokenizer": "package_tokenizer" } } } }, "mappings": { "dynamic_templates": [ { "string_fields": { "match": "*", "match_mapping_type": "string", "mapping": { "type": "keyword" } } } ], "properties": { "time": { "type": "date", "format": "epoch_millis" }, "properties": { "type": "object", "properties": { "emans-client-time": { "type": "date", "format": "epoch_millis" }, "bundle-id": { "type": "short" }, "camel-routeId": { "type": "text", "analyzer": "package_analyzer" }, "emans-client-commit": { "type": "keyword" } } }, "message": { "type": "text" }, "thread": { "type": "text" }, "logger": { "type": "text", "analyzer": "package_analyzer" }, "exception": { "type": "text", "analyzer": "stacktrace_analyzer" } } } }, "priority": 1, "composed_of": [ "com-e-common" ] }

Logs (if relevant)

Caused by: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'PatternTokenizer.group'
at co.elastic.clients.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:76)
at co.elastic.clients.elasticsearch._types.analysis.PatternTokenizer.(PatternTokenizer.java:62)
at co.elastic.clients.elasticsearch._types.analysis.PatternTokenizer.(PatternTokenizer.java:49)
at co.elastic.clients.elasticsearch._types.analysis.PatternTokenizer$Builder.build(PatternTokenizer.java:168)
at co.elastic.clients.elasticsearch._types.analysis.PatternTokenizer$Builder.build(PatternTokenizer.java:121)
at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:80)
at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:209)

[tomas0svk]

The same applies also to Analyzer.pattern (see my index template enclosed)

Error deserializing co.elastic.clients.elasticsearch._types.analysis.Analyzer: Property 'type' not found (JSON path: template.settings.analysis.analyzer.stacktrace_analyzer) (line no=23, column no=35, offset=541)

Official documentation states 'Analyzer type. Accepts built-in analyzer types. For custom analyzers, use custom or omit this parameter.' so the parameter should be optional

[elasticsearchmachine]

Pinging @elastic/clients-team (Team:Clients)

[vbhavsar]

@tomas0svk The spec defines PatternTokenizer.flags and PatternTokenizer.group as required fields so the behavior seems correct.

[tomas0svk]

@tomas0svk The spec defines PatternTokenizer.flags and PatternTokenizer.group as required fields so the behavior seems correct.

According to this https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-tokenizer.html these parameters are not required for REST API (and so it also works with no values provided using REST client like CURL). I would expect identical behavior for REST and JAVA client.

[vbhavsar]

@tomas0svk Good point. In that case the spec should be updated to make the parameters optional.