Client version 8.2.2 does not build correct search request with runtime_mappings

[sothawo]

Java API client version

8.2.2

Java version

17.0.3

Elasticsearch Version

8.2.2

Problem description

The following code stores a document and then should do a search with a runtime_mappings field:

ElasticsearchClient client = createTheClient();

String index = "testindex";

var p = new Product("p1", 42.0);

client.index(ir -> ir
		.index(index)
		.document(p));

client.indices().flush(f -> f.index(index));

RuntimeField runtimeField = RuntimeField.of(rf -> rf 
		.type(RuntimeFieldType.Double) 
		.script(Script.of(s -> s 
				.inline(i -> i. 
						source("emit(doc['price'].value * 1.19)") 
				) 
		)) 
); 

client.search(sr -> sr 
		.index(index) 
		.runtimeMappings("priceWithTax", Collections.singletonList(runtimeField)), // NOTE: the builder accepts only lists here
		Person.class); 

The request that is sent to Elasticsearch has the following body:

{
  "runtime_mappings": {
    "priceWithTax": [
      {
        "script": {
          "source": "emit(doc['price'].value * 1.19)"
        },
        "type": "double"
      }
    ]
  }
}

Note that the priceWithTax property is an array, the client builder requires a list here.

This request leads to an error response from the server:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Expected map for runtime field [priceWithTax] definition but got a java.util.ArrayList"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "testindex",
        "node" : "ef-uEMwKS1mbgKHO-Gz3mg",
        "reason" : {
          "type" : "mapper_parsing_exception",
          "reason" : "Expected map for runtime field [priceWithTax] definition but got a java.util.ArrayList"
        }
      }
    ]
  },
  "status" : 400
}

Neither the documentation for runtime_fields in a search request nor for runtime_fields in a mapping specify this as an array.

[sothawo]

Any news on this issue? Error still exists in 8.3.3.

[MatthewSwarbrick]

Also having this issue, the only difference is that I'm using withJson to build the SearchRequest object.
The json string works fine when testing, but when built into the SearchRequest it converts the runtime mapping to an array which breaks it as @sothawo has explained above

We need to use runtime mappings, the only way to get this working for us now is to use a native client and building json manually which takes out the benefits of this library

[MatthewSwarbrick]

Looks like the type is incorrect in SearchRequest.java
Instead of:

private final Map<String, List<RuntimeField>> runtimeMappings;

It should be:
private final Map<String, RuntimeField> runtimeMappings;

I wanted to create a PR but this is generated code and can't be updated through PR

[sothawo]

ping @swallez

[swallez]

runtime_mappings is incorrectly defined in the spec as a map of either a single mapping or an array of mappings when it should really be a map of single mappings. Fix in progress.

[sothawo]

Any info when this will be released?