Multiget() with unknown id throws UnexpectedTransportException: Unable to deserialize union.

[hroi]

Elastic.Clients.Elasticsearch version: 8.0.4

Elasticsearch version: 8.5.3

.NET runtime version: 7.0.101

Operating system version: Darwin 22.2.0

Description of the problem including expected versus actual behavior:

Expected behaviour: Returns a response which includes entries for found and not found ids.
Actual behaviour
MultiGet() throws UnexpectedTransportException if an id does not exist in the index.

Steps to reproduce:

[Fact]
public async Task MultigetAsyncRepro()
{
    var nodePool = new SingleNodePool(_elasticsearchInstance.Uri);
    var auth = new BasicAuthentication(_elasticsearchInstance.Username, _elasticsearchInstance.Password);
    var settings = new ElasticsearchClientSettings(nodePool).Authentication(auth);
    var client = new ElasticsearchClient(settings);
    var foos = new Foo[] { new() { Id = "001", Name = "FooA" }, new() { Id = "002", Name = "FooB" } };
    await client.IndexManyAsync(foos, "foos");

    // ok
    var response1 =
        await client.MultiGetAsync<Foo>(new MultiGetRequest("foos") { Ids = new Ids(new[] { "001", "002" }) });
    response1.Docs.ElementAt(0).Match(ok => Assert.Equal(foos[0], ok.Source), _ => Assert.Fail("err"));
    response1.Docs.ElementAt(1).Match(ok => Assert.Equal(foos[1], ok.Source), _ => Assert.Fail("err"));

    // blows up
    var response2 =
        await client.MultiGetAsync<Foo>(new MultiGetRequest("foos")
            {
                Ids = new Ids(new[] { "001", "002", "nonexistant" })
            });
    response2.Docs.ElementAt(0).Match(ok => Assert.Equal(foos[0], ok.Source), _ => Assert.Fail("err"));
    response2.Docs.ElementAt(1).Match(ok => Assert.Equal(foos[1], ok.Source), _ => Assert.Fail("err"));
    response2.Docs.ElementAt(2).Match(ok => Assert.False(ok.Found), _ => Assert.Fail("err"));

}

The above throws an exception on the second MultigetAsync() call:

Elastic.Transport.UnexpectedTransportException: Unable to deserialize union.

Elastic.Transport.UnexpectedTransportException
Unable to deserialize union.
   at Elastic.Transport.DefaultHttpTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline)
   at Elastic.Transport.DefaultHttpTransport`1.RequestAsync[TResponse](HttpMethod method, String path, PostData data, RequestParameters requestParameters, CancellationToken cancellationToken)
   at Integrations.Test.EsDal.EsDalTests.MultigetAsyncRepro() in /Users/hroi/bdq/Daffy/tests/Integrations.Test/EsDal/EsDalTests.cs:line 78
   at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_0.<<InvokeTestMethodAsync>b__1>d.MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 264
--- End of stack trace from previous location ---
   at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90

System.Text.Json.JsonException
Unable to deserialize union.
   at Elastic.Clients.Elasticsearch.Serialization.ResponseItemConverterFactory.ResponseItemConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/Serialization/ResponseItemConverterFactory.cs:line 91
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[TValue](Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo, ReadStack& state)
   at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.ReadFromStreamAsync[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyAsync[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpTransportClient.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultHttpTransport`1.RequestAsync[TResponse](HttpMethod method, String path, PostData data, RequestParameters requestParameters, CancellationToken cancellationToken)

Expected behavior

response2.Docs.ElementAt(0).Match(ok => Assert.Equal(foos[0], ok.Source), _ => Assert.Fail("err"));
response2.Docs.ElementAt(1).Match(ok => Assert.Equal(foos[1], ok.Source), _ => Assert.Fail("err"));
response2.Docs.ElementAt(2).Match(ok => Assert.False(ok.Found), _ => Assert.Fail("err"));

[stevejgordon]

Thanks for raising this issue. I'll take a look into this.