Skip to content

IndexSettingsConverter tries to deserialize routing_path as collection of string even if there is a string in raw JSON response #8233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
matteocalisti opened this issue Jun 12, 2024 · 1 comment · Fixed by #8234
Labels
8.x Relates to a 8.x client version Area: Generator Category: Bug

Comments

@matteocalisti
Copy link

Elastic.Clients.Elasticsearch version: 8.13.15, 8.14.0, 8.14.1

Elasticsearch version: 8.14.0

.NET runtime version: net6.0

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:

What happens: when trying to load index informations via Elastic.Clients.Elasticsearch.IndexManagement.IndicesNamespacedClient.GetAsync(indexName), deserialization of IndexSettings fails because the property RoutingPath is expected to be an ICollection<string>, IndexSettingsConverter tries to deserialize the property as an ICollection<string>, but the raw response from Elasticsearch backend has a string in place of the ICollection. The exception is shown below.

Desired behaviour: Converter correctly deserializes the string to a list of strings.

Somewhat similar to #8190 , but is this case I cannot apply the attribute SingleOrManyCollectionConverterAttribute to the property or apply another converter to IndexSettings because it already has the attribute

[JsonConverter(typeof(IndexSettingsConverter))]
public sealed partial class IndexSettings

At the moment I'm trying to use the low level client to get the response and deserialize it with custom code.

Steps to reproduce:
1.
2.
3.

Expected behavior
A clear and concise description of what you expected to happen.

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

# FailureReason: Unrecoverable/Unexpected BadResponse while attempting GET on http://127.0.0.1:59317/.ds-data-stream-unittest_d113a864-9211-4099-8f9c-26762aca2b03_timeseries-2024.06.11-000001
 - [1] BadResponse: Node: http://127.0.0.1:59317/ Exception: JsonException Took: 00:00:00.6247743
# Audit exception in step 1 BadResponse:
System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
# Inner Exception: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
# Exception:
Elastic.Transport.UnexpectedTransportException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
 ---> System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.ICollection`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 6.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 303
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.IndexManagement.IndexSettingsConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Types/IndexManagement/IndexSettings.g.cs:line 129
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](Utf8JsonReader& reader, JsonSerializerOptions options)
   at Elastic.Clients.Elasticsearch.Serialization.DictionaryResponseConverterFactory.ResolvableDictionaryResponseConverterInner`3.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch.Shared/Serialization/DictionaryResponseConverter.cs:line 78
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elastic.Transport.DistributedTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   at [...]

Raw response (taken from Kibana)

{
	".ds-data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries-2024.06.11-000001": {
		"aliases": {},
		"mappings": {
			"dynamic": "true",
			"_data_stream_timestamp": {
				"enabled": true
			},
			"_source": {
				"mode": "synthetic"
			},
			"properties": {
				"@timestamp": {
					"type": "date_nanos",
					"format": "strict_date_optional_time_nanos"
				},
				"code": {
					"type": "keyword",
					"time_series_dimension": true
				}
			}
		},
		"settings": {
			"index": {
				"mode": "time_series",
				"codec": "best_compression",
				"routing": {
					"allocation": {
						"include": {
							"_tier_preference": "data_hot"
						}
					}
				},
				"hidden": "true",
				"number_of_shards": "3",
				"time_series": {
					"end_time": "2024-06-11T15:08:40.000Z",
					"start_time": "2024-06-11T12:38:40.000Z"
				},
				"provided_name": ".ds-data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries-2024.06.11-000001",
				"creation_date": "1718116720832",
				"number_of_replicas": "1",
				"routing_path": "code",
				"uuid": "L9GG3ZXNRC6WnIkZAdoHNA",
				"version": {
					"created": "8505000"
				}
			}
		},
		"data_stream": "data-stream-unittest_b47fda94-ce27-4eef-ae25-4d54f296afe8_timeseries"
	}
}
@flobernd
Copy link
Member

Fixed in 8.14.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.x Relates to a 8.x client version Area: Generator Category: Bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants