Timestamp handling regression 2.7.12 -> 3.0.0

[lpandzic]

Following code works fine on 2.7.12 but fails on 3.0.0:

@Test
void shouldNotFail() {
    // given
    var given = Map.<String, Object>of("lastAssigment", "2023-06-05T11:08:33.107451Z");

    // when
    var actual = BDDAssertions.catchRuntimeException(() -> mapper.fromHash(given));

    // then
    then(actual).isNull();
}

org.opentest4j.AssertionFailedError: 
expected: 
  null
 but was: 
  org.springframework.data.mapping.MappingException: Could not resolve subtype of [simple type, class java.lang.Object]: missing type id property '@class'
   at [Source: (byte[])"{"lastAssigment":"2023-06-05T11:08:33.107451Z"}"; line: 1, column: 47]

[lpandzic]

Classpath contains following jackson modules:
jackson-datatype-jdk8
jackson-module-parameter-names
jackson-module-kotlin

[lpandzic]

Quick note: it happens on 3.0.0-M1

[jxblum]

Upon closer analysis of this issue, this problem does not only affect timestamps (such as date/time fields/properties), but also non-properly-evaluated scalar values, such as BigDecimals and BigIntegers, too (though values for fields & properties of these types can and should be treated as scalar values).

For instance, in test class Jackson2HashMapperUnitTests (source), there are 3 tests failing as of SD Redis 3.0.0-M5 for the reason shown above:

• dateValueShouldBeTreatedCorrectly (source)
• bigDecimalShouldBeTreatedCorrectly (source)
• bigIntegerShouldBeTreatedCorrectly (source)

Technically, caused by (for example):

Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_NUMBER_INT), expected START_ARRAY: need Array value to contain `As.WRAPPER_ARRAY` type information for class java.math.BigInteger
 at [Source: (byte[])"{"@class":"org.springframework.data.redis.mapping.Jackson2HashMapperUnitTests$WithBigWhatever","bigI":10}"; line: 1, column: 103] (through reference chain: org.springframework.data.redis.mapping.Jackson2HashMapperUnitTests$WithBigWhatever["bigI"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1950)
...
..
.

This specific problem reproduces with the (flattened) Jackson2HashMapperUnitTests.dateValueShouldBeTreatedCorrectly test case method, already.

Unfortunately, this test class is not being run as part of the SD Redis test suite using $ make test, for some non-apparent reason.

Additionally, this problem only occurs for the "flattened" configuration and works properly with non-flattened configuration.

[jxblum]

Upon even closer inspection, the only reason why this properly worked in the past (i.e. with SD Redis 2.7.x or earlier) is because either the "hash" (Map object passed to fromHash(..)) had to contain type metadata (such as with the @class JSON metadata property, which could be provided by a JsonTypeInfo.Id.CLASS declaration on the "valueType" to be mapped) or by specifically calling the fromHash(..) method with the target Class type, as was previously available in SD Redis 2.7.x, (along with this), but was then subsequently removed in SD Redis 3.0 and later, here, and then simply given a type of Object.class for an unknown reason, thereby resulting in a loss of information necessary to properly resolve the mapping.

And, once again, since the Jackson2HashMapperUnitTests was not being run as part of the SD Redis test suite, this regression was missed.

With the proper fix, your "hash" Map object argument to the Jackson2HashMapper.fromHash(..) method would simply need to specify:

Map<String, Object> hash = Map.of("@class", example.app.SomeValueTypeWithTimestamp.class.getName(), 
    "lastAssigment", "2023-06-05T11:08:33.107451Z");

...

[jxblum]

There is still some issue with newer versions Jackson combined with SD Redis 3.0.x+ that is not yet clear to me. More investigation is needed...

[lpandzic]

This was tested against jackson bom 2.14.2 first and 2.13.5 second as I first suspected this could be an issue with jackson. On other projects during Spring Boot 3 migration I've already faced issues with jackson 2.15.0 and 2.15.1 and records. Changing jackson doesn't affect this case in any way from my perspective (and the provided test).

[lpandzic]

Some change between 2.7.12 and 3.0.0-M1 is causing this.
During my investigation yesterday my next suspect on the list was this commit - 93c25b0 but I haven't checked it as 2.7.12 was good enough for me so I moved on.

[jxblum]

With the help of @christophstrobl, we managed to resolve this issue.

However, your test case shown above is not exactly precise, particularly in how Jackson would resolve a hash from a non-flattened arrangement.

In the flattened case, you would get back a LinkedHashMap, and in the non-flattened (hierarchical) case, an Exception of a similar type would still be thrown. Jackson is somewhat dumb in that it still needs additional help (specifically in the polymorphic case) to resolve property value types. However, given the correct metadata, you'd even get back strongly typed Object, if that is what you wanted.

To demonstrate this, I have written 2 cases to show by example what you would need to pass to the Jackson2HashMapper.fromHash(:Map) method if the Map argument you passed did not originate from the Jackson2HashMapper.toHash(:Object) method. See test cases for the flattened configuration and unflattened configuration.

[jxblum]

Unable to back port this fix to 3.0.x since SD Redis 3.0.x is based on Jackson 2.14.x yet, and disabling the MappingFeature requiring type ID for subtypes was not introduced until Jackson 2.15 (essentially this, which was necessary to implement this fix).

I am not sure what the alternative and equivalent configuration would be for Jackson 2.14.

However, given SD Redis 3.1.0 is already GA, requiring users to upgrade from SD Redis 2.x immediately to 3.1.x is not out of the question, particularly given SD Redis 3.0.x will be out of OSS support as of November 2023.