Skip to content

Add config option for Decimal128 representation of numerics. #4916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.5.0-SNAPSHOT</version>
<version>4.5.x-GH-3444-SNAPSHOT</version>
<packaging>pom</packaging>

<name>Spring Data MongoDB</name>
Expand Down
2 changes: 1 addition & 1 deletion spring-data-mongodb-distribution/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<parent>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.5.0-SNAPSHOT</version>
<version>4.5.x-GH-3444-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion spring-data-mongodb/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<parent>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb-parent</artifactId>
<version>4.5.0-SNAPSHOT</version>
<version>4.5.x-GH-3444-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,8 @@ private JsonSchemaProperty computeSchemaForProperty(List<MongoPersistentProperty
Class<?> rawTargetType = computeTargetType(property); // target type before conversion
Class<?> targetType = converter.getTypeMapper().getWriteTargetTypeFor(rawTargetType); // conversion target type

if ((rawTargetType.isPrimitive() || ClassUtils.isPrimitiveArray(rawTargetType)) && targetType == Object.class) {

if ((rawTargetType.isPrimitive() || ClassUtils.isPrimitiveArray(rawTargetType)) && targetType == Object.class || ClassUtils.isAssignable(targetType, rawTargetType) ) {
targetType = rawTargetType;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,10 @@ static Collection<Object> getConvertersToRegister() {

List<Object> converters = new ArrayList<>();

converters.add(BigDecimalToStringConverter.INSTANCE);
converters.add(BigDecimalToDecimal128Converter.INSTANCE);
converters.add(StringToBigDecimalConverter.INSTANCE);
converters.add(Decimal128ToBigDecimalConverter.INSTANCE);
converters.add(BigIntegerToStringConverter.INSTANCE);
converters.add(StringToBigIntegerConverter.INSTANCE);
converters.add(BigIntegerToDecimal128Converter.INSTANCE);

converters.add(URLToStringConverter.INSTANCE);
converters.add(StringToURLConverter.INSTANCE);
converters.add(DocumentToStringConverter.INSTANCE);
Expand All @@ -111,6 +109,7 @@ static Collection<Object> getConvertersToRegister() {
converters.add(IntegerToAtomicIntegerConverter.INSTANCE);
converters.add(BinaryToByteArrayConverter.INSTANCE);
converters.add(BsonTimestampToInstantConverter.INSTANCE);
converters.add(NumberToNumberConverterFactory.INSTANCE);

converters.add(VectorToBsonArrayConverter.INSTANCE);
converters.add(ListToVectorConverter.INSTANCE);
Expand Down Expand Up @@ -193,6 +192,17 @@ public Decimal128 convert(BigDecimal source) {
}
}

/**
* @since 5.0
*/
enum BigIntegerToDecimal128Converter implements Converter<BigInteger, Decimal128> {
INSTANCE;

public Decimal128 convert(BigInteger source) {
return new Decimal128(new BigDecimal(source));
}
}

enum StringToBigDecimalConverter implements Converter<String, BigDecimal> {
INSTANCE;

Expand All @@ -212,6 +222,7 @@ public BigDecimal convert(Decimal128 source) {
}
}

@WritingConverter
enum BigIntegerToStringConverter implements Converter<BigInteger, String> {
INSTANCE;

Expand All @@ -220,6 +231,7 @@ public String convert(BigInteger source) {
}
}

@ReadingConverter
enum StringToBigIntegerConverter implements Converter<String, BigInteger> {
INSTANCE;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@
import org.springframework.data.convert.SimplePropertyValueConversions;
import org.springframework.data.convert.WritingConverter;
import org.springframework.data.mapping.model.SimpleTypeHolder;
import org.springframework.data.mongodb.core.convert.MongoConverters.BigDecimalToStringConverter;
import org.springframework.data.mongodb.core.convert.MongoConverters.BigIntegerToStringConverter;
import org.springframework.data.mongodb.core.convert.MongoConverters.StringToBigDecimalConverter;
import org.springframework.data.mongodb.core.convert.MongoConverters.StringToBigIntegerConverter;
import org.springframework.data.mongodb.core.mapping.MongoPersistentProperty;
import org.springframework.data.mongodb.core.mapping.MongoSimpleTypes;
import org.springframework.lang.Nullable;
Expand Down Expand Up @@ -154,6 +158,7 @@ public static class MongoConverterConfigurationAdapter {
private static final Set<Class<?>> JAVA_DRIVER_TIME_SIMPLE_TYPES = Set.of(LocalDate.class, LocalTime.class, LocalDateTime.class);

private boolean useNativeDriverJavaTimeCodecs = false;
private BigDecimalRepresentation bigDecimals = BigDecimalRepresentation.STRING;
private final List<Object> customConverters = new ArrayList<>();

private final PropertyValueConversions internalValueConversion = PropertyValueConversions.simple(it -> {});
Expand Down Expand Up @@ -298,6 +303,20 @@ public MongoConverterConfigurationAdapter useSpringDataJavaTimeCodecs() {
return useNativeDriverJavaTimeCodecs(false);
}

/**
* Configures the representation to for {@link java.math.BigDecimal} and {@link java.math.BigInteger} values in
* MongoDB. Defaults to {@link BigDecimalRepresentation#STRING}.
*
* @param representation the representation to use.
* @return this.
* @since 4.5
*/
public MongoConverterConfigurationAdapter bigDecimal(BigDecimalRepresentation representation) {

Assert.notNull(representation, "BigDecimalDataType must not be null");
this.bigDecimals = representation;
return this;
}
/**
* Optionally set the {@link PropertyValueConversions} to be applied during mapping.
* <p>
Expand Down Expand Up @@ -347,15 +366,26 @@ ConverterConfiguration createConverterConfiguration() {
svc.init();
}

List<Object> converters = new ArrayList<>(STORE_CONVERTERS.size() + 7);

if (bigDecimals == BigDecimalRepresentation.STRING) {

converters.add(BigDecimalToStringConverter.INSTANCE);
converters.add(StringToBigDecimalConverter.INSTANCE);
converters.add(BigIntegerToStringConverter.INSTANCE);
converters.add(StringToBigIntegerConverter.INSTANCE);
}

if (!useNativeDriverJavaTimeCodecs) {
return new ConverterConfiguration(STORE_CONVERSIONS, this.customConverters, convertiblePair -> true,

converters.addAll(customConverters);
return new ConverterConfiguration(STORE_CONVERSIONS, converters, convertiblePair -> true,
this.propertyValueConversions);
}

/*
* We need to have those converters using UTC as the default ones would go on with the systemDefault.
*/
List<Object> converters = new ArrayList<>(STORE_CONVERTERS.size() + 3);
converters.add(DateToUtcLocalDateConverter.INSTANCE);
converters.add(DateToUtcLocalTimeConverter.INSTANCE);
converters.add(DateToUtcLocalDateTimeConverter.INSTANCE);
Expand All @@ -375,6 +405,7 @@ ConverterConfiguration createConverterConfiguration() {

@ReadingConverter
private enum DateToUtcLocalDateTimeConverter implements Converter<Date, LocalDateTime> {

INSTANCE;

@Override
Expand Down Expand Up @@ -406,5 +437,25 @@ public LocalDate convert(Date source) {
private boolean hasDefaultPropertyValueConversions() {
return propertyValueConversions == internalValueConversion;
}

}

/**
* Strategy to represent {@link java.math.BigDecimal} and {@link java.math.BigInteger} values in MongoDB.
*
* @since 4.5
*/
public enum BigDecimalRepresentation {

/**
* Store values as {@link Number#toString() String}. Using strings retains precision but does not support range
* queries.
*/
STRING,

/**
* Store numbers using {@link org.bson.types.Decimal128}. Requires MongoDB Server 3.4 or later.
*/
DECIMAL128
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
package org.springframework.data.mongodb.core.convert;

import static java.time.ZoneId.*;
import static org.assertj.core.api.Assertions.*;
import static org.mockito.Mockito.*;
import static org.springframework.data.mongodb.core.DocumentTestUtils.*;
import static org.springframework.data.mongodb.test.util.Assertions.*;

import java.math.BigDecimal;
import java.math.BigInteger;
Expand All @@ -32,6 +32,7 @@
import java.util.function.Function;
import java.util.stream.Stream;

import org.assertj.core.api.Assertions;
import org.assertj.core.data.Percentage;
import org.bson.BsonDouble;
import org.bson.BsonUndefined;
Expand Down Expand Up @@ -2551,8 +2552,11 @@ void writeUnwrappedTypeWithComplexValue() {
assertThat(target) //
.containsEntry("address", new org.bson.Document("s", "1007 Mountain Drive").append("city", "Gotham")) //
.doesNotContainKey("street") //
.doesNotContainKey("city"); //

// use exact key matching, do not dive into nested documents
Assertions.assertThat(target) //
.doesNotContainKey("address.s") //
.doesNotContainKey("city") //
.doesNotContainKey("address.city");
}

Expand Down Expand Up @@ -3360,7 +3364,70 @@ void writesByteArrayAsIsIfNoFieldInstructionsGiven() {
converter.write(source, target);

assertThat(target.get("arrayOfPrimitiveBytes", byte[].class)).isSameAs(source.arrayOfPrimitiveBytes);
}

@Test // GH-3444
void convertsBigIntegerToDecimal128IfFieldTypeIndicatesConversion() {

WithExplicitTargetTypes source = new WithExplicitTargetTypes();
source.bigInteger = BigInteger.valueOf(101);

org.bson.Document target = new org.bson.Document();
converter.write(source, target);

assertThat(target.get("bigInteger")).isEqualTo(new Decimal128(source.bigInteger.longValueExact()));
}

@Test // GH-3444
void usesDecimal128NumericFormat() {

MappingMongoConverter converter = createConverter(MongoCustomConversions.BigDecimalRepresentation.DECIMAL128);

BigDecimalContainer container = new BigDecimalContainer();
container.value = BigDecimal.valueOf(2.5d);
container.map = Collections.singletonMap("foo", container.value);

org.bson.Document document = new org.bson.Document();
converter.write(container, document);

assertThat(document.get("value")).isInstanceOf(Decimal128.class);
assertThat(((org.bson.Document) document.get("map")).get("foo")).isInstanceOf(Decimal128.class);
}

@Test // GH-3444
void usesStringNumericFormat() {

MappingMongoConverter converter = createConverter(MongoCustomConversions.BigDecimalRepresentation.STRING);

BigDecimalContainer container = new BigDecimalContainer();
container.value = BigDecimal.valueOf(2.5d);
container.map = Collections.singletonMap("foo", container.value);

org.bson.Document document = new org.bson.Document();
converter.write(container, document);

assertThat(document).containsEntry("value", "2.5");
assertThat(document).containsEntry("map.foo", "2.5");
}

private MappingMongoConverter createConverter(
MongoCustomConversions.BigDecimalRepresentation bigDecimalRepresentation) {

MongoCustomConversions conversions = MongoCustomConversions.create(
it -> it.registerConverter(new ByteBufferToDoubleHolderConverter()).bigDecimal(bigDecimalRepresentation));

MongoMappingContext mappingContext = new MongoMappingContext();
mappingContext.setApplicationContext(context);
mappingContext.setSimpleTypeHolder(conversions.getSimpleTypeHolder());
mappingContext.afterPropertiesSet();

mappingContext.getPersistentEntity(Address.class);

MappingMongoConverter converter = new MappingMongoConverter(resolver, mappingContext);
converter.setCustomConversions(conversions);
converter.afterPropertiesSet();

return converter;
}

org.bson.Document write(Object source) {
Expand Down Expand Up @@ -4017,6 +4084,9 @@ static class WithExplicitTargetTypes {
@Field(targetType = FieldType.DECIMAL128) //
BigDecimal bigDecimal;

@Field(targetType = FieldType.DECIMAL128)
BigInteger bigInteger;

@Field(targetType = FieldType.INT64) //
Date dateAsLong;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,16 @@ public class Payment {
----
{
"_id" : ObjectId("5ca4a34fa264a01503b36af8"), <1>
"value" : NumberDecimal(2.099), <2>
"date" : ISODate("2019-04-03T12:11:01.870Z") <3>
"value" : NumberDecimal(2.099), <2>
"date" : ISODate("2019-04-03T12:11:01.870Z") <3>
}
----
<1> String _id_ values that represent a valid `ObjectId` are converted automatically. See xref:mongodb/template-crud-operations.adoc#mongo-template.id-handling[How the `_id` Field is Handled in the Mapping Layer]
for details.
<2> The desired target type is explicitly defined as `Decimal128` which translates to `NumberDecimal`. Otherwise the
<2> The desired target type is explicitly defined as `Decimal128` which translates to `NumberDecimal`.
Otherwise, the
`BigDecimal` value would have been truned into a `String`.
<3> `Date` values are handled by the MongoDB driver itself an are stored as `ISODate`.
<3> `Date` values are handled by the MongoDB driver itself are stored as `ISODate`.
====

The snippet above is handy for providing simple type hints. To gain more fine-grained control over the mapping process,
Expand Down Expand Up @@ -103,3 +104,11 @@ class MyMongoConfiguration extends AbstractMongoClientConfiguration {
}
}
----

[[mongo.numeric-conversion]]
== Big Number Format

MongoDB in its early days did not have support for large numeric values such as `BigDecimal`.
To persist `BigDecimal` and `BigInteger` values, Spring Data MongoDB converted values their `String` representation.
With MongoDB Server 3.4, `org.bson.types.Decimal128` offers a native representation for `BigDecimal` and `BigInteger`.
You can use the to the native representation by either annotating your properties with `@Field(targetType=DECIMAL128)` or by configuring the big decimal representation in `MongoCustomConversions` through `MongoCustomConversions.create(config -> config.bigDecimal(…))`.
Original file line number Diff line number Diff line change
Expand Up @@ -165,13 +165,13 @@ calling `get()` before the actual conversion

| `BigInteger`
| converter +
`String`
| `{"value" : "741" }`
`NumberDecimal`, `String`
| `{"value" : NumberDecimal(741) }`, `{"value" : "741" }`

| `BigDecimal`
| converter +
`String`
| `{"value" : "741.99" }`
`NumberDecimal`, `String`
| `{"value" : NumberDecimal(741.99) }`, `{"value" : "741.99" }`

| `URL`
| converter
Expand Down