[SR-13015] Decimal calculation incorrect on Linux

[swift-ci]

Previous ID	SR-13015
Radar	rdar://problem/64517015
Original Reporter	mattjgalloway (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Environment

macOS (Catalina 10.15.5):

$ swift --version
Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53)
Target: x86_64-apple-darwin19.5.0

Linux (Debian stretch, using Ubuntu 16.04 binaries):

$ swift --version
Swift version 5.2.4 (swift-5.2.4-RELEASE)
Target: x86_64-unknown-linux-gnu

en_GB locale

Additional Detail from JIRA

Votes	4
Component/s	Foundation
Labels	Bug
Assignee	None
Priority	Medium

md5: 5c0852cd4db2eecc9ea3b60ca7a075dc

Issue Description:

I've noticed through a rather strange turn of events that there appears to be a problem in Foundation's Decimal class on Linux.

Take for example this code:

let a = Decimal(string: "119.993")!
let b = Decimal(string: "4.1565")!
let c = Decimal(string: "18.209")!
let d = Decimal(string: "258.469")!
let result = (a * b) + (c * (a / d))

On macOS you'll get the following result:

result: Decimal = 507.204346

On Linux you'll get the following result:

result: Decimal = 166.921979

My calculator agrees with macOS.

If I change pretty much any of the values for a, b, c and d then the result will end up being the same on both platforms. For example, if you change a to 119.99 - so simply removing the final 3 - then the result is exactly the same on both platforms.

Apologies for the convoluted calculation - I've been trying to get a minimal test case, but whenever I change anything the problem goes away.

[spevans]

This can be simplified down to

print(Decimal(string: "498.7509045")! + Decimal(string: "8.453441368210501065891847765109162027")! == Decimal(string: "507.2043458682105010658918477651091")!) 

Which gives true on macOS (which is correct) and false on Linux.

[swift-ci]

Comment by Matt Galloway (JIRA)

@spevans thanks for that! Yes you're right. And if you remove the 7 from the end of the second decimal then it calculates correctly.

[swift-ci]

Comment by Matt Galloway (JIRA)

I've narrowed down to even this:

Decimal(string: "429.5000001")! + Decimal(string: "0.000000000000000000000000000000000001")! == Decimal(string: "429.500000100000000000000000000000000001")!

That's true on macOS but false on Linux. If you lower that first number by any amount then it calculates correctly on Linux.

[swift-ci]

Comment by Matt Galloway (JIRA)

I think it's in here:
https://github.com/apple/swift-corelibs-foundation/blob/6793a4ef79fcc00a76bbce2a3f3b688c3a1ad986/Sources/Foundation/Decimal.swift#L1367

At that point, a becomes 89.217633 when doing the addition in my 429.5000001 + 0.000000000000000000000000000000000001 example above.

I'll keep going down this rabbit hole.

[spevans]

Yeah It looks to be NSDecimalNormalized here's a test case for this example

 func test_13015() {
        let a = Decimal(string: "498.7509045")!
        let b = Decimal(string: "8.453441368210501065891847765109162027")!

        var aNormalized = a
        var bNormalized = b
        let normalizeError = NSDecimalNormalize(&aNormalized, &bNormalized, .plain)
        XCTAssertEqual(normalizeError, NSDecimalNumber.CalculationError.lossOfPrecision)
        XCTAssertEqual(aNormalized.exponent, -31)
        XCTAssertEqual(aNormalized._mantissa.0, 0)
        XCTAssertEqual(aNormalized._mantissa.1, 21760)
        XCTAssertEqual(aNormalized._mantissa.2, 45355)
        XCTAssertEqual(aNormalized._mantissa.3, 11455)
        XCTAssertEqual(aNormalized._mantissa.4, 62709)
        XCTAssertEqual(aNormalized._mantissa.5, 14050)
        XCTAssertEqual(aNormalized._mantissa.6, 62951)
        XCTAssertEqual(aNormalized._mantissa.7, 0)
        XCTAssertEqual(bNormalized.exponent, -31)
        XCTAssertEqual(bNormalized._mantissa.0, 56467)
        XCTAssertEqual(bNormalized._mantissa.1, 17616)
        XCTAssertEqual(bNormalized._mantissa.2, 59987)
        XCTAssertEqual(bNormalized._mantissa.3, 21635)
        XCTAssertEqual(bNormalized._mantissa.4, 5988)
        XCTAssertEqual(bNormalized._mantissa.5, 63852)
        XCTAssertEqual(bNormalized._mantissa.6, 1066)
        XCTAssertEqual(bNormalized._mantissa.7, 1628)
        let result = a + b
        XCTAssertEqual(result, Decimal(string: "507.2043458682105010658918477651091")!)
    }

[weissi]

@swift-ci create

[weissi]

@swift-ci create

[spevans]

#2827

[spevans]

Fixed in swift-DEVELOPMENT-SNAPSHOT-2020-06-22-a thanks to @buttaface

Backport to 5.3 is #2829

$ cat sr-13015.swift
import Foundation

let a = Decimal(string: "119.993")!
let b = Decimal(string: "4.1565")!
let c = Decimal(string: "18.209")!
let d = Decimal(string: "258.469")!
let result = (a * b) + (c * (a / d))
print(result)
print(result == Decimal(string: "507.2043458682105010658918477651091")!)

$~/swift-build/swift-DEVELOPMENT-SNAPSHOT-2020-06-22-a-ubuntu18.04/usr/bin/swift sr-13015.swift 
507.2043458682105010658918477651091
true