s3 ListObjectsV2Command throws an error when a file size(in bytes) exceed 32-bit integer

[kamote]

Describe the bug

s3 ListObjectsV2Command throws an error when a file size(in bytes) exceed 32-bit integer.
Error occurred when listing a files on s3with size of 2.3GB. issue may relates to this commit c1a7dd5

Your environment

SDK version number

@aws-sdk/client-s3/@3.29.0

Is the issue in the browser/Node.js/ReactNative?

Node.js

Details of the browser/Node.js/ReactNative version

v14.16.0

Steps to reproduce

Please share code or minimal repo, and steps to reproduce the behavior.

Observed behavior

A clear and concise description of what happens.

Expected behavior

A clear and concise description of what you were expecting to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

TypeError: Expected 32-bit integer, got 2246119204\n    at expectSizedInt (/var/app/current/node_modules/@aws-sdk/smithy-client/dist/cjs/parse-utils.js:156:15)\n    at expectInt32 (/var/app/current/node_modules/@aws-sdk/smithy-client/dist/cjs/parse-utils.js:133:32)\n    at strictParseInt32 (/var/app/current/node_modules/@aws-sdk/smithy-client/dist/cjs/parse-utils.js:365:40)\n    at deserializeAws_restXml_Object (/var/app/current/node_modules/@aws-sdk/client-s3/dist/cjs/protocols/Aws_restXml.js:11980:62)\n    at /var/app/current/node_modules/@aws-sdk/client-s3/dist/cjs/protocols/Aws_restXml.js:11997:16\n    at Array.map (<anonymous>)\n    at deserializeAws_restXmlObjectList (/var/app/current/node_modules/@aws-sdk/client-s3/dist/cjs/protocols/Aws_restXml.js:11993:10)\n    at deserializeAws_restXmlListObjectsV2Command (/var/app/current/node_modules/@aws-sdk/client-s3/dist/cjs/protocols/Aws_restXml.js:7272:29)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:93:5)\n    at async /var/app/current/node_modules/@aws-sdk/middleware-serde/dist/cjs/deserializerMiddleware.js:6:20\n    at async /var/app/current/node_modules/@aws-sdk/middleware-signing/dist/cjs/middleware.js:11:20\n    at async StandardRetryStrategy.retry (/var/app/current/node_modules/@aws-sdk/middleware-retry/dist/cjs/StandardRetryStrategy.js:51:46)\n    at async /var/app/current/node_modules/@aws-sdk/middleware-logger/dist/cjs/loggerMiddleware.js:6:22\n    at async Object.listObjects

[hoangph271]

Reproduce:

• Create a bucket
• Upload any file larger than 2.1 GB (32-bit int)
• Use listObjectsV2 to list this file

Observed behavior:

The app crash with this error message:

Unhandled Rejection (TypeError): Expected 32-bit integer, got 4213057563

Expected behavior:

The app doesn't crash and list the file successfully

Screenshot:

[AllanZhengYP]

@kamote Thank you for reporting this. I can verify this is a bug with S3 client.

[AllanZhengYP]

Because now we are parsing the response more strictly, previously the size in the S3 API model was incorrectly labeled as an integer number and was not surfaced.

aws-sdk-js-v3/codegen/sdk-codegen/aws-models/s3.2006-03-01.json

Lines 12103 to 12105 in b9d88c4

	"com.amazonaws.s3#Size": {
	"type": "integer"
	},

Since JavaScript language is not applicable to integure, we should skip the strictParseInt32() validation for integers when deserializing the response.

/cc @adamthom-amzn

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.