Skip to content

Additional UTF8Json overloads, consolidate how we tell it to write/serialize. #4313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jan 10, 2020

Conversation

Mpdreamz
Copy link
Member

@Mpdreamz Mpdreamz commented Jan 9, 2020

Continuation of #4312

Pulling in the bits initially opened here: #4209

Either memory stream or to serialize on the writer.

This consolidates and funnels places that need to write bytes/streams/objects directly.

Isolated from #4172 and #4191 which both grew to be too big to review

Prior:

Method Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
NEST 530.15 ms 10.534 ms 13.323 ms 527.06 ms 17000.0000 4000.0000 - 197.13 MB
ListOfObjects 417.04 ms 8.318 ms 20.089 ms 410.26 ms 11000.0000 - - 75.67 MB
StaticListOfObjects 386.24 ms 4.305 ms 4.027 ms 385.53 ms 11000.0000 - - 74.47 MB
PrecomputedPostDataOfObjects 396.67 ms 4.197 ms 3.720 ms 396.63 ms 11000.0000 - - 73.82 MB
StaticListOfStrings 45.40 ms 1.077 ms 3.073 ms 44.50 ms 4000.0000 - - 32.51 MB
PrecomputedPostDataOfStrings 46.32 ms 0.922 ms 2.507 ms 45.65 ms 4000.0000 - - 32.48 MB
PostDataString 45.91 ms 2.028 ms 5.949 ms 46.76 ms - - - 32.15 MB
PrecomputedPostDataString 29.56 ms 0.583 ms 1.066 ms 29.47 ms - - - 7.95 MB
PostDataByteArray 29.18 ms 0.578 ms 1.601 ms 29.08 ms - - - 7.95 MB
PostDataReadOnlyMemory 28.03 ms 0.559 ms 1.372 ms 27.80 ms - - - 7.95 MB
PostDataStreamHandler 28.09 ms 0.552 ms 1.090 ms 28.03 ms - - - 7.95 MB

After:

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
NEST 329.47 ms 4.382 ms 3.885 ms 8000.0000 2000.0000 - 145.15 MB
ListOfObjects 300.87 ms 4.014 ms 3.755 ms 11000.0000 - - 75 MB
StaticListOfObjects 292.50 ms 6.562 ms 6.138 ms 11000.0000 - - 73.76 MB
PrecomputedPostDataOfObjects 293.36 ms 3.864 ms 3.614 ms 11000.0000 - - 74.5 MB
StaticListOfStrings 29.14 ms 0.582 ms 1.092 ms 4000.0000 - - 32.65 MB
PrecomputedPostDataOfStrings 28.86 ms 0.556 ms 0.546 ms 4000.0000 - - 32.5 MB
PostDataString 29.76 ms 1.149 ms 3.369 ms - - - 32.17 MB
PrecomputedPostDataString 18.10 ms 0.357 ms 0.587 ms - - - 7.95 MB
PostDataByteArray 17.57 ms 0.346 ms 0.355 ms - - - 7.95 MB
PostDataReadOnlyMemory 17.42 ms 0.232 ms 0.193 ms - - - 7.95 MB
PostDataStreamHandler 17.66 ms 0.337 ms 0.426 ms - - - 7.95 MB

Copy link
Contributor

@russcam russcam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Left one minor nitpick

The error and std dev generally look much better after too! 🎉

New types

* `ReadOnlyMemory`, only available in `netstandard2.1` allows you to
post `ReadOnlyMemory<byte>`
* `StreamHandler` allows you to inject a stream handler and write to the
request streams the best way you see fit yourself

Performance

* `PostData.String` now reuses bytes
* `PostData.MultiJson(string[]) uses way less allocations
Expose reusable stream handling on `PostData`

Add tests for new types so that they behave and write as expected
Either memory stream or to serialize on the writer.

This consolidates and funnels places that need to write
bytes/streams/objects directly

(cherry picked from commit de62914)
@Mpdreamz Mpdreamz force-pushed the feature/master/utf8-json-continued branch from 1e7ff71 to e08f2ac Compare January 10, 2020 08:24
@Mpdreamz Mpdreamz changed the base branch from feature/master/postdata-improvements to master January 10, 2020 08:24
@Mpdreamz Mpdreamz merged commit fadef66 into master Jan 10, 2020
@Mpdreamz Mpdreamz deleted the feature/master/utf8-json-continued branch January 10, 2020 08:25
Mpdreamz added a commit that referenced this pull request Jan 10, 2020
…rialize. (#4313)

* Several improvements to PostData

New types

* `ReadOnlyMemory`, only available in `netstandard2.1` allows you to
post `ReadOnlyMemory<byte>`
* `StreamHandler` allows you to inject a stream handler and write to the
request streams the best way you see fit yourself

Performance

* `PostData.String` now reuses bytes
* `PostData.MultiJson(string[]) uses way less allocations

* Adds more write overloads to Utf8Json

Either memory stream or to serialize on the writer.

This consolidates and funnels places that need to write
bytes/streams/objects directly

(cherry picked from commit de62914)

* add missing hunk after git add -p session

(cherry picked from commit fc289b1)

* add comment explaining the check for possible fast path in sourceformatter

(cherry picked from commit ffc4e46)

* Move fastpath check into method and use it for Deserialize too

(cherry picked from commit f365b30)

* update formatters to use a better check for fast path

(cherry picked from commit b07e0a8)

* Move StreamHandler in to its own type.

Expose reusable stream handling on `PostData`

Add tests for new types so that they behave and write as expected

* Zap IInternalSerializerWithFormatter

* Make sure WriteRaw for memory streams assumes a memorycopy

* Make sure final newline is written again for bulk requests

* Address PR feedback

* reoorder variable assignments

(cherry picked from commit fadef66)
Mpdreamz added a commit that referenced this pull request Jan 10, 2020
* Several improvements to PostData (#4312)

* Several improvements to PostData

New types

* `ReadOnlyMemory`, only available in `netstandard2.1` allows you to
post `ReadOnlyMemory<byte>`
* `StreamHandler` allows you to inject a stream handler and write to the
request streams the best way you see fit yourself

Performance

* `PostData.String` now reuses bytes
* `PostData.MultiJson(string[]) uses way less allocations

* Move StreamHandler in to its own type.

Expose reusable stream handling on `PostData`

Add tests for new types so that they behave and write as expected

* update post-data docs

* Refactor SerializableData and reuse methods from PostData

(cherry picked from commit af93854)

* Additional UTF8Json overloads, consolidate how we tell it to write/serialize. (#4313)

* Several improvements to PostData

New types

* `ReadOnlyMemory`, only available in `netstandard2.1` allows you to
post `ReadOnlyMemory<byte>`
* `StreamHandler` allows you to inject a stream handler and write to the
request streams the best way you see fit yourself

Performance

* `PostData.String` now reuses bytes
* `PostData.MultiJson(string[]) uses way less allocations

* Adds more write overloads to Utf8Json

Either memory stream or to serialize on the writer.

This consolidates and funnels places that need to write
bytes/streams/objects directly

(cherry picked from commit de62914)

* add missing hunk after git add -p session

(cherry picked from commit fc289b1)

* add comment explaining the check for possible fast path in sourceformatter

(cherry picked from commit ffc4e46)

* Move fastpath check into method and use it for Deserialize too

(cherry picked from commit f365b30)

* update formatters to use a better check for fast path

(cherry picked from commit b07e0a8)

* Move StreamHandler in to its own type.

Expose reusable stream handling on `PostData`

Add tests for new types so that they behave and write as expected

* Zap IInternalSerializerWithFormatter

* Make sure WriteRaw for memory streams assumes a memorycopy

* Make sure final newline is written again for bulk requests

* Address PR feedback

* reoorder variable assignments

(cherry picked from commit fadef66)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants