-
Notifications
You must be signed in to change notification settings - Fork 185
Performance improvement on codec mapping #409
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
type: enhancement
A general enhancement
Milestone
Comments
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
May 21, 2021
…decoding performances. * Switched the codec list to a thread safe variant to avoid the synchronized blocks. Even though `CopyOnWriteArrayList` is not super performant it should work fine in this context where the list should not be frequently updated. * Switched `mockito-core` to `mockito-junit-jupiter` for Junit 5 support
Thanks a lot for your suggestion. I had something like this on my mind since iterations aren't really efficient to determine a codec. |
@mp911de Interested to work in this project. Are you accepting new contributors ? |
Sure. Feel free to pick an issue, submit a pull request or participate in issue discussions. |
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 1, 2021
…decoding performances. * Switched the codec list to a thread safe variant to avoid the synchronized blocks. Even though `CopyOnWriteArrayList` is not super performant it should work fine in this context where the list should not be frequently updated. * Switched `mockito-core` to `mockito-junit-jupiter` for Junit 5 support
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 4, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis.
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 7, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 7, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 7, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not * Enable unixDomainSocketTest IT only when running on Linux
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 7, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not * Disabled unixDomainSocketTest IT when running on Mac or Windows
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 8, 2021
…decoding performances. * Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Switched the codec list to a thread safe variant to avoid the synchronized blocks. Even though `CopyOnWriteArrayList` is not super performant it should work fine in this context where the list should not be frequently updated. * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not * Switched `mockito-core` to `mockito-junit-jupiter` for Junit 5 support * Disabled unixDomainSocketTest IT when running on Mac or Windows
4 tasks
frdeboffles
pushed a commit
to frdeboffles/r2dbc-postgresql
that referenced
this issue
Sep 8, 2021
* Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not * Disabled unixDomainSocketTest IT when running on Mac or Windows
mp911de
added a commit
that referenced
this issue
Sep 22, 2021
Rename CodecFinder to CodecLookup. Rename default implementations to CachedCodecLookup and DefaultCodecLookup. Extract CodecMetadata interface and turn getFormats() into a default method. Refactor how CodecLookup obtains its actual codecs to prevent methods allowing to alter the internal state of the cache component through updateCodecs(…). The delegate is typically a CodecRegistry for iteration over the actual codecs. Reinstate socket tests on MacOS as sockets are supported on BSD via kqueue. Remove overly complex spy arrangements from tests. Refine tests. [resolves #410][#409] Signed-off-by: Mark Paluch <[email protected]>
mp911de
pushed a commit
that referenced
this issue
Sep 22, 2021
…rmances. * Switched the codec list to a thread safe variant to avoid the synchronized blocks. Even though `CopyOnWriteArrayList` is not super performant it should work fine in this context where the list should not be frequently updated. * Switched `mockito-core` to `mockito-junit-jupiter` for Junit 5 support * Refactored the codec registry to use a CodecFinder (default to SPI definition in the classpath) * Provided 2 implementations of the codec finder, one without cache and another with cache * Added a build cache method that will attempt to fill the cache when the codecs are updated. This cannot covers all the cases like the nested arrays, therefore for those type the cache will be filled dynamically on per-request basis * Added microbenchmarks for codec encode and decode using the cache based implementation or not * Disabled unixDomainSocketTest IT when running on Windows [#444][resolves #409]
mp911de
added a commit
that referenced
this issue
Sep 22, 2021
Rename CodecFinder to CodecLookup. Rename default implementations to CachedCodecLookup and DefaultCodecLookup. Extract CodecMetadata interface and turn getFormats() into a default method. Refactor how CodecLookup obtains its actual codecs to prevent methods allowing to alter the internal state of the cache component through updateCodecs(…). The delegate is typically a CodecRegistry for iteration over the actual codecs. Reinstate socket tests on MacOS as sockets are supported on BSD via kqueue. Remove overly complex spy arrangements from tests. Refine tests. [resolves #444][#409] Signed-off-by: Mark Paluch <[email protected]>
4 tasks
mp911de
added a commit
that referenced
this issue
Sep 22, 2021
Fix benchmarks after polishing. [#444][#409] Signed-off-by: Mark Paluch <[email protected]>
mp911de
added a commit
that referenced
this issue
Sep 23, 2021
Use constructor delegation in DefaultCodecs. [#444][#409] Signed-off-by: Mark Paluch <[email protected]>
mp911de
added a commit
that referenced
this issue
Sep 23, 2021
Fix benchmarks after polishing. [#444][#409] Signed-off-by: Mark Paluch <[email protected]>
mp911de
added a commit
that referenced
this issue
Sep 23, 2021
Use constructor delegation in DefaultCodecs. [#444][#409] Signed-off-by: Mark Paluch <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Feature Request
While using this driver for fetching many rows from our table I noticed it was taking quite some time to build the
Row
objects. I hooked a profiler to the code and noticed it was spending quite some time for each row on thecodec.canDecode
method.I wrote this little perf test to validate this finding:
Running this on my machine results in:
Is your feature request related to a problem? Please describe
I would rather use this driver and webflux than going the web mvc route and jdbc.
Describe the solution you'd like
I think the class
DefaultCodecs
would benefits from some simple caching mechanism.I did implement a simple caching (PR to come) and the results of the above test comes to:
Describe alternatives you've considered
I haven't found alternatives
Teachability, Documentation, Adoption, Migration Strategy
NA
The text was updated successfully, but these errors were encountered: