Zero-pad fixed-length binary fields #401
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MySQL is zero-padding fixed-length binary fields [1], but is not storing trailing zeros to the binlog. Consequently, when reading values of fixed-length binary fields, the value must be padded with zeros up to the specified length. The `test_fixed_length_binary` test case exemplifies the issue. `varbinary` columns are not padded and not affected. This commit extends the `information_schema` query in `__get_table_information` to fetch information about the length limitation of the field and stores this information in the `fixed_binary_length` attribute of `Column` instances. Upon decoding of binary fields (which share the same binlog field type value of `254` as strings), the value is zero padded at the end, when it does not meet the specified length as retrieved from the information schema. Fixes julien-duponchelle#400. [1] https://dev.mysql.com/doc/refman/5.7/en/binary-varbinary.html
|
Thanks I 'm going to merge that ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
MySQL is zero-padding fixed-length binary fields [1], but is not storing trailing zeros to the binlog. Consequently, when reading values of fixed-length binary fields, the value must be padded with zeros up to the specified length.
The
test_fixed_length_binarytest case exemplifies the issue.varbinarycolumns are not padded and not affected.This commit extends the
information_schemaquery in__get_table_informationto fetch information about the length limitation of the field and stores this information in thefixed_binary_lengthattribute ofColumninstances.Upon decoding of binary fields (which share the same binlog field type value of
254as strings), the value is zero padded at the end, when it does not meet the specified length as retrieved from the information schema.I've been running a fork with the fix against our fleet of ~100 mysql servers (5.7 and 8.0) for over a week now, with countless transactions. There are no indications of problems and the trailing whitespace problem is indeed fixed. Most frequent case are UUID fields that are stored as
binary(16)and where 1 out of ~256 records would miss the trailing zero. Those cases are all fixed with this patch.Fixes #400.
[1] https://dev.mysql.com/doc/refman/5.7/en/binary-varbinary.html