-
Notifications
You must be signed in to change notification settings - Fork 1k
MySQL 8.3 will introduce new GTID format #845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
An example event from
The event is serialized with the new The The UUID of the GTID is encoded as A >>> print(f'{0xaaaaaaaa888866664444222222222222 >> 1:0x}');
55555555444433332222111111111111 The checksum here is According to
|
The Examples: MySQL 9.1
Note the
Note the MySQL 8.0
Note the
Note the |
A first attempt in decoding this. Some info on this is available here: https://github.com/mysql/mysql-server/blob/61a3a1d8ef15512396b4c2af46e922a19bf2b174/sql/rpl_gtid_set.cc#L1364 package main
import (
"encoding/binary"
"fmt"
)
func main() {
cases := []struct {
b []byte
}{
{
[]byte{1, 2, 0, 0, 0, 0, 0, 1}, // tagged, sidnr=2
},
{
[]byte{1, 1, 0, 0, 0, 0, 0, 1}, // tagged, sidnr=1
},
{
[]byte{1, 0, 0, 0, 0, 0, 0, 1}, // tagged, sidnr=0
},
{
[]byte{0, 0, 0, 0, 0, 0, 0, 0}, // untagged, sidnr=0
},
{
[]byte{1, 0, 0, 0, 0, 0, 0, 0}, // untagged, sidnr=1
},
}
for _, c := range cases {
fmt.Printf("\nraw bytes= %08b\n", c.b)
if c.b[7] == 1 {
fmt.Println("tagged gtid format")
sid_mask := []byte{0, 255, 255, 255, 255, 255, 255, 0}
// Apply the mask
for i, _ := range c.b {
c.b[i] &= sid_mask[i]
}
c.b = append(c.b, 0)
// sidnr, encoded
fmt.Printf("sidnr = %08b\n", c.b[1:])
// sidnr
n := binary.LittleEndian.Uint64(c.b[1:])
fmt.Printf("sidnr = %d\n", n)
} else {
fmt.Println("classic gtid format")
n := binary.LittleEndian.Uint64(c.b)
fmt.Printf("sidnr = %d\n", n)
}
}
} |
Issue: closes go-mysql-org#845 The `PreviousGTIDsEvent` / `PREVIOUS_GTIDS_LOG_EVENT` has changed to work with tagged GTIDs. First the `uuidCount` has changed, it encodes the GTID format. Here format 1 is tagged and format 0 is untagged. Then each entry may have a tag. If there is a tag then the uuid itself isn't printed but the tag is appended to the last entry. Examples: - `896e7882-18fe-11ef-ab88-22222d34d411:1-3` regular format, compatible with both formats - `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1` tagged format. Combination of `896e7882-18fe-11ef-ab88-22222d34d411:1-4` and `896e7882-18fe-11ef-ab88-22222d34d411:aaaa:1` - `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1:abc:1-3:bbbbb:1:bbbbbb:1:x:1,896e7882-18fe-11ef-ab88-22222d34d412:1-2` Combination of: ``` 896e7882-18fe-11ef-ab88-22222d34d411:1-4 :aaaa:1 🔤1-3 :bbbbb:1 :bbbbbb:1 ❌1, 896e7882-18fe-11ef-ab88-22222d34d412:1-2 ``` Please also see: `mysqlbinlog --read-from-remote-server --hexdump $binlogfile` to see how MySQL encodes/decodes this. See also: - https://dev.mysql.com/doc/refman/8.4/en/replication-gtids-concepts.html
Issue: closes go-mysql-org#845 The `PreviousGTIDsEvent` / `PREVIOUS_GTIDS_LOG_EVENT` has changed to work with tagged GTIDs. First the `uuidCount` has changed, it encodes the GTID format. Here format 1 is tagged and format 0 is untagged. Then each entry may have a tag. If there is a tag then the uuid itself isn't printed but the tag is appended to the last entry. Examples: `896e7882-18fe-11ef-ab88-22222d34d411:1-3` regular format, compatible with both formats `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1` tagged format. Combination of - `896e7882-18fe-11ef-ab88-22222d34d411:1-4` - `896e7882-18fe-11ef-ab88-22222d34d411:aaaa:1` `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1:abc:1-3:bbbbb:1:bbbbbb:1:x:1,896e7882-18fe-11ef-ab88-22222d34d412:1-2` Combination of: ``` 896e7882-18fe-11ef-ab88-22222d34d411:1-4 :aaaa:1 🔤1-3 :bbbbb:1 :bbbbbb:1 ❌1, 896e7882-18fe-11ef-ab88-22222d34d412:1-2 ``` Please also see: `mysqlbinlog --read-from-remote-server --hexdump $binlogfile` to see how MySQL encodes/decodes this. See also: - https://dev.mysql.com/doc/refman/8.4/en/replication-gtids-concepts.html
Issue: ref go-mysql-org#845 The `PreviousGTIDsEvent` / `PREVIOUS_GTIDS_LOG_EVENT` has changed to work with tagged GTIDs. First the `uuidCount` has changed, it encodes the GTID format. Here format 1 is tagged and format 0 is untagged. Then each entry may have a tag. If there is a tag then the uuid itself isn't printed but the tag is appended to the last entry. Examples: `896e7882-18fe-11ef-ab88-22222d34d411:1-3` regular format, compatible with both formats `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1` tagged format. Combination of - `896e7882-18fe-11ef-ab88-22222d34d411:1-4` - `896e7882-18fe-11ef-ab88-22222d34d411:aaaa:1` `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1:abc:1-3:bbbbb:1:bbbbbb:1:x:1,896e7882-18fe-11ef-ab88-22222d34d412:1-2` Combination of: ``` 896e7882-18fe-11ef-ab88-22222d34d411:1-4 :aaaa:1 🔤1-3 :bbbbb:1 :bbbbbb:1 ❌1, 896e7882-18fe-11ef-ab88-22222d34d412:1-2 ``` Please also see: `mysqlbinlog --read-from-remote-server --hexdump $binlogfile` to see how MySQL encodes/decodes this. See also: - https://dev.mysql.com/doc/refman/8.4/en/replication-gtids-concepts.html
* replication: Support GTID tag in PreviousGTIDsEvent Issue: ref #845 The `PreviousGTIDsEvent` / `PREVIOUS_GTIDS_LOG_EVENT` has changed to work with tagged GTIDs. First the `uuidCount` has changed, it encodes the GTID format. Here format 1 is tagged and format 0 is untagged. Then each entry may have a tag. If there is a tag then the uuid itself isn't printed but the tag is appended to the last entry. Examples: `896e7882-18fe-11ef-ab88-22222d34d411:1-3` regular format, compatible with both formats `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1` tagged format. Combination of - `896e7882-18fe-11ef-ab88-22222d34d411:1-4` - `896e7882-18fe-11ef-ab88-22222d34d411:aaaa:1` `896e7882-18fe-11ef-ab88-22222d34d411:1-4:aaaa:1:abc:1-3:bbbbb:1:bbbbbb:1:x:1,896e7882-18fe-11ef-ab88-22222d34d412:1-2` Combination of: ``` 896e7882-18fe-11ef-ab88-22222d34d411:1-4 :aaaa:1 🔤1-3 :bbbbb:1 :bbbbbb:1 ❌1, 896e7882-18fe-11ef-ab88-22222d34d412:1-2 ``` Please also see: `mysqlbinlog --read-from-remote-server --hexdump $binlogfile` to see how MySQL encodes/decodes this. See also: - https://dev.mysql.com/doc/refman/8.4/en/replication-gtids-concepts.html * Add test * Add more tests * Update replication/event.go Co-authored-by: lance6716 <[email protected]> * Update based on review * add my suggestions Signed-off-by: lance6716 <[email protected]> * add test for long tag Signed-off-by: lance6716 <[email protected]> --------- Signed-off-by: lance6716 <[email protected]> Co-authored-by: lance6716 <[email protected]>
One step of progress:
package main
import (
"encoding/hex"
"fmt"
"strings"
)
// Example Event:
// # at 158
// #250211 17:53:57 server id 1 end_log_pos 240 CRC32 0x4031f7db
// # Position Timestamp Type Source ID Size Source Pos Flags
// # 0000009e a5 80 ab 67 2a 01 00 00 00 52 00 00 00 f0 00 00 00 00 00
// # 000000b1 02 76 00 00 02 02 25 02 dc f0 09 02 30 f9 03 22 |.v..........0...|
// # 000000c1 bd 03 ad 02 21 02 44 44 5a 68 51 03 22 04 04 06 |......DDZhQ.....|
// # 000000d1 0c 61 61 62 62 63 63 08 00 0a 04 0c 7f be e9 c3 |.aabbcc.........|
// # 000000e1 ab e0 2d 06 10 39 03 12 c3 02 0b db f7 31 40 |.....9.......1.|
// # GTID last_committed=0 sequence_number=1 rbr_only=no original_committed_timestamp=1739292837931454 immediate_commit_timestamp=1739292837931454 transaction_length=206
// # original_commit_timestamp=1739292837931454 (2025-02-11 17:53:57.931454 CET)
// # immediate_commit_timestamp=1739292837931454 (2025-02-11 17:53:57.931454 CET)
// /*!80001 SET @@session.original_commit_timestamp=1739292837931454*//*!*/;
// /*!80014 SET @@session.original_server_version=90200*//*!*/;
// /*!80014 SET @@session.immediate_server_version=90200*//*!*/;
// SET @@SESSION.GTID_NEXT= '896e7882-18fe-11ef-ab88-22222d34d411:aabbcc:1'/*!*/;
// In this event:
// 25 02 dc f0 09 02 30 f9 03 22 bd 03 ad 02 21 02 44 44 5a 68 51 03 22
// is the serialized version of this
// 896e7882-18fe-11ef-ab88-22222d34d411
func main() {
// UUID encoded as part of a MySQL GTID with a tag
// As used in Binlog Event Type: 0x23
uuid := "896e7882-18fe-11ef-ab88-22222d34d411"
uuid = strings.ReplaceAll(uuid, "-", "")
fmt.Println(uuid)
v2, _ := hex.DecodeString(uuid)
fmt.Printf("\n%s\t%s\n", "Decoded", "Encoded")
for _, c := range v2 {
fmt.Printf("%x", c)
if c < 0x80 {
fmt.Printf("\t%02x\n", c*2)
} else if c < 0xc0 {
v := c - 0x80
fmt.Printf("\t%02x 02\n", (v<<2)+1)
} else if c <= 0xff {
v := c - 0xc0
fmt.Printf("\t%02x 03\n", (v<<2)+1)
} else {
fmt.Printf("\n")
}
}
} Output:
(See also https://dev.mysql.com/doc/dev/mysql-server/9.2.0/PageLibsMysqlSerialization.html ) This is the event body: (excluding header and checksum) (so 59 bytes)
The event body has:
|
The fields are defined here: With this I think I now have the format: Event:
Body:
Decoded format:
|
For the number encoding:
|
|
https://dev.mysql.com/doc/relnotes/mysql/8.3/en/news-8-3-0.html
thanks for @dveeden to remind us
PreviousGTIDsEvent
replication: Support GTID tag in PreviousGTIDsEvent #952GTID_TAGGED_LOG_EVENT
mysql.ParseGTIDSet(mysql.MySQLFlavor, ...)
/mysql.ParseMysqlGTIDSet()
The text was updated successfully, but these errors were encountered: