Skip to content

parse gtid for parseHandler if MySQL works in GTID_MODE, and begin to startWithGTID after mysqldump is done #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Dec 7, 2019

Conversation

jianhaiqing
Copy link
Contributor

@jianhaiqing jianhaiqing commented Oct 24, 2019

Purpose of This PR:

  1. if MySQL is GTID_MODE=ON, canal would be start to replicate with GTID after mysqldump is done.
  2. Add one more func GtidSet() for parseHandler, to save gtid set after parsing gtid from mysqldump.

@siddontang
Copy link
Collaborator

PTAL @GregoryIan

@siddontang
Copy link
Collaborator

CI failed @jianhaiqing

@jianhaiqing
Copy link
Contributor Author

More details about GTID between MySQL and MariaDB.
mysqldump --set-gtid-purged from MySQL is not compatible with mysqldump of MariaDB, while MariaDB use --gtid instead

 [MariaDB]
 mysqldump --host=`hostname -i` --port=3306 --user=root --password=my-secret-pw --master-data --gtid --single-transaction --skip-lock-tables --compact --skip-opt --quick --no-create-info --skip-extended-insert --skip-tz-utc --hex-blob --default-character-set=utf8 longrun t6

-- CHANGE MASTER TO MASTER_LOG_FILE='mariadb-bin.000001', MASTER_LOG_POS=1138;
CHANGE MASTER TO MASTER_USE_GTID=slave_pos;
SET GLOBAL gtid_slave_pos='0-1-4';
INSERT INTO `t6` VALUES (1,11);
INSERT INTO `t6` VALUES (2,11);

[MySQL]
mysqldump --host=`hostname -i` --port=23306 --user=root --password=my-secret-pw --master-data --set-gtid-purged --single-transaction --skip-lock-tables --compact --skip-opt --quick --no-create-info --skip-extended-insert --skip-tz-utc --hex-blob --default-character-set=utf8 longrun t6

SET @MYSQLDUMP_TEMP_LOG_BIN = @@SESSION.SQL_LOG_BIN;
SET @@SESSION.SQL_LOG_BIN= 0;
SET @@GLOBAL.GTID_PURGED='5906b63d-4009-11e9-81f3-00505690245b:1-144473,
abc8b882-b168-11e8-a790-1866daabb9a4:1-3439091213';
CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000258', MASTER_LOG_POS=4372;
INSERT INTO `t6` VALUES (1,11);
INSERT INTO `t6` VALUES (2,11);
INSERT INTO `t6` VALUES (3,11);
SET @@SESSION.SQL_LOG_BIN = @MYSQLDUMP_TEMP_LOG_BIN;

@IANTHEREAL
Copy link
Collaborator

@WangXiangUSTC PTAL

@IANTHEREAL
Copy link
Collaborator

I'm not quite understand the purpose of this pr, do you want to automatically replicate binlog data starting from gtid if gtid is turned on for both the source database and the target data?

@jianhaiqing
Copy link
Contributor Author

I'm not quite understand the purpose of this pr, do you want to automatically replicate binlog data starting from gtid if gtid is turned on for both the source database and the target data?

Yes, if replicate start with GTID, it would be easier to continue to replicate after master-slave switch occurs.

@jianhaiqing jianhaiqing closed this Dec 3, 2019
@jianhaiqing
Copy link
Contributor Author

As you know, there might be SLB in front of master or slave, so gtid replication would be better.

@jianhaiqing jianhaiqing reopened this Dec 3, 2019
@jianhaiqing
Copy link
Contributor Author

Purpose of This PR:

  1. if MySQL is GTID_MODE=ON, canal would be start to replicate with GTID after mysqldump is done.
  2. Add one more func GtidSet() for parseHandler, to save gtid set after parsing gtid from mysqldump.

@jianhaiqing jianhaiqing changed the title add gtid parsed from mysqldump output when --set-gtid-purged is set parse gtid for parseHandler if MySQL works in GTID_MODE, and begin to startWithGTID after mysqldump is done Dec 4, 2019
@WangXiangUSTC
Copy link
Contributor

Purpose of This PR:

  1. if MySQL is GTID_MODE=ON, canal would be start to replicate with GTID after mysqldump is done.
  2. Add one more func GtidSet() for parseHandler, to save gtid set after parsing gtid from mysqldump.

can update thease description in the first comment

@WangXiangUSTC
Copy link
Contributor

LGTM

Copy link
Collaborator

@siddontang siddontang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

@siddontang siddontang merged commit 08dd4b5 into go-mysql-org:master Dec 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants