fix - Bug/increase history list length (#431)

Overview

issue number #430

When using the pymysqlreplication package, there was a recurring issue with the history list length value under the following circumstances:

    MySQL version 8.x or higher.
    Continuous execution of DML (Data Manipulation Language) statements.

This problem led to a gradual slowing down of SELECT queries.

For detailed information, you can refer to the following reference:

https://minervadb.xyz/troubleshooting-innodb-history-length-with-hung-mysql-transaction/

As a result, I investigated the problem area and made necessary modifications. After making the changes, you were able to compare the performance before and after the fix, using monitoring tools such as Prometheus and MySQL Exporter.

---------

Co-authored-by: davinc71998 <[email protected]>
Co-authored-by: sean <[email protected]>

Author: 3 people

Diff for: pymysqlreplication/binlogstream.py

@@ -256,6 +256,7 @@ def __connect_to_ctl(self):
             self._ctl_connection_settings = dict(self.__connection_settings)
         self._ctl_connection_settings["db"] = "information_schema"
         self._ctl_connection_settings["cursorclass"] = DictCursor
+        self._ctl_connection_settings["autocommit"] = True
         self._ctl_connection = self.pymysql_wrapper(**self._ctl_connection_settings)
         self._ctl_connection._get_table_information = self.__get_table_information
         self.__connected_ctl = True
@@ -653,8 +654,10 @@ def __get_table_information(self, schema, table):
                         table_schema = %s AND table_name = %s
                     ORDER BY ORDINAL_POSITION
                     """, (schema, table))
+                result = cur.fetchall()
+                cur.close()
 
-                return cur.fetchall()
+                return result
             except pymysql.OperationalError as error:
                 code, message = error.args
                 if code in MYSQL_EXPECTED_ERROR_CODES: