Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We listen to the following binlog events:
For updates and deletes we use ALTER TABLE command. We cannot use mempool to optimize.
The main issue seems to be CH is not able to handle all the update events. CH after applying mutations, before marking them as finished, checks if there are other pending mutations. In our case, mutations are constantly arriving so CH always find new pending mutations so never marks the applied mutations as finished. The number of mutations increases flooding of files the HDD and, if using replication, collapsing the zookeeper because it has too many entries on the mutations znode.
We use this approach for the BeBanjo PoC and it works, during a time, because didn't use replication. After a couple of weeks, we start having issues with the mutation files.
We have discarded this approach in favor of using inserts for updates and deletes and handling mutations the same way ITX does.