1 year ago

#372870

test-img

Scourrge

Kafka Mirrormaker 2.0 slow recovery after connection loss

we have a problem with slow recovery of Kafka Mirrormaker 2.0 after connection between two clusters was terminated and reconnected again.

We have 2 kafka clusters A and B and mirroring is done bidirectionally A <-> B.

Kafka Mirrormaker 2.0 is running on the same network as Cluster A. When connection between clusters is severed and re-established again, mirroring from B -> A comes back fairly quickly. In the matter of seconds. But mirroring from A -> B comes back really slowly. About 5 to 10 minutes which is a problem for us.

My guess is that this is because Mirrormaker runs on the same network as Cluster A and therefore consumer has connection to Cluster A all the time but producer doesn't have connection to Cluster B all the time. And when message is read from Cluster A and cannot be produced to Cluster B there is where problem lies.

Does anybody have any idea how to setup Mirrormaker 2.0 so it doesn't take so long to re-establish mirroring between local cluster to remote cluster?

apache-kafka

aws-msk

apache-kafka-mirrormaker

0 Answers

Your Answer

Accepted video resources