1 year ago

#321073

test-img

Pankaj Yelulkar

Mirror-maker 2 is not copying old data

I have 2 MSK kafka clusters A and B . I want to copy data from A to B using MM2 . It copies new data but older data is not getting copied from a to B . MM2 configuration file -


A.bootstrap.servers=<A broker url>
B.bootstrap.servers=<B broker url>

A->B.enabled=true
topics = .*
groups = .*
A.config.storage.replication.factor=3
B.config.storage.replication.factor=3

A.offset.storage.replication.factor=3
B.offset.storage.replication.factor=3

A.status.storage.replication.factor=3
B.status.storage.replication.factor=3

offset-syncs.topic.replication.factor=3
heartbeats.topic.replication.factor=3
checkpoints.topic.replication.factor=3


replication.policy.class = org.apache.kafka.connect.mirror.IdentityReplicationPolicy
tasks.max=10
replication.factor=3
refresh.topics.enabled=true
sync.topic.configs.enabled=true
refresh.topics.interval.seconds=20

#topics.blacklist=.*[\-\.]internal, .*\.replica, __consumer_offsets
#groups.blacklist=console-consumer-.*, connect-.*, __.*
A->B.emit.heartbeats.enabled=true
A->B.emit.checkpoints.enabled=true
B->A.emit.heartbeats.enabled=true
B->A.emit.checkpoints.enabled=true

sync.group.offsets.enabled=true
sync.group.offsets.interval.seconds=60


B.security.protocol=SSL
B.ssl.truststore.location=/home/ubuntu/tls/ap-southeast-2-ci-stg/kafka.client.truststore.jks
B.ssl.keystore.location=/home/ubuntu/tls/ap-southeast-2-ci-stg/kafka.client.keystore.jks
B.ssl.keystore.password=
B.ssl.key.password=

Can somebody help on it ?

apache-kafka

aws-msk

apache-kafka-mirrormaker

0 Answers

Your Answer

Accepted video resources