Given a setup of 4 instances of a SpringBoot 2.x application with Kafka listeners.
- 2 instances are running in Azure
- 2 instances are running in a local DC.
The Kafka cluster is configured equally in all instances and is also running in the local DC. When messages arrive in a topic, they will only be processed by one of the app instances from the local DC. Only when the local app instances are stopped, the messages will be processed by one of the app instances in Azure.
Answer:
Your observation suggests a clear imbalance in Kafka consumer group behavior, favoring the local data center (DC) instances over the Azure instances, despite them being part of the same Spring Boot app and Kafka consumer group. The key question is: why do only local instances get partitions assigned and process messages?