问题描述
限时送ChatGPT账号..这是我学习 kafka 的早期阶段.我正在检查本地机器中的每个 kafka 属性/概念.
所以我遇到了这个属性min.insync.replicas
,这是我的理解.如果我误解了什么,请纠正我.
min.insync.replicas
数量的关注者.min.insync.replicas
还包括领导者.如果可用的 live brokers(indirectly, in sync replicas )少于指定的 min.insync.replicas
,那么生产者将引发无法发布的异常消息.以下是我创建上述场景所遵循的步骤
在本地启动了 3 个代理,代理 ID 为 0、1 和 2创建主题 insync 并将min.insync.replicas
设置为 2使用以下命令sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2
<块引用>
主题:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2主题:insync 分区:0 Leader:2 Replicas:2,0,1 Isr:1,2,0
此时,我确保我提供的属性是由 kafka 选择的
我开始使用以下命令从终端发送消息并使用它们
生产者:./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties
消费者:./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync
此时,我能够成功发送和接收消息.
购买了 2 个经纪人(0 和 2)并描述了主题并导致了以下内容<块引用>
主题:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2主题:insync 分区:0 Leader:1 Replicas:2,0,1 Isr:1
此时,In Sync Replicas 只有 1(Isr: 1)
然后我尝试生成消息并且它起作用了.我能够从控制台生产者发送消息,我可以在控制台消费者中看到这些消息.
我的 Kafka 版本:kafka_2.10-0.10.0.0
以下是生产者属性:
<块引用>bootstrap.servers=localhost:9092
compression.type=none
batch.size=20
acks=all
我预计生产者会以 NotEnoughReplicasException
失败,如 this.
公共类NotEnoughReplicasException扩展 RetriableException
分区的同步副本数低于>min.insync.replicas
但它工作正常.
我错过了什么吗?如何创建场景?
*************** 编辑 **********************>
我没有从控制台生成器生成消息,而是尝试从 Java 代码生成消息.这一次,我在 kafka broker 中得到了预期的异常.虽然我在生产者(java代码)中期望它.由于这个实验提出了更多问题,我发布了另一个问题一>.
解决方案acks 是否设置为全部"?如果没有,请尝试将其设置为全部
Its my early days in learning kafka. And I am checking out every kafka property/concept in my local machine.
So I came across this property min.insync.replicas
and here is my understanding. Please correct me if I've misunderstood anything.
min.insync.replicas
number of followers.
min.insync.replicas
also includes the leader.
If number of available live brokers( indirectly, in sync replicas ) are less than the specified min.insync.replicas
, then producer will raise an exception failing to publish the message.
Following are the steps I followed to create the above scenario
Started 3 brokers in local with broker Ids 0, 1 and 2 created the topic insync and setmin.insync.replicas
to 2
using the following command
sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2
Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0
At this point, I made sure the property I've provided is picked by kafka
I started sending messages and consuming them from terminal using following command
Producer: ./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties
Consumer: ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync
At this point, I was able to send and receive messages successfully.
Bought down 2 brokers (0 and 2) and described the topic and resulted in following
Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 1 Replicas: 2,0,1 Isr: 1
At this point, the In Sync Replicas are just 1(Isr: 1)
Then I tried to produce the message and it worked. I was able to send messages from console-producer and I could see those messages in console consumer.
My Kafka version: kafka_2.10-0.10.0.0
following are the producer properties:
bootstrap.servers=localhost:9092
compression.type=none
batch.size=20
acks=all
I expected the producer to fail with NotEnoughReplicasException
as mentioned in this.
public class NotEnoughReplicasException extends RetriableException
Number of insync replicas for the partition is lower than >min.insync.replicas
but it worked normally.
Am I missing something? How can I create the scenario?
*************** EDIT **********************
Instead of producing the messages from console producer, I tried to generate messages from java code. This time, I got the expected exception in the kafka broker. Although I expected it in the producer (java code). As this experiment is raising more questions, I've posted another question.
解决方案is acks set to "all"? if not, try setting it to all
这篇关于Kafka 配置 min.insync.replicas 不起作用的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
更多推荐
[db:关键词]
发布评论