卡夫卡查询

本文介绍了卡夫卡查询的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我有以下问题关于Kafkka。如果您能解决这些问题，我们是否会感激？

I have the following questions with regards to Kafkka. Appreciate if you can address them ?

1.我们是否支持HDInsight中的故障转移 Kafka经纪人？

1.Do we have the support for failover in HDInsight Kafka brokers?

2.在Kafka经纪人重启期间是否会有任何数据丢失。

2.Will there be any loss of data during a Kafka broker reboot.

3.Will HDInsight Kafka 自动更改领导者&如果代理发生故障，主题分区的副本配置将不会 是数据丢失？

3.Will HDInsight Kafka automatically change the leader & replica configuration for a topic partition if a broker goes down so that there won’t be data loss?

4.一旦领导者（经纪人节点）回来，它会自动恢复配置吗？

4.Will it automatically revive the configuration once the leader(broker node) is back?

推荐答案

嗨Raven，

Hi Raven,

HDInsight Kafka是100％开源的Kafka，所以打开的一切来源Kafka也支持我们的支持。

HDInsight Kafka is the 100% open source Kafka, so everything that open source Kafka supports we support as well.

在云环境中，代理可能由于各种原因（例如VM在主机上移动）或者由于内存不足而导致代理重启而失败（客户正在运行）一些占用资源的进程或者如果客户在小型虚拟机上运行太多分区等等。

In a cloud environment brokers can fail due to various reasons such as VMs moving across host machines, or simply broker restarts due to out of memory (customer running some processes which eat up resources or if the customer is running too many partitions on small VMs etc).

在HDInsight中，我们为客户提供了选择运行Kafka群集的灵活性。

In HDInsight we give customers the flexibility to choose how they want to run their Kafka cluster.

为避免代理重启期间数据丢失是我们的建议：

In order to avoid data loss during broker restarts here are our recommendations:

将min.insync.replicas设置为3

在生产者中设置ACKS = all

Unclean.leader.election.enabled = false

每个代理不超过4000个分区。

避免重新启动代理进程

对于Kafka消息丢失情况的必要阅读，我建议

For required reading on Kafka message loss scenarios I would recommend

jack-vanlightly/blog/2018/9/14/how-to-lose-messages- on-a-kafka-cluster-part1

jack-vanlightly/blog/2018/9/18/how-to-lose-messages-on-a-kafka-cluster-part -2

希望这会有所帮助。