卡夫卡胶印管理

本文介绍了卡夫卡胶印管理的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我们正在使用Kafka 0.10 ...我在网上(和文档中)看到一些有冲突的信息，这些信息涉及当enable.automit为TRUE时，如何在kafka中管理偏移量.检索消息的相同poll()方法是否也按配置的时间间隔处理提交?

We are using Kafka 0.10... I'm seeing some conflicting information online (and in documentation) regarding how offsets are managed in kafka when enable.automit is TRUE. Does the same poll() method that retrieves messages also handle the commits at the configured intervals?

如果我在单线程应用程序中从民意调查中检索消息，请在SAME线程中处理该消息以完成操作(包括处理错误)，这意味着直到处理完成后才会再次调用poll()，然后我认为是不是害怕丢失消息，对吗?仅当poll()在随后的调用中尝试提交时才有效(当然，如果automit.interval.ms已通过).如果提交是在收到消息后立即完成的(在我的应用处理消息之前)，这将对我们不起作用....

If i retrieve messages from poll in a single threaded application, process the messages to completion (including handling errors) in the SAME thread, meaning poll() will not be invoked again until after my processing is complete, then I presume there is no fear in losing messages, correct? This only works if poll() attempts the commit at the subsequent invocation (if the automit.interval.ms has passed, of course). If the commits are done immediately upon receiving the messages (prior to my app processing the messages), this will not work for us....

这很重要，因为我想确定的是，如果我们使用自动提交策略，则不会丢失消息.我们可以容忍重复的消息，但我们不能容忍丢失的数据.

This is important, as I want to be certain we won't lose messages if we use the automatic commit policy. Duplicate messages are tolerable for us, we just have no tolerance for lost data.

感谢您的澄清！

推荐答案

检索消息的相同poll()方法是否也按配置的时间间隔处理提交?

Does the same poll() method that retrieves messages also handle the commits at the configured intervals?

是的. (如果为enable.automit=true.)

如果我在单线程应用程序中从民意调查中检索消息，请在SAME线程中处理该消息以完成操作(包括处理错误)，这意味着直到处理完成后才会再次调用poll()，然后我认为是不是害怕丢失消息，对吗?

If i retrieve messages from poll in a single threaded application, process the messages to completion (including handling errors) in the SAME thread, meaning poll() will not be invoked again until after my processing is complete, then I presume there is no fear in losing messages, correct?

是的

仅当poll()在随后的调用中尝试提交(当然，如果automit.interval.ms已经通过)时，这才起作用

This only works if poll() attempts the commit at the subsequent invocation (if the automit.interval.ms has passed, of course)

这就是它的完成方式.

有关更多详细信息，请参见此处: docs.confluent.io/current /clients/consumer.html

See here for further details: docs.confluent.io/current/clients/consumer.html