在Kafka，Fluentd和Logstash中，哪种服务器日志记录解决方案的处理器消耗较少？(Which server logging solution is less processor cons

在Kafka，Fluentd和Logstash中，哪种服务器日志记录解决方案的处理器消耗较少？(Which server logging solution is less processor consuming among Kafka, Fluentd and Logstash?)

我有一个分布式日志系统来监控负载平衡的服务器实体。 对我来说，基本的是，服务器不会在日志记录过程中投入大量处理器时间，从而允许应用程序以尽可能多的资源运行。

很高兴知道，哪些替代品在处理器时间方面“更便宜”，或者在这种情况下推荐任何其他解决方案。

I have a distributed logging system to monitor load-balanced server entities. It is basic for me, that the server does not invest a lot of processor time in the logging process, allowing the application to run with the maximum resources possible.

It would be nice to know, which of those alternatives is "cheaper" in terms of processor time or, in case, to be recommended any other solution for this matter.

最满意答案

首先，Kafka 不是日志收集器。 它是一个分布式消息队列，可以作为消费者和生产者与Fluentd和Logstash等日志收集器一起使用。

我们不是提出意见，而是提出一些数据。

CPU使用率：这取决于您使用Fluentd和/或Logstash在客户端进行的过滤和数据处理量。 如果您进行最少的处理，两者都可以每秒处理10,000多条消息。 两者都可以利用多个CPU（例如http://docs.fluentd.org/articles/in_multiprocess ） 内存：Fluentd使用大约40MB的内存，Logstash使用大约100MB的内存。 如果这太多了，Logstash有Beats和Fluentd作为Fluentd Forwarders（ https://github.com/fluent/fluentd-forwarder ）。

First of all, Kafka is not a log collector. It's a distributed message queue and can work with log collectors like Fluentd and Logstash both as consumers and producers.

Instead of opinions, let's put some numbers.

CPU Usage: This depends on how much filtering and data processing you do on the client-side using Fluentd and/or Logstash. If you do minimal processing, both can process 10,000+ messages per second. Both can take advantage of multiple CPUs (http://docs.fluentd.org/articles/in_multiprocess, for example) Memory: Fluentd uses about 40MB of memory and Logstash uses about 100MB of memory. If this is too much, Logstash has Beats and Fluentd as Fluentd Forwarders (https://github.com/fluent/fluentd-forwarder).