Kafka：Confluent的HDFS连接器是否支持Snappy压缩？

本文介绍了Kafka：Confluent的HDFS连接器是否支持Snappy压缩？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我没有在HDFS连接器上看到任何用于压缩的配置文件docs docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html 。

I don't see any configurations for compression on the HDFS connector docs docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html.

• 它支持压缩吗？
• 如果是，我需要在属性文件中添加什么？

推荐答案

Snappy压缩最近添加到了用于Avro的HDFS连接器中。要启用它，您需要将属性 avro.codec 设置为 snappy 。

Snappy compression was recently added to the HDFS Connector for Avro. To enable it you'll need to set the property avro.codec to snappy.

Parquet自一开始就可用，它是导出Parquet文件时使用的编解码器。

With Parquet it's been available since the beginning and it is the codec used when exporting parquet files.

以下是为Avro添加它的PR：

Here's the PR that added it for Avro:

> github/confluentinc/kafka-connect-hdfs/pull/255

此配置属性将在即将发布的连接器4.1.0版本中提供。这也是指向文档链接的更新，以反映此添加。

This config property will become available in the forthcoming 4.1.0 release of the connector. That's also when the link to the docs will be updated to reflect this addition.