Kubernetes上的多经纪人Kafka如何设置KAFKA

本文介绍了Kubernetes上的多经纪人Kafka如何设置KAFKA_ADVERTISED_HOST_NAME的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我当前拥有3个Kafka经纪人的Kafka部署文件如下:

My current Kafka deployment file with 3 Kafka brokers looks like this:

apiVersion: apps/v1beta1 kind: StatefulSet metadata: name: kafka spec: selector: matchLabels: app: kafka serviceName: kafka-headless replicas: 3 updateStrategy: type: RollingUpdate podManagementPolicy: Parallel template: metadata: labels: app: kafka spec: containers: - name: kafka-instance image: wurstmeister/kafka ports: - containerPort: 9092 env: - name: KAFKA_ADVERTISED_PORT value: "9092" - name: KAFKA_ADVERTISED_HOST_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: KAFKA_ZOOKEEPER_CONNECT value: "zookeeper-0.zookeeper-headless.default.svc.cluster.local:2181,\ zookeeper-1.zookeeper-headless.default.svc.cluster.local:2181,\ zookeeper-2.zookeeper-headless.default.svc.cluster.local:2181" - name: BROKER_ID_COMMAND value: "hostname | awk -F '-' '{print $2}'" - name: KAFKA_CREATE_TOPICS value: hello:2:1 volumeMounts: - name: data mountPath: /var/lib/kafka/data volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 50Gi

这将创建3个Kafka代理作为有状态集，并使用带有FQDN(完全合格域名)的Kubedns服务连接到Zookeeper集群，例如:

This creates 3 Kafka brokers as a Stateful Set and connects to the Zookeeper cluster using the Kubedns service with FQDN (Fully Qualified Domain Names) such as:

zookeeper-0.zookeeper-headless.default.svc.cluster.local:2181

经纪人ID是根据广告连播名称生成的:

Broker IDs are generated based on the pod name:

- name: BROKER_ID_COMMAND value: "hostname | awk -F '-' '{print $2}'"

结果:

kafka-0 = 0 kafka-1 = 1 kafka-2 = 2

但是，为了对Kafka经纪人使用Kubedns名称，

However, In order to use the Kubedns names for the Kafka brokers:

kafka-0.kafka-headless.default.svc.cluster.local:9092 kafka-1.kafka-headless.default.svc.cluster.local:9092 kafka-2.kafka-headless.default.svc.cluster.local:9092

我需要能够根据pod的名称将KAFKA_ADVERTISED_HOST_NAME变量设置为上述FQDN值.

I need to be able to set the KAFKA_ADVERTISED_HOST_NAME variable to the above FQDN values based on the name of the pod.

目前，我已将变量设置为广告连播的名称:

Currently I have the variable set to the name of the pod:

- name: KAFKA_ADVERTISED_HOST_NAME valueFrom: fieldRef: fieldPath: metadata.name

结果:

KAFKA_ADVERTISED_HOST_NAME=kafka-0 KAFKA_ADVERTISED_HOST_NAME=kafka-1 KAFKA_ADVERTISED_HOST_NAME=kafka-2

但是以某种方式，我需要附加其余的DNS名称.

But somehow I would need to append the rest of the DNS name.

有没有一种方法可以直接设置DNS值?

Is there a way I could set the DNS value directly?

类似的东西:

- name: KAFKA_ADVERTISED_HOST_NAME valueFrom: fieldRef: fieldPath: kubedns.name

推荐答案

我设法通过pod定义内的命令字段解决了这个问题:

I managed to solve the problem with a command field inside the pod definition:

command: - sh - -c - "export KAFKA_ADVERTISED_HOST_NAME=$(hostname).kafka-headless.default.svc.cluster.local && start-kafka.sh"

这将运行一个shell命令，该命令根据hostname值导出公告的主机名环境变量.

This runs a shell command which exports the advertised hostname environment variable based on the hostname value.