Airflow k8s操作员XCOM

编程入门 行业动态 更新时间:2024-10-25 00:27:31
本文介绍了Airflow k8s操作员XCOM-禁止握手状态403的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧! 问题描述

当我在Airflow版本1.10中使用 KubernetesPodOperator 运行docker映像时

When I run a docker image using KubernetesPodOperator in Airflow version 1.10

一旦Pod成功完成任务,气流就会尝试通过k8s流客户端与Pod建立连接来获取xcom值.

Once the pod finishes the task successfullly, airflow tries to get the xcom value by making a connection to the pod via k8s stream client.

以下是我遇到的错误:

[2018-12-18 05:29:02,209] {{models.py:1760}} ERROR - (0) Reason: Handshake status 403 Forbidden Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py", line 249, in websocket_call client = WSClient(configuration, get_websocket_url(url), headers) File "/usr/local/lib/python3.6/site-packages/kubernetes/stream/ws_client.py", line 72, in __init__ self.sock.connect(url, header=header) File "/usr/local/lib/python3.6/site-packages/websocket/_core.py", line 223, in connect self.handshake_response = handshake(self.sock, *addrs, **options) File "/usr/local/lib/python3.6/site-packages/websocket/_handshake.py", line 79, in handshake status, resp = _get_resp_headers(sock) File "/usr/local/lib/python3.6/site-packages/websocket/_handshake.py", line 152, in _get_resp_headers raise WebSocketBadStatusException("Handshake status %d %s", status, status_message) websocket._exceptions.WebSocketBadStatusException: Handshake status 403 Forbidden

我正在为此使用K8s服务帐户

I'm using K8s service account for this

DAG配置

xcom = true,

xcom=true,

get_logs = True,

get_logs=True,

in_cluster = true

in_cluster=true

推荐答案

所以我们也遇到了这个问题,我们不得不修改rbac规则,特别是必须添加动词"create"的资源"pods/exec" "和获取"

So we also hit this problem, we had to modify our rbac rules, in particular we had to add the resource "pods/exec" with the verbs "create" and "get"

--- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: airflow-runner rules: - apiGroups: [""] resources: ["deployments", "pods", "pods/log", "pods/exec", "persistentvolumeclaims"] verbs: ["*"] - apiGroups: [""] resources: ["secrets"] resourceNames: ["singleuser-image-credentials"] verbs: ["read","list","watch","create","get"]

更多推荐

Airflow k8s操作员XCOM

本文发布于:2023-11-24 03:23:21,感谢您对本站的认可!
本文链接:https://www.elefans.com/category/jswz/34/1623793.html
版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系,我们将在24小时内删除。
本文标签:操作员   Airflow   k8s   XCOM

发布评论

评论列表 (有 0 条评论)
草根站长

>www.elefans.com

编程频道|电子爱好者 - 技术资讯及电子产品介绍!