Apache Airflow不会腌制DAG

本文介绍了Apache Airflow不会腌制DAG的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我想恢复DAG对象，以便在DAG运行后更好地检查某些依赖项（例如，特定操作员消耗了哪些数据）。我正在使用 postgres：9.6 作为元数据数据库后端。

I would like to recover DAG objects so that I can better inspect certain dependencies after DAG runs (e.g. what data is consumed by specific operators). I am using postgres:9.6 as metadata database backend.

这似乎受 donot_pickle 配置变量，默认情况下表示必须腌制所有DAG：

This seems to be supported via the donot_pickle configuration variable, which by default indicates all DAGs must be pickled:

[core] # Whether to disable pickling dags donot_pickle = False

我进行了一些测试DAG（3）可用，但其相应的 pickle_id 为空：

I have some test DAGs (3) available but their corresponding pickle_id is empty:

> select pickle_id from dag; pickle_id --------- (3 rows)

泡菜表也为空：

> select count(*) from dag_pickle; count ------ 0 (1 row)

这里可能出了什么问题？我在文档中找不到任何引用。

What might be going wrong here? I was not able to find any reference in the docs.

推荐答案

有两种启用酸洗的方法：

There are 2 ways to enable pickling:

• DONT_PICKLE = False 仅与回填作业有关
调度程序命令行参数中的
• -p，-do_pickle 启用对计划作业的酸洗（ airflow.apache/cli.html#Named%20Arguments_repeat18 ）

• DONT_PICKLE=False in scheduler config is only relevant for backfill jobs
• -p, -do_pickle in scheduler command line arguments enables pickling for scheduled jobs (airflow.apache/cli.html#Named%20Arguments_repeat18)