了解Apache气流中的树状视图

本文介绍了了解Apache气流中的树状视图的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我从

解决方案

它们不是单独的实例。您可以看到以下内容：

在树状视图中，两个圆圈的开始/结束日期和持续时间将完全相同。

在甘特视图中，您只会看到 print_date 的单个实例的持续时间。

通常，您不能像复制节点一样将DAG映射到树形视图。

I setup the dag from the airflow.apache/tutorial.html as is, the only change being that I have set the dag to run at an interval of 5 minutes with a start date of 2017-12-17 T13:40:00 UTC. I enabled the dag before 13:40, so there was no backfill and my machine is running on UTC. The dag ran as expected(i.e at an interval of 5 minutes starting at 13:45 UTC)

Now, when I go to the tree view, I am failing to understand the graph. There are 3 tasks in total. 'sleep'(t2) has upstream set to 'printdate' (t1) and 'templated'(t3) too has upstream set to 'printdate'(t1). Then why is the graph showing two 'printdate's ?? Are they separate task instances of that task? If yes, then how do I make sure that only 1 task instance of t1 runs (diamond pattern). There are also 4 green rectangular boxes(with two 'printdate's), instead of 3.

# t1, t2 and t3 are examples of tasks created by instantiating operators t1 = BashOperator( task_id='print_date', bash_command='date', dag=dag) t2 = BashOperator( task_id='sleep', bash_command='sleep 5', retries=3, dag=dag) templated_command = """ {% for i in range(5) %} echo "{{ ds }}" echo "{{ macros.ds_add(ds, 7)}}" echo "{{ params.my_param }}" {% endfor %} """ t3 = BashOperator( task_id='templated', bash_command=templated_command, params={'my_param': 'Parameter I passed in'}, dag=dag) t2.set_upstream(t1) t3.set_upstream(t1)

Second, why is the time above the dag runs (green circles), showing 8.40, 8.45 - ? What time/timezone is that? I have set start_date for the dag to 13.40 and my machine set to UTC.

解决方案

They are not separate instances. You can see this:

In Tree View, the start/end dates and duration of both circles will be exactly the same.

In Gantt view, you will see the duration for only a single instance of print_date.

In general, you can't map a DAG to a tree view without duplicating nodes like they've done.