如何运行简单的气流DAG

本文介绍了如何运行简单的气流DAG的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我对Airflow完全陌生。我想在指定的日期运行一个简单的DAG。我正在努力使开始日期，执行日期和回填之间有所不同。而运行DAG的命令是什么？

I am totally new to Airflow. I would like to run a simple DAG at a specified date. I'm struggling to make difference between the start date, the execution date, and backfilling. And what is the command to run the DAG?

这是我从那以后尝试过的命令：

Here is what I've tried since:

airflow run dag_1 task_1 2017-1-23

第一次运行该命令，任务正确执行，但是当我再次尝试时，它不起作用。

The first time I ran that command, the task executed correctly, but when I tried again it did not work.

这是我运行的另一个命令：

Here is another command I ran:

airflow backfill dag_1 -s 2017-1-23 -e 2017-1-24

我不知道该命令会带来什么。 DAG会每天从23到24执行吗？

I don't know what to expect from this command. Will the DAGs execute every day from 23 to 24?

在运行上述两个命令之前，我已经这样做了：

Before running the two commands above, I did this:

airflow initdb airflow scheduler airflow webserver -p 8085 --debug &

这是我的DAG

from airflow import DAG from airflow.operators.bash_operator import BashOperator from datetime import datetime, timedelta default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2017, 1, 23, 12), 'email': ['airflow@airflow'], 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5), } dag = DAG( 'dag_1', default_args=default_args, schedule_interval=timedelta(1)) t1 = BashOperator( task_id='create_clients', bash_command='Rscript /scripts/Cli.r', dag=dag) t2 = BashOperator( task_id='create_operation', bash_command='Rscript Operation.r', retries=3, dag=dag) t2.set_upstream(t1)

截屏：树状视图

更新 >

airflow run dag_1 task_1 2017-1-23T10:34

推荐答案

如果使用

airflow run dag_1 task_1 2017-1-23

运行已保存并正在运行它再也无能为力，您可以尝试通过强制使其重新运行

The run is saved and running it again won't do anything you can try to re-run it by forcing it

airflow run --force=true dag_1 task_1 2017-1-23

airflow backfill命令将运行当时将执行的所有执行从开始到结束日期指定的时间段。这将取决于您在DAG上设置的时间表，如果将其设置为每小时触发一次，则它应该运行24次，但它也不会重新执行以前执行的运行。

The airflow backfill command will run any executions that would have run in the time period specified from the start to end date. It will depend what schedule you set on the DAG, if you set it to trigger every hour it should run 24 times, but it also won't re-execute previously executed runs.

您可以清除任务，就像它从未运行过

You can clear the task as if it NEVER ran

airflow clear dag_1 -s 2017-1-23 -e 2017-1-24

也请在此处检查cli文档： airflow.incubator.apache/cli.html

Also check the cli docs here: airflow.incubator.apache/cli.html