本文介绍了Airflow:如何从其他服务器进行SSH和运行BashOperator的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
SSH到server1 启动Hive shell 运行Hive命令 SSH to server1
start Hive shell
run Hive command
是否可以使用SSH切换到其他服务器并使用Airbnb的Airflow运行BashOperator? 我正在尝试用Airflow运行一个配置单元sql命令,但是我需要SSH到另一个机器上才能运行配置单元外壳。 我的任务应如下所示:
谢谢!
解决方案我认为我已经弄清楚了:
创建一个用户界面中管理>连接下的SSH连接。注意:如果您重置数据库,连接将被删除
在Python文件中添加以下内容
导入SSHHook sshHook = SSHHook(conn_id =<您从UI中获得的连接ID>)
添加SSH操作员任务
t1 = SSHExecuteOperator( task_id = task1, bash_command =< YOUR COMMAND> ;, ssh_hook = sshHook, dag = dag)
谢谢!
Is there a way to ssh to different server and run BashOperator using Airbnb's Airflow? I am trying to run a hive sql command with Airflow but I need to SSH to a different box in order to run the hive shell. My tasks should look like this:
Thanks!
解决方案I think that I just figured it out:
Create a SSH connection in UI under Admin > Connection. Note: the connection will be deleted if you reset the database
In the Python file add the following
from airflow.contrib.hooks import SSHHook sshHook = SSHHook(conn_id=<YOUR CONNECTION ID FROM THE UI>)Add the SSH operator task
t1 = SSHExecuteOperator( task_id="task1", bash_command=<YOUR COMMAND>, ssh_hook=sshHook, dag=dag)Thanks!
更多推荐
Airflow:如何从其他服务器进行SSH和运行BashOperator
发布评论