Google Cloud Composer和Google Cloud SQL代理

本文介绍了Google Cloud Composer和Google Cloud SQL代理的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我有一个使用Cloud Composer和Cloud SQL的项目. 我能够连接到Cloud SQL，因为我编辑了airflow-sqlproxy-service的Yaml，并将我的Cloud SQL实例添加到用于airflow-db的云代理上，映射到端口3307.

I have a project with Cloud Composer and Cloud SQL. I am able to connect to Cloud SQL because i edited the yaml of airflow-sqlproxy-service and added my Cloud SQL instance on cloud proxy used for the airflow-db, mapping to port 3307.

工作人员可以在端口3307上连接到airflow-sqlproxy-service，但我认为Web服务器无法连接到此端口.

The workers can connect to airflow-sqlproxy-service on port 3307 but i think the webserver can't connect to this.

我是否需要添加一些防火墙规则以映射3307端口，以便Web服务器或UI可以连接到airflow-sqlproxy-service?

Do i need to add some firewall rule to map the 3307 port so the webserver or the UI can connect to airflow-sqlproxy-service?

i.stack.imgur/LwKQK.png

i.stack.imgur/CJf7Q.png

i.stack.imgur/oC2dJ.png

最诚挚的问候.

推荐答案

Composer当前不支持从Web服务器配置其他sql代理.此类情况的一种解决方法是拥有一个单独的DAG，该DAG将从另一个数据库(通过有权有权访问的工作人员)向所需的信息中加载气流变量，然后根据该变量生成DAG， Web服务器可以访问.

Composer does not currently support configuring additional sql proxies from the webserver. One workaround for cases like this is to have a separate DAG which loads Airflow Variables with the information needed from the other database (via the workers which do have access), then generate a DAG based on the Variable which the webserver can access.

github/apache/incubator-airflow/pull/4170 最近被合并(在Composer中当前不可用)，它定义了CloudSQL连接类型.将来这可能会适用于这些用例.

github/apache/incubator-airflow/pull/4170 recently got merged (not currently available in Composer), which defines a CloudSQL connection type. This might work for these use cases in the future.