从梁管道连接谷歌云 sql postgres 实例

本文介绍了从梁管道连接谷歌云 sql postgres 实例的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

限时送ChatGPT账号..

我想从在谷歌数据流上运行的 apache 光束管道连接谷歌云 sql postgres 实例.
我想使用 Python SDK 来做到这一点.
我无法为此找到适当的文档.
在云 SQL 中如何指导我没有看到任何有关数据流的文档.
https://cloud.google/sql/docs/postgres/

I want to connect google cloud sql postgres instance from apache beam pipeline running on google dataflow.
I want to do this using Python SDK.
I am not able to find proper documentation for this.
In cloud SQL how to guide I dont see any documentation for dataflow.
https://cloud.google/sql/docs/postgres/

谁能提供文档链接/github 示例?

Can someone provide documentation link/github example?

推荐答案

您可以使用 relational_db.Write 和 relational_db.Read 转换从 beam-nuggets 如下:

You can use the relational_db.Write and relational_db.Read transforms from beam-nuggets as follows:

首先安装beam-nuggests:

First install beam-nuggests:

pip install beam-nuggets

阅读:

import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
from beam_nuggets.io import relational_db

with beam.Pipeline(options=PipelineOptions()) as p:
    source_config = relational_db.SourceConfiguration(
        drivername='postgresql+pg8000',
        host='localhost',
        port=5432,
        username='postgres',
        password='password',
        database='calendar',
    )
    records = p | "Reading records from db" >> relational_db.Read(
        source_config=source_config,
        table_name='months',
    )
    records | 'Writing to stdout' >> beam.Map(print)

用于写作:

import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
from beam_nuggets.io import relational_db

with beam.Pipeline(options=PipelineOptions()) as p:
    months = p | "Reading month records" >> beam.Create([
        {'name': 'Jan', 'num': 1},
        {'name': 'Feb', 'num': 2},
    ])
    source_config = relational_db.SourceConfiguration(
        drivername='postgresql+pg8000',
        host='localhost',
        port=5432,
        username='postgres',
        password='password',
        database='calendar',
        create_if_missing=True,
    )
    table_config = relational_db.TableConfiguration(
        name='months',
        create_if_missing=True
    )
    months | 'Writing to DB' >> relational_db.Write(
        source_config=source_config,
        table_config=table_config
    )

这篇关于从梁管道连接谷歌云 sql postgres 实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！