我想将数据从BigQuery中的表格传输到另一个Redshift中。 我的计划数据流如下：
BigQuery - > Google Cloud Storage - > Amazon S3 - > Redshift
我了解Google Cloud Storage Transfer Service，但我不确定它可以帮助我。来自Google Cloud文档： $ b
云端存储转移服务
此页面描述了云存储传输服务，您可以使用快速将在线数据导入Google云端存储。
<我知道这项服务可用于将数据导入到Google云端存储中，而不是从中导出数据。
有没有一种方法可以将数据从Google Cloud中导出存储到Amazon S3？
解决方案您可以使用gsutil将数据从Google Cloud Storage存储桶复制到Amazon存储桶，命令如：
gsutil -m rsync -rd gs：// your-gcs-bucket s3：// your-s3 -bucket请注意，上面的-d选项会导致gsutil rsync删除S3存储桶中的对象，不存在于您的GCS存储桶中（除了添加新对象外）。如果您只想从GCS添加新对象到您的S3存储桶，您可以忽略该选项。
I would like to transfer data from a table in BigQuery, into another one in Redshift. My planned data flow is as follows:
BigQuery -> Google Cloud Storage -> Amazon S3 -> Redshift
I know about Google Cloud Storage Transfer Service, but I'm not sure it can help me. From Google Cloud documentation:
Cloud Storage Transfer Service
This page describes Cloud Storage Transfer Service, which you can use to quickly import online data into Google Cloud Storage.
I understand that this service can be used to import data into Google Cloud Storage and not to export from it.
Is there a way I can export data from Google Cloud Storage to Amazon S3?
解决方案You can use gsutil to copy data from a Google Cloud Storage bucket to an Amazon bucket, using a command such as:
gsutil -m rsync -rd gs://your-gcs-bucket s3://your-s3-bucketNote that the -d option above will cause gsutil rsync to delete objects from your S3 bucket that aren't present in your GCS bucket (in addition to adding new objects). You can leave off that option if you just want to add new objects from your GCS to your S3 bucket.
更多推荐
将数据从Google Cloud Storage导出到Amazon S3
发布评论