如何使用python将本地CSV上传到Google大查询

本文介绍了如何使用python将本地CSV上传到Google大查询的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我正在尝试使用python将本地CSV上传到Google大查询

I'm trying to upload a local CSV to google big query using python

def uploadCsvToGbq(self,table_name): load_config = { 'destinationTable': { 'projectId': self.project_id, 'datasetId': self.dataset_id, 'tableId': table_name } } load_config['schema'] = { 'fields': [ {'name':'full_name', 'type':'STRING'}, {'name':'age', 'type':'INTEGER'}, ] } load_config['sourceFormat'] = 'CSV' upload = MediaFileUpload('sample.csv', mimetype='application/octet-stream', # This enables resumable uploads. resumable=True) start = time.time() job_id = 'job_%d' % start # Create the job. result = bigquery.jobs.insert( projectId=self.project_id, body={ 'jobReference': { 'jobId': job_id }, 'configuration': { 'load': load_config } }, media_body=upload).execute() return result

当我运行此命令时，它会引发类似

when I run this it throws error like

"NameError:未定义全局名称'MediaFileUpload'"

"NameError: global name 'MediaFileUpload' is not defined"

是否需要任何模块，请帮助.

whether any module is needed please help.

推荐答案

pip install --upgrade google-api-python-client

然后在您的python文件顶部写入:

Then on top of your python file write:

from googleapiclient.http import MediaFileUpload

但是请注意，您会错过一些括号.更好地写:

But care you miss some parenthesis. Better write:

result = bigquery.jobs().insert(projectId=PROJECT_ID, body={'jobReference': {'jobId': job_id},'configuration': {'load': load_config}}, media_body=upload).execute(num_retries=5)

顺便说一句，您将上载所有CSV行，包括定义列的第一行.

And by the way, you are going to upload all your CSV rows, including the top one that defines columns.