适用于 Javascript v3 S3Client GetObjectCommand 的 AWS SDK 给出错误:文件意外结束"/>
适用于 Javascript v3 S3Client GetObjectCommand 的 AWS SDK 给出错误:文件意外结束
在我的 AWS Lambda 函数中,我有这段代码将 S3 存储桶中的 zip 文件的内容提取到本地文件系统中。它使用 nodejs 流来避免首先下载整个 zip 文件,以节省内存:
const streamUnzipFromS3ToLocal = async (
objectBucket: string,
objectKey: string,
extractDir: string,
) => {
logger.info(`Start to stream unzip ${objectBucket}:${objectKey} to local.`);
var start = new Date().getTime();
try {
const data = await S3.getFile(objectBucket, objectKey); // data.Body is a readable stream;
data.Body.on('error', (error) => {
logger.error(`Error while reading the stream: ${error}`);
console.log(error);
throw error;
});
await pipeline(data.Body, unzipper.Extract({ path: extractDir }));
logger.info(`${objectKey} is decompressed to ${extractDir}`);
} catch (error) {
logger.error(`Failed to stream unzip file ${objectKey}: ${error}`);
console.log(error);
throw error
}
var end = new Date().getTime();
var time = (end - start) / 1000;
logger.info(`Finished stream-unzipping ${objectKey} to local. execution time: ${time} sec;`);
return time;
};
和
S3.getFile
如下:
export const getFile = async (bucket_name, object_key) => {
let res;
const params = {
Bucket: bucket_name,
Key: object_key,
};
try {
const x = new GetObjectCommand(params);
const data = await s3Client.send(new GetObjectCommand(params));
res = data;
} catch (err) {
logger.error(`Failed when s3Controller getFile: ${JSON.stringify(err)}`);
throw err;
}
return res;
};
使用上述代码在 S3 存储桶中流解压缩一个非常大的文件时出现此错误:
Failed to stream unzip file myapp/a/s/d/2GBfile.zip: Error: unexpected end of file
和
Error while reading the stream: Error: aborted
怎么回事?
我该如何解决这个问题?
回答如下:更多推荐
适用于 Javascript v3 S3Client GetObjectCommand 的 AWS SDK 给出错误:文件意外结束
发布评论