如何绕过 Amazon Aurora 1MB 数据 API 限制?

编程入门 行业动态 更新时间:2024-10-05 13:19:21

如何绕过 Amazon Aurora 1MB <a href=https://www.elefans.com/category/jswz/34/1771445.html style=数据 API 限制?"/>

如何绕过 Amazon Aurora 1MB 数据 API 限制?

我正在努力从 Amazon Aurora Postgres DB 检索商店列表,我需要完整的列表。

我有超过 20.000 家商店,但由于 Amazon Aurora 数据 API 的 1MB 限制而失败..

我收到的消息:


{
    "error": "INTERNAL_SERVER_ERROR",
    "message": "select \"e0\".*, \"e1\".\"id\" as \"location_id\" from \"shop\" as \"e0\" left join \"location\" as \"e1\" on \"e0\".\"id\" = \"e1\".\"shop_id\" where \"e0\".\"name\" is not null and \"e0\".\"online\" = false and \"e0\".\"status\" = 'active' and \"e1\".\"formatted_address\" is not null and \"e1\".\"latitude\" is not null and \"e1\".\"longitude\" is not null order by \"e0\".\"uuid\" asc limit 10000 - Database returned more than the allowed response size limit"
}

I use mikro-orm and heres sample of my code:

    @Get()
    @Middleware(
        validateQueryParams(
            Joi.object({
                offset: Joi.number().integer().raw().min(0),
                limit: Joi.number().integer().raw().min(0).max(ShopRepository.MaxResults),
            })
        )
    )
    async list(req: Request, res: Response, next: NextFunction) {
        logger.trace('ShopLocationController::list() - Enter');

        const { limit, offset } = parseFixedLengthCollectionParams(req.query);

        // Connect to all jurisdictional databases and retrieve their shop locations
        const databases = Adapter.allJurisdictionalDataApis();
        const results: ShopSetResult[] = await Promise.all(
            databases.map(async database => {
                await database.connect();
                const shopRepository = database.em.getRepository(Shop) as ShopRepository;
                const [shops, total] = await shopRepository.findHavingLocation(0, limit);
                return { shops, total };
            })
        );
        const completeShopsInJurisdictionBlocks = results.map(
            (shopset: ShopSetResult) => shopset.shops
        );
        const completeShopSet = completeShopsInJurisdictionBlocks.flat();

        // Calculate the total by adding all the totals from all the jurisdictions together
        const total = results.reduce((runningTotal, shopset) => (runningTotal += shopset.total), 0);

        // Sort shops and apply offsets and limits
        completeShopSet.sort((a, b) => (a.uuid > b.uuid ? 1 : -1));
        const shopsAfterOffsetAndLimit = completeShopSet.slice(
            offset,
            limit ? offset + limit : undefined
        );

        const list = toCollection(shopsAfterOffsetAndLimit, total, offset);
        return jsonSuccess(res, list);
    }

任何人都可以根据我的代码提供示例以某种方式对其进行分页但最后返回完整列表吗?

回答如下:

更多推荐

如何绕过 Amazon Aurora 1MB 数据 API 限制?

本文发布于:2024-05-30 11:56:23,感谢您对本站的认可!
本文链接:https://www.elefans.com/category/jswz/34/1770471.html
版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系,我们将在24小时内删除。
本文标签:数据   Aurora   Amazon   API   MB

发布评论

评论列表 (有 0 条评论)
草根站长

>www.elefans.com

编程频道|电子爱好者 - 技术资讯及电子产品介绍!