pubsub订阅者出错：超出最大邮件大小(Error in pubsub subscriber: Max message size exceeded)

pubsub订阅者出错：超出最大邮件大小(Error in pubsub subscriber: Max message size exceeded)

我正在使用Google Cloud Pubsub作为我的应用程序。 pubsub主题的订阅者使用Javascript编写并在Nodej上运行。 我正在使用谷歌提供的官方pubsub客户端。

代码如下所示：

var topic = gcloud.pubsub({projectId: 'myProjectId'}).topic('topicName'); var pubsub = gcloud.pubsub({projectId: 'myProjectId'}); var sub = pubsub.subscription('subName', {topic: topic}); sub.on('error', function(err) { console.error(err); }); sub.on('message', messageHandler);

我的错误处理程序不断触发此错误消息：

Error: Max message size exceeded

首先，这是在订户上接收的非常奇怪的消息。 如果邮件太大，则在发布时应该被拒绝。

更重要的是，我的订户似乎在某个时刻似乎已经死亡。 它一直处理这些消息，直到大约80个错误然后停止。 永远不会再次调用消息处理程序。

我怎样才能解决这个问题？

将SDK版本更新为0.46.1。 新的错误消息：

Received message larger than max (10406691 vs. 4194304)

I am using Google Cloud Pubsub for my application. The subscriber to the pubsub topic is written in Javascript and runs on Nodejs. I am using the official pubsub client provided by google.

The code looks like this:

var topic = gcloud.pubsub({projectId: 'myProjectId'}).topic('topicName'); var pubsub = gcloud.pubsub({projectId: 'myProjectId'}); var sub = pubsub.subscription('subName', {topic: topic}); sub.on('error', function(err) { console.error(err); }); sub.on('message', messageHandler);

My error handler keeps getting triggered with this error message:

Error: Max message size exceeded

First of all, this is a very strange message to be receiving on the subscriber. If the message is too large, it should have been rejected when it was published.

Much more importantly, is that my subscriber just seems to die at some point. It keeps processing these messages until about 80 errors and then just stops. The message handler never gets called again.

How can I fix this?

Updated SDK version to 0.46.1. New error message:

Received message larger than max (10406691 vs. 4194304)

最满意答案

这可能是Node.js Pub / Sub客户端库中的一个错误，其中没有在gRPC通道上设置max_receive_message_length属性。 在这种情况下，“消息大小”不是指发布到Pub / Sub的消息的大小（根据最大发布请求大小可以达到~10MB），而是指消息中允许的消息大小。响应发送到gRPC中的请求。 我建议在Google Cloud Platform Node.js存储库中创建一个问题。

在此期间，您可以尝试将订阅上的maxInProgress属性设置为较小的数字，这应该强制订阅者提取较小批量的消息，这应该使响应更小。 目前，如果未指定，单个响应中的最大邮件数默认为1000.但是，如果您的邮件大小很大，这可能没有帮助。

This is likely a bug in the Node.js Pub/Sub client library where the max_receive_message_length property is not being set on the gRPC channel. The "message size" in this case does not refer to the size of a message published to Pub/Sub (which can be up to ~10MB based on the maximum publish request size), but rather to the size of the message allowed in a response sent to a request in gRPC. I would recommend creating an issue in the Google Cloud Platform Node.js repository.

In the meantime, you could try setting the maxInProgress property on the subscription to a smaller number, which should force the subscriber to pull smaller batches of messages, which should make the response smaller. Currently, if not specified, the maximum number of messages in a single response defaults to 1000. However, if your messages are large in size, this may not help.