面向服务的体系结构

面向服务的体系结构--AMQP或HTTP(Service Oriented Architecture - AMQP or HTTP)

一点背景。

非常大的单片Django应用程序。 所有组件都使用相同的数据库。 我们需要分离服务，以便我们可以独立升级系统的某些部分而不影响其余部分。

我们使用RabbitMQ作为Celery的经纪人。

现在我们有两个选择：

使用REST接口的HTTP服务。 JSONRPC通过AMQP传递给事件循环服务

我的团队倾向于HTTP，因为这是他们所熟悉的，但我认为使用RPC而非AMQP的优势远大于它。

AMQP为我们提供了轻松添加负载平衡和高可用性的功能，并保证消息传递。

而使用HTTP，我们必须创建客户端HTTP包装器以使用REST接口，我们必须放入负载平衡器并设置该基础架构才能拥有HA等。

使用AMQP，我可以产生另一个服务实例，它将连接到与其他实例相同的队列，并且实现高可用性，负载平衡。

我对AMQP有什么想法吗？

A little background.

Very big monolithic Django application. All components use the same database. We need to separate services so we can independently upgrade some parts of the system without affecting the rest.

We use RabbitMQ as a broker to Celery.

Right now we have two options:

HTTP Services using a REST interface. JSONRPC over AMQP to a event loop service

My team is leaning towards HTTP because that's what they are familiar with but I think the advantages of using RPC over AMQP far outweigh it.

AMQP provides us with the capabilities to easily add in load balancing, and high availability, with guaranteed message deliveries.

Whereas with HTTP we have to create client HTTP wrappers to work with the REST interfaces, we have to put in a load balancer and set up that infrastructure in order to have HA etc.

With AMQP I can just spawn another instance of the service, it will connect to the same queue as the other instances and bam, HA and load balancing.

Am I missing something with my thoughts on AMQP?

最满意答案

首先，

REST，RPC - 体系结构模式，AMQP - 在TCP / IP之上运行的有线和HTTP应用协议 AMQP是HTTP - 通用协议时的一个特定协议，因此，与AMQP相比，HTTP具有很高的开销 在HTTP本质是同步的情况下，AMQP本质是异步的 REST和RPC都使用数据序列化，该格式取决于您的基础设施。 如果你在任何地方使用python，我认为你可以使用python本地序列化 - pickle ，它应该比JSON或任何其他格式更快。 HTTP + REST和AMQP + RPC都可以在异构和/或分布式环境中运行

因此，如果您选择使用什么：HTTP + REST或AMQP + RPC，则答案确实是基础结构复杂性和资源使用的主题。 没有任何特定的要求，两种解决方案都可以正常工作，但我宁愿做一些抽象以便能够透明地在它们之间切换。

你告诉你的团队熟悉HTTP但不熟悉AMQP。 如果开发时间是你获得答案的重要时间。

如果你想以最小的复杂度构建高可用性基础架构，我想AMQP协议就是你想要的。

我对他们两人都有一定的经验，而且RESTful服务的优势在于：

他们很好地映射在Web界面上 人们熟悉它们 易于调试（由于HTTP的通用目的） 轻松为第三方服务提供API。

基于AMQP解决方案的优势：

该死的快 灵活 易于维护 易于扩展 成本效益（资源使用含义）

请注意，您可以在基于AMQP的API上为RESTful API提供第三方服务，而REST不是协议而是范例，但您应该考虑构建您的AQMP RPC api。 我这样做是为了向外部第三方服务提供API，并为那些在旧代码库上运行的基础架构部分或无法添加AMQP支持的部分提供API访问。

如果我是对的，你的问题是如何更好地组织软件不同部分之间的通信，而不是如何为最终用户提供API。

如果你有一个高负载的项目RabbitMQ是一个很好的软件，你可以很容易地添加任何数量的工作在不同的机器上运行。 它也具有镜像和集群功能。 还有一点，RabbitMQ建立在Erlang OTP的基础之上，这是一个高可靠性，稳定的平台...（bla-bla-bla），不仅适用于市场营销，也适用于工程师。 当nginx日志将所有磁盘空间放在RabbitMQ运行的同一分区上时，我只有一次RabbitMQ的问题。

At first,

REST, RPC - architecture patterns, AMQP - wire-level and HTTP - application protocol which run on top of TCP/IP AMQP is a specific protocol when HTTP - general-purpose protocol, thus, HTTP has damn high overhead comparing to AMQP AMQP nature is asynchronous where HTTP nature is synchronous both REST and RPC use data serialization, which format is up to you and it depends of infrastructure. If you are using python everywhere I think you can use python native serialization - pickle which should be faster than JSON or any other formats. both HTTP+REST and AMQP+RPC can run in heterogeneous and/or distributed environment

So if you are choosing what to use: HTTP+REST or AMQP+RPC, the answer is really subject of infrastructure complexity and resource usage. Without any specific requirements both solution will work fine, but i would rather make some abstraction to be able switch between them transparently.

You told that your team familiar with HTTP but not with AMQP. If development time is an important time you got an answer.

If you want to build HA infrastructure with minimal complexity I guess AMQP protocol is what you want.

I had an experience with both of them and advantages of RESTful services are:

they well-mapped on web interface people are familiar with them easy to debug (due to general purpose of HTTP) easy provide API to third-party services.

Advantages of AMQP-based solution:

damn fast flexible cost-effective (in resources usage meaning)

Note, that you can provide RESTful API to third-party services on top of your AMQP-based API while REST is not a protocol but rather paradigm, but you should think about it building your AQMP RPC api. I have done it in this way to provide API to external third-party services and provide access to API on those part of infrastructure which run on old codebase or where it is not possible to add AMQP support.

If I am right your question is about how to better organize communication between different parts of your software, not how to provide an API to end-users.

If you have a high-load project RabbitMQ is damn good piece of software and you can easily add any number of workers which run on different machines. Also it has mirroring and clustering out of the box. And one more thing, RabbitMQ is build on top of Erlang OTP, which is high-reliable,stable platform ... (bla-bla-bla), it is good not only for marketing but for engineers too. I had an issue with RabbitMQ only once when nginx logs took all disc space on the same partition where RabbitMQ run.

UPD (May 2018): Saurabh Bhoomkar posted a link to the MQ vs. HTTP article written by Arnold Shoon on June 7th, 2012, here's a copy of it:

I was going through my old files and came across my notes on MQ and thought I’d share some reasons to use MQ vs. HTTP:

If your consumer processes at a fixed rate (i.e. can’t handle floods to the HTTP server [bursts]) then using MQ provides the flexibility for the service to buffer the other requests vs. bogging it down. Time independent processing and messaging exchange patterns — if the thread is performing a fire-and-forget, then MQ is better suited for that pattern vs. HTTP. Long-lived processes are better suited for MQ as you can send a request and have a seperate thread listening for responses (note WS-Addressing allows HTTP to process in this manner but requires both endpoints to support that capability). Loose coupling where one process can continue to do work even if the other process is not available vs. HTTP having to retry. Request prioritization where more important messages can jump to the front of the queue. XA transactions – MQ is fully XA compliant – HTTP is not. Fault tolerance – MQ messages survive server or network failures – HTTP does not. MQ provides for ‘assured’ delivery of messages once and only once, http does not. MQ provides the ability to do message segmentation and message grouping for large messages – HTTP does not have that ability as it treats each transaction seperately. MQ provides a pub/sub interface where-as HTTP is point-to-point.

UPD (Dec 2018): As noticed by @Kevin in comments below, it's questionable that RabbitMQ scales better then RESTful servies. My original answer was based on simply adding more workers, which is just a part of scaling and as long as single AMQP broker capacity not exceeded, it is true, though after that it requires more advanced techniques like Highly Available (Mirrored) Queues which makes both HTTP and AMQP-based services have some non-trivial complexity to scale at infrastructure level.

After careful thinking I also removed that maintaining AMQP broker (RabbitMQ) is simpler than any HTTP server: original answer was written in Jun 2013 and a lot of changed since that time, but the main change was that I get more insight in both of approaches, so the best I can say now that "your mileage may vary".

Also note, that comparing both HTTP and AMQP is apple to oranges to some extent, so please, do not interpret this answer as the ultimate guidance to base your decision on but rather take it as one of sources or as a reference for your further researches to find out what exact solution will match your particular case.