我们计划使用SEMrush API，它允许访问与域名和搜索关键字相关的SEO数据。 根据他们的使用条款 ，他们限制其使用以避免杀死他们的服务器：
您每秒执行的请求数不得超过10个，同时请求数不得超过2个。
我们将在PHP中构建一个简单的工具，该工具基于域名聚合数据，并且正在寻找有关如何满足该要求的基础知识。 我们正在计划数百/数千个潜在的同时用户。
也许有人可以在PHP中提供一些伪代码让我们这样做 - 或者它真的就像强制实际的API请求函数在每个命令之间休眠1秒一样简单吗？ 我没有很多API和大量并发用户的经验，所以任何帮助都表示赞赏。
We plan to use the SEMrush API, which allows access to SEO data relating to domain names and search keywords. Under their Terms of Use, they limit their usage to avoid killing their servers:
You may not perform more than 10 requests per second, nor more than 2 simultaneous requests.
We are going to be building a simple tool in PHP that aggregates data based on a domain name and are looking for the basics on how to fulfill that requirement. We are planning for hundreds/thousands of potential simultaneous users.
Maybe someone can provide some pseudo code in PHP that would let us do this - or is it really just as simple as forcing the actual API request function to sleep for 1 second in between each command? I don't have a lot of experience with APIs and large amounts of concurrent users so any help is appreciated.
最满意答案
PHP实际上不是用于并发编程的最佳语言。 但是，有一些第三方解决方案可以与PHP一起使用，可以帮助您实现目标。
您需要的是一个可以处理实际请求的作业管理器或队列系统。 由于这是一个后端工具（ 至少这是我从你的问题中收集的 ），它不需要PHP来处理对作业本身的实际控制，而只是让一些控制过程安排这些单独的工作并将它们交给你PHP脚本使您可以有效地施加这些限制。
我的第一个建议是尝试像gearman这样的东西，这是一个很棒的工作经理，并且在PHP中有一个扩展，可以帮助你与库进行交互。
另一个建议是看看像amqp或zmq这样的队列系统，其中一些系统在PHP中也有扩展 。
所以这是你的一个示例场景......
您有一个接受这些请求的PHP脚本，并将它们交给您的作业管理器或通过套接字排队。 作业管理器或队列将存储请求并以可以集中和控制以施加这些限制的方式将其分发给各个工作人员。 我给你的链接中有一些例子可以帮助你实现目标。 然而，如果不经过精心设计和考虑，在没有这些工具的帮助下纯粹在PHP中完成它将证明是非常棘手的，并且可能会出现在一些非常边缘的错误行为中。
PHP is really not the best language to use for concurrent programming. However, there are some third party solutions that you can use along-side of PHP that can help you achieve your goals.
What you need is a job-manager or a queue system that can handle the actual requests for you. Since this is a back-end tool (at least that's what I gathered from your question) it doesn't require PHP to handle the actual control over the jobs themselves, but just have some controlling process schedule these individual jobs and hand them to your PHP scripts so that you can effectively impose these limits.
My first suggestion would be to try something like gearman, which is a great job manager and has an extension in PHP to help you interface with the library.
Another suggestion is to take a look at queue systems like amqp or zmq, some of which also have extensions in PHP.
So here's an example scenario for you...
You have a PHP script that accepts these requests and hands them off to your job manager or queue over a socket. The job manager or queue will store the request and distribute it off to the individual workers in an a way that can be centralized and controlled to impose these limits. There are some examples from the links I gave you that can help you get there. However, doing it purely in PHP without the aid of these tools will prove quite tricky and could wind up in some very edge-case buggy behavior if not carefully crafted and considered.
更多推荐
发布评论