Scrapy：设置条件settings.py(Scrapy: Set conditional settings.py)

Scrapy：设置条件settings.py(Scrapy: Set conditional settings.py)

我想仅在文件存在时才应用特定设置和download_middlewares（代理）。 仅当具有代理列表的.txt存在时，蜘蛛应该使用代理运行，否则，它应该在没有代理的情况下运行，具有默认IP和端口。

我尝试了以下但它对我不起作用：

settings.py

import os.path if os.path.isfile("../proxies.txt"): BOT_NAME = 'whatever' SPIDER_MODULES = ['whatever.spiders'] NEWSPIDER_MODULE = 'whatever.spiders' RETRY_ENABLED = False REDIRECT_ENABLED = False DOWNLOAD_TIMEOUT = 15 COOKIES_ENABLED = False LOG_ENABLED = True DOWNLOADER_MIDDLEWARES = { 'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 110, 'whatever.middlewares.ProxyMiddleware': 100 } else: BOT_NAME = 'whatever' SPIDER_MODULES = ['whatever.spiders'] NEWSPIDER_MODULE = 'whatever.spiders' DOWNLOADER_MIDDLEWARES = { }

这有什么解决方法吗？

感谢你们！

I'd like to apply specific settings and download_middlewares (proxies) only if a file exists. The spider should run with proxies only if the .txt with the proxies list exists, otherwise, it should run without proxies, with the default IP and port.

I've tried the following but it doesn't work for me:

settings.py

import os.path if os.path.isfile("../proxies.txt"): BOT_NAME = 'whatever' SPIDER_MODULES = ['whatever.spiders'] NEWSPIDER_MODULE = 'whatever.spiders' RETRY_ENABLED = False REDIRECT_ENABLED = False DOWNLOAD_TIMEOUT = 15 COOKIES_ENABLED = False LOG_ENABLED = True DOWNLOADER_MIDDLEWARES = { 'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 110, 'whatever.middlewares.ProxyMiddleware': 100 } else: BOT_NAME = 'whatever' SPIDER_MODULES = ['whatever.spiders'] NEWSPIDER_MODULE = 'whatever.spiders' DOWNLOADER_MIDDLEWARES = { }

Is there any workaround for this?

Thank you guys!

最满意答案

您列出的代码应该。 其他代码段中是否有任何错误？

The code you listing should. Is there anything error in other code snippet?