pyspider 项目如何retry failed的网页,而不重跑已经成功的网页
有个retries参数可以配置crawl_config = {'headers': {'User-Agent':red.srandmember("USER_AGENT"),'cookie':red.srandmember("AILIBABA_1688_COOKIE")},'proxy':"10.10.10.10:80",'timeout':10,'retries':100}