爬虫的时候,最后什么都没有返回,是因为内容太多吗
response = requests.get('https://ncov.dxy.cn/ncovh5/view/pneumonia')
home_page = response.content.decode()
soup = BeautifulSoup(home_page, 'lxml')
script = soup.find(id='getListByCountryTypeService2true').text
print(script)
谢谢大家!
看下response的结果是不是200
😂😂是的,内容太多了,你本地存不下
应该这样写吧!
import requests
from bs4 import BeautifulSoup
url = 'https://ncov.dxy.cn/ncovh5/view/pneumonia'
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3877.400 QQBrowser/10.8.4507.400',
}
response = requests.get(url=url, headers=headers)
response.encoding = 'utf-8'
html = BeautifulSoup(response.text, 'lxml')
content = html.select_one("#getListByCountryTypeService2true")
print(content)
运行结果: