想问一下大家,爬虫结果为空是为什么呢

爬虫的时候,最后什么都没有返回,是因为内容太多吗



response = requests.get('https://ncov.dxy.cn/ncovh5/view/pneumonia')
home_page = response.content.decode()
soup = BeautifulSoup(home_page, 'lxml')
script = soup.find(id='getListByCountryTypeService2true').text
print(script)


谢谢大家!

看下response的结果是不是200

😂😂是的,内容太多了,你本地存不下

应该这样写吧!

import requests
from bs4 import BeautifulSoup

url = 'https://ncov.dxy.cn/ncovh5/view/pneumonia'
headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3877.400 QQBrowser/10.8.4507.400',
}
response = requests.get(url=url, headers=headers)
response.encoding = 'utf-8'
html = BeautifulSoup(response.text, 'lxml')
content = html.select_one("#getListByCountryTypeService2true")
print(content)
 

运行结果:

img