直接上代码!
import requests
import json
headers = {
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:51.0) Gecko/20100101 Firefox/51.0'
}
base_url = 'http://search.kongfz.com/product_result/?key=走出思维的误区'
resp = requests.get(base_url, headers=headers)
resp.encoding='utf-8'
html_doc=resp.text
a=json.loads(html_doc)
返回的错误是:
a=json.loads(html_doc)
File "E:\python\lib\json__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "E:\python\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "E:\python\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
网页响应没问题,就是用json的时候出现了问题,请大神帮忙分析,最好写个小代码,我的目的就是爬一下书本的价格。
这个数据应该是动态加载的。
真正的数据在这个网址下面:
http://search.kongfz.com/product_result/?key=%E8%B5%B0%E5%87%BA%E6%80%9D%E7%BB%B4%E7%9A%84%E8%AF%AF%E5%8C%BA&pagenum=1&ajaxdata=1&type=1&_=1585723537261
代码为:
import requests
import json
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:51.0) Gecko/20100101 Firefox/51.0'
}
base_url = 'http://search.kongfz.com/product_result/?key=%E8%B5%B0%E5%87%BA%E6%80%9D%E7%BB%B4%E7%9A%84%E8%AF%AF%E5%8C%BA&pagenum=1&ajaxdata=1&type=1&_=1585723537261'
resp = requests.get(base_url, headers=headers)
resp.encoding='utf-8'
html_doc=resp.text
a=json.loads(html_doc)
list1=a['data']['itemList']
for i in range(len(list1)):
print(list1[i]['author'],list1[i]['itemname'],list1[i]['price'])
运行结果: