上课的爬虫题!添加完了ueragent,还是显示不出来内容。也没有报错,是哪里还有问题嘛!!谢谢~~
你如果代码不截图,直接发源码,这边还可以调试看看
import requests
import json
from lxml import etree
url="https://www.jobui.com/trends/quanguo-pythonkaifagongchengshi/"
header={"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6 Safari/605.1.15"}
response = requests.get(url,headers=header)
content = response.content.decode()
html = etree.HTML(content)
res = html.xpath("//a[@class='cfix job-hover-color']")
List = []
for i in res:
j={}
j["place"]=i.xpath("./span[@class='fl']/text()")[1].strip()
x=i.xpath("./span[@class='fr']/text()")[0].split(",")[0].split("K")[0]
if'.' in x:
#y=print(int(float(x)*1000))
j["date"] = int(float(x)*1000)
else:
j["date"] = x
print(j)
List.append(j)
print(List)
with open("02.json","w",encoding="utf-8") as f:
json.dump(List,f,ensure_ascii=False,indent=2)
print("success!")