用PYTHON 的 requests库请求一个post 状态码是200,但是返回内容是空。但同样的数据在fiddler 组合器,或者其它请求调试器 包括网页上找的中反回来JOSN。
得到的结果
另一个
网页上找的都可以
https://www.amazon.co.uk/gp/delivery/ajax/address-change.html
post
Host: www.amazon.co.uk
X-Requested-With: XMLHttpRequest
anti-csrftoken-a2z: gPEyD4jV7jHGbNAexAsKUIoQvXEwiNSOwFo3VfAAAAAMAAAAAGLPhJxyYXcAAAAA;hAQburVCSLlXy2uqtg9lK8QvCXBXwLD2A8Kfc6eVNr8pAAAAAGLPhJwAAAAB
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 Edg/103.0.1264.49
Content-Length: 115
locationType=LOCATION_INPUT&zipCode=NG10+1PU&storeContext=generic&deviceType=web&pageType=Gateway&actionSource=glow
import requests
import json
url = 'https://www.amazon.co.uk/gp/delivery/ajax/address-change.html'
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 Edg/103.0.1264.49',
'viewport-width': '1278',
'Host': 'www.amazon.co.uk',
'Connection': 'keep-alive',
'X-Requested-With': 'XMLHttpRequest',
'anti-csrftoken-a2z': 'gPEyD4jV7jHGbNAexAsKUIoQvXEwiNSOwFo3VfAAAAAMAAAAAGLPhJxyYXcAAAAA;hAQburVCSLlXy2uqtg9lK8QvCXBXwLD2A8Kfc6eVNr8pAAAAAGLPhJwAAAAB',
'Content-Type': 'application/x-www-form-urlencoded',
}
data = {
'locationType': 'LOCATION_INPUT',
'zipCode': 'NG10 1PU',
'storeContext': 'generic',
'deviceType': 'web',
'pageType': 'Gateway',
'actionSource': 'glow',
'almBrandId': 'undefined'
}
zip_code = requests.post(url=url, headers=header, data=data)
print(zip_code.text)
print(zip_code.cookies)
得到这个空的JOSN数据
body中的data参数是用urlencoded形式传过去的,用urlencode处理一下
import requests
from urllib.parse import urlencode
import requests
from urllib.parse import urlencode
url = 'https://www.amazon.co.uk/gp/delivery/ajax/address-change.html'
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 Edg/103.0.1264.49',
'viewport-width': '1278',
'Host': 'www.amazon.co.uk',
'Connection': 'keep-alive',
'X-Requested-With': 'XMLHttpRequest',
'anti-csrftoken-a2z': 'gPEyD4jV7jHGbNAexAsKUIoQvXEwiNSOwFo3VfAAAAAMAAAAAGLPhJxyYXcAAAAA;hAQburVCSLlXy2uqtg9lK8QvCXBXwLD2A8Kfc6eVNr8pAAAAAGLPhJwAAAAB',
'Content-Type': 'application/x-www-form-urlencoded'
}
data = {
'locationType': 'LOCATION_INPUT',
'zipCode': 'NG10 1PU',
'storeContext': 'generic',
'deviceType': 'web',
'pageType': 'Gateway',
'actionSource': 'glow',
'almBrandId': 'undefined'
}
post_data = urlencode(data)
zip_code = requests.post(url=url, data= post_data, headers=header)
print(zip_code.text)
https://www.cnpython.com/qa/153454
根据代码的链接,发现现在打不开该网页了,没法进行测试,拿不到数据可能是该页面并不是静态页面来的,可能是一个动态页面,需要你打开控制台自己去找出存储数据的链接,请求该链接拿到页面数据.
动态网页不是这么爬的,你这只能爬取静态
关于该问题,我找了一篇非常好的博客,你可以看看是否有帮助,链接:python requests中的post请求的参数问题