如何使用Java+JS等技术手段获取阿里云域名交易页面特定域名的一口价,并没有我想象的那么简单,求大神看具体描述后指点一二!!!

举例:需要抓取的页面为阿里云域名交易某页面,如:

https://wanwang.aliyun.com/nametrade/detail/online.html?spm=5176.8076989.1407540.22.68fb1838nRC7wJ&productType=2&domainName=wawafang.com

页面截图:图片说明

需求:想通过程序把指点域名(如wawafang.com)的价格获取到,就这么简单,但发现并不那么容易做到。

1.首先,我通过调试查看了该页面的网页源代码,但搜索后并没法发现wawafang.com和价格5000这样的关键词。

图片说明

至此判断,通过上面的URL请求并没有一次性把域名及对应的信息请求过来,而是页面加载后再次ajax请求再获取,通过调试工具去找查看,找到如下:

图片说明

然后,看了下请求URL为:

https://domainapi.aliyun.com/onsale/detail?domainName=wawafang.com&orgType=undefined&productType=2&token=tdomain-aliyun-com:q8hsaMOagyeu7hquFzyTgIHwSkolKVcd&callback=jQuery11110902116498803003_1587029597566&_=1587029597567

把这个url,输入浏览器请求,并没有获取到想要的数据!通过网上查找相关资料,应该是https,token相关原因导致的,而且这个token值每次刷新请求都是变化的,至此,不知该怎么办了!

图片说明

有没有大神指点下,接下来该如何解决?

我这里看到的,特别是cookie postdata referer useragent,你自己对照下
token是第一次请求返回的

Request URL:https://domainapi.aliyun.com/onsale/detail?domainName=wawafang.com&orgType=undefined&productType=2&token=tdomain-aliyun-com:G35GvFJY1tUaBVZLfNPldnx6zkAbtzFy&callback=jQuery111107828746029851859_1587055463406&_=1587055463407
Request Method:GET
Status Code:200 
Remote Address:127.0.0.1:443
Response Headers
content-language:zh-CN
content-length:978
content-type:application/javascript;charset=UTF-8
date:Thu, 16 Apr 2020 16:43:46 GMT
eagleeye-traceid:0b57ff8215870554266676188e2bbe
server:Tengine
set-cookie:_s0=eNrz4A12DQ729PeL9%2FV3cfUxiKvOTLFS8nRyMjPzcjTU9YyydI0KD%2FeOiDAxNDBxdfFxDg4x1A13Nw%2BIMrL01jVxj%2FC0VNJJLrEyNLUwNzA1NTI0tzQ200lMhgmYGJmZmRvo5FZYGVoYGNRGAQCoiR0m; Path=/; HttpOnly
status:200
strict-transport-security:max-age=0
timing-allow-origin:*
x-content-type-options:nosniff
Request Headers
:authority:domainapi.aliyun.com
:method:GET
:path:/onsale/detail?domainName=wawafang.com&orgType=undefined&productType=2&token=tdomain-aliyun-com:G35GvFJY1tUaBVZLfNPldnx6zkAbtzFy&callback=jQuery111107828746029851859_1587055463406&_=1587055463407
:scheme:https
accept:*/*
accept-encoding:gzip, deflate, sdch
accept-language:en-US,en;q=0.8
cookie:cna=cXgfFyp07x8CAd5eXeeBdPFz; JSESSIONID=IBB66JA1-IZ9EZWWKXX4104EDLCST1-WG7PZ29K-4GXI9; _s0=eNrz4A12DQ729PeL9%2FV3cfUxiKvOTLFS8nRyMjPzcjTU9YyydI0KD%2FeOiDAxNDBxdfFxDg4x1A13Nw%2BIMrL01jVxj%2FC0VNJJLrEyNLUwNzA1NTI0tzQ200lMhgsYA6GpTm6FlaGFgUFtFACoFB0e; aliyun_choice=CN; l=eBTx3LumQAQb8xHhBOfwPurza77OSIRAguPzaNbMiT5PO_165DdfWZXtn3TBC3GOh6k2R3l_-770BeYBqIVlDBgq10vXgIkmn; isg=BGJi3cUrSCXDT1R5VvM91M_xs-jEs2bNbMFvUaz7jlWAfwL5lEO23ej5r7GD795l
referer:https://wanwang.aliyun.com/nametrade/detail/online.html?spm=5176.8076989.1407540.22.68fb1838nRC7wJ&productType=2&domainName=wawafang.com
user-agent:Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36
Query String Parameters
view source
view URL encoded
domainName:wawafang.com
orgType:undefined
productType:2
token:tdomain-aliyun-com:G35GvFJY1tUaBVZLfNPldnx6zkAbtzFy
callback:jQuery111107828746029851859_1587055463406
_:1587055463407

楼主这问题属于爬虫里比较高级的爬取动态内容技术,可以应用webdriver或者phantomjs等技术,如有兴趣可以一起交流学习。