There is a site researcherid. On author page, for example, http://www.researcherid.com/rid/A-7364-2016 i need to retrieve information from Citation metrics
window. That`s the window generated by ajax. How can i do it with Python ? What tools i have for it? I tried:
payload = {'key': '721889',
'listid': 'LIST1',
'displayName': 'My Publications', 'publicProfile': 'true', '_': ''
}
r = requests.post('http://www.researcherid.com/rid/A-7364-2016',
data=payload, headers={
'X-Requested-With': 'XMLHttpRequest'
})
print(r.text)
But it does`t work.
You can use Selenium
's webdriver tools in order to get dynamically loaded content. Simply create a webdriver instance, go to Citation metrics
page and load source into python. After that, you can scrape content with libraries such as LXML
or BeautifulSoup
.