I have a problem for scrape a URL, if I use a dev tools of Chrome i see the URL and preview if content, but if I intent open URL from browser return me 404 not found.
This is URL with AJAX data . I Need the parte.of odds comparison: http://www.betexplorer.com/soccer/england/premier-league/wolves-manchester-city/Kz6hM15m/ This is a AJAX data URL : www.betexplorer.com/gres/ajax/matchodds.php?p=0&e=Kz6hM15m&b=1x2 this URL return 404 not found in browser but in fiesta URL show the odds . Is possible ger the data from secondo url?
Thank you.
Upon quick inspection of the site you have mentioned in your question, I found out it is checking for referer and user-agent, when handling the request.
Adding header "Referer" with value of "http://www.betexplorer.com/" and a random "browser-like" user agent, yields the json response with requested odds :)
For example:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.betexplorer.com/gres/ajax/matchodds.php?p=0&e=Kz6hM15m&b=1x2");
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.12011-10-16 20:23:00");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, "http://www.betexplorer.com/");
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
$output = curl_exec($ch);
curl_close($ch);