I'm trying to make a php script which can take a specific tv channel schedule from a specific website written in Yii Framework. I'm trying to get the full HTML using php cURL but I can't find this section.
$curl = curl_init('http://port.hu/tv');
curl_setopt ($curl, CURLOPT_POSTFIELDS, 'id=tvchannel-3&date=2017-02-05'); // for example
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_POST, 1);
$content = curl_exec($curl);
echo $content;
I tried before with php file_get_contents with the same result. Any idea? Thanks.
You cannot scrape directly from the website page, because looks like the website is using ajax (I guess) to load the data onto the page. So what I did, I monitor the Network activity on the page using Chrome Developer Tools, and I found this API url:
http://port.hu/tvapi?channel_id=tvchannel-3&i_datetime_from=2017-02-05&i_datetime_to=2017-02-10
It returned JSON strings, and the dev does not secure the API. So no need to scrape anymore, just load the JSON API directly.