I'm writing a script that uses curl to fetch the contents of a webpage. I search the headers for Set-Cookie
to collect the names of all cookies set by the page. The problem is that some cookies are set from within a javascript file loaded into the page and of course these cookies aren't included in the response headers.
So, is there a way to get all cookies from a webpage using curl, including the ones that is generated through javascript?
Currently I'm doing something like this:
$curl = curl_init();
curl_setopt($curl, CURLOPT_VERBOSE, 1);
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_URL, <www.example.com>);
$response = curl_exec($curl);
$http_code = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($http_code == 200) {
$headers = substr($response, 0, curl_getinfo($curl, CURLINFO_HEADER_SIZE));
// check headers for all cookies sent in response
}
curl_close($curl);
cURL isn't going to run javascript, so no, there won't be a way to get those cookies. You'll have to emulate or automate a browser in order to achieve that. Fortunately there are tools to help you. Maybe take a look at PHP-webdriver