SO, I have been fighting with a piece of code that I want to use to get a remote page's source code using curl.
The code executes successfully, both in the browser and on command line. However, I get the of the main file only. When parameters are added, they are not considered whatsoever in the output.
The Code: STACK : Ubuntu, Nginx, PHP-FPM 7.2
$urlcontent = 'https://XXX.YYY.COM/file/?var1=value1' ;
// Create a new cURL resource
$curl = curl_init();
if (!$curl) {
die("Couldn't initialize a cURL handle");
}
// Set the file URL to fetch through cURL
curl_setopt($curl, CURLOPT_URL, $urlcontent);
// Set a different user agent string (Googlebot)
curl_setopt($curl, CURLOPT_USERAGENT, 'CodiBot/2.1');
// Follow redirects, if any
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
// Fail the cURL request if response code = 400 (like 404 errors)
curl_setopt($curl, CURLOPT_FAILONERROR, true);
// Return the actual result of the curl result instead of success code
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
// Wait for 10 seconds to connect, set 0 to wait indefinitely
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 10);
// Execute the cURL request for a maximum of 50 seconds
curl_setopt($curl, CURLOPT_TIMEOUT, 50);
// Do not check the SSL certificates
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
// Fetch the URL and save the content in $html variable
$html = curl_exec($curl);
// Check if any error has occurred
if (curl_errno($curl))
{
echo 'cURL error: ' . curl_error($curl);
}
else
{
// cURL executed successfully
print_r(curl_getinfo($curl));
print_r($html);
}
curl_close($curl);
PROBLEM
I get the content for https://XXX.YYY.COM/file
but not the corresponding ?var1=value1
part. IN other words, as I feed info to be retrieved to DB I get only the html of the main file.
I tried :
curl_setopt($ch, CURLOPT_POSTFIELDS, 'foo=1&bar=2&baz=3');
I know the remote server may have CORS enabled, but I tried the same url using a remote curl retriever and it succeeded. SO, it may not be the remote server