This feels beyond janky but our vendor has given me no other options. I need to access their form and extract the data from the resulting csv generated via cURL (or an alternative). The only example they are willing to provide is cold fusion (i know...)
<cfsetlocal.username = 'username' />
<cfsetlocal.password = 'Password' />
<cfhttp url="http://stupidvendor.com/users"
method="POST"
username="#local.username#"
password="#local.password#"
result="result">
<cfhttpparam type="URL" name="header_required" value="0" /> </cfhttp>
Downloading <cfoutput>#result.ResponseHeader.Location#</cfoutput><br><br>
<cfhttpurl="#result.ResponseHeader.Location#" method="get"
username="#local.username#" password="#local.password#"
file="Result.csv"
path="#expandPath('.')#"/>
Saved as Result.csv
I tried many many attempts but as I'm not expert in cURL any suggestions or links are much appreciated thank you all very much.
UPDATE: I'm able to get the contents of the url but it's an html form which needs to be submitted to generate a CSV which I then need the contents of. It's these last steps that I'm unsure how to approach or if this is even feasible with cURL
I'm also not familiar with ColdFusion so I hope this might help some. If the form is protected by Apache/Nginx authentication requirements, you need to pass the user name and password using CURLOPT_USERPWD
. Here's a use example.
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERPWD, "user@stupidvendor.com:my_password");
curl_setopt($ch, CURLOPT_URL, 'http://stupidvendor.com/users');
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
The result you get back will be the contents of the page. Since you didn't specify where the CSV is coming from, I'm not sure how to help there.
Several applications I've built require a login to get information but they use a normal login form, then store the cookie (session ID) then it can access the protected data.
To submit form data you just need to simulate what the form is doing. So if the form does a POST request to a page, then you do a cURL to that page with the required POST data. As an example without knowing anything about the form.
// This is the data the from would submit
$postFields = array(
'document' => 'report',
'format' => 'csv'
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERPWD, "user@stupidvendor.com:my_password");
// This is the URL the form submits to
curl_setopt($ch, CURLOPT_URL, 'http://stupidvendor.com/users');
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Assuming the from does a POST not a GET submission
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($postFields));
curl_setopt($ch, CURLOPT_POST, true);
$result = curl_exec($ch);
curl_close($ch);
From there, you need to look at the data in $result
to see what they are sending, is it the file itself or a link to the file in an HTML document.