I'm wanting to make a scrape from a site to compile info about argentina's airports. the site is http://www.anac.gov.ar/anac/web/index.php/2/310/informacion-aeronautica/notam . im trying to make curl fetch the data that in the page is accesed with an ajax query
<script type="text/javascript">
var lugares = document.getElementById("locations");
$(lugares).change(
function() {
$('.loading').show();
$('.ajaxbody').hide();
$.ajax(
{
url: "/notam/pib",
type: "POST",
dataType: "html",
data: {indicador: $(lugares).val()}
}).done(function(html) {
tabla = document.getElementById('pib');
$(tabla).find('table').remove();
$(tabla).append(html);
$('.loading').hide();
$('.ajaxbody').fadeIn('slow');
});
});
</script>
this is how i use curl curl -X POST -H "dataType:html" -d'/notam/p ib'-H "indicador=TRE" http://www.anac.gov.ar/anac/web/index.php/2/310/informacio n-aeronautica/notam
but curl wants to resolve the -H "indicador=TRE" as it were an url but it is data being send in the POST.
im seeing the real POST made by the page with wireshark and this is what it sees
POST /notam/pib HTTP/1.1
Host: 186.153.175.229
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://186.153.175.229/portal/notam
Content-Length: 13
Cookie: PHPSESSID=k1daatvv9882drooofid7cvvc6
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
indicador=TREHTTP/1.1 200 OK
Date: Tue, 18 Feb 2014 23:30:41 GMT
Server: Apache/2.2.22 (Ubuntu)
X-Powered-By: Fat-Free Framework
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Access-Control-Allow-Origin: http://geo.anac.gov.ar
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 1512
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
...........Y.r.H...O.gm.j."..1DsE.Dj.<P....PF........>..=......QI61...G........f.W.......7..s.x..I.......Q/w.
....P...Zx.^..\.......9.zq....^.N.p.E.!........3.y.On..[.hL.....Q<...7...%.,|h......S....#.D...w.K.._.!...l..
..Q?...]#....g....x}..a.ir..
.E.Hx.eo;.._...4..j..Iqj|.o...[F...|.....:B......Oe`....!0"...,..M..M.O....k.h2,..t...VLPZ-.V...E....C..!.*.v..E..].....s.....-...&.......j2;...G....q.68...2F..G....m..S.[dp$.
..e..Di.. ....AxK.)...S...y.=..7....>?...yA.t.F.@.l.tD..?.ME5l.Z8..8...@.-.....f[8.f..=<4...t.pR(>n.1.............."..4.W.I.2...1UY.U..M.L.....G... et....].5.E
.q...;q.6..?.{... .i.+@*......X.. ...A..6.%0....Z...2.W..J....Mi...n-E......A.G.PM.........*m.4N.......N....p*..............R.v#..
P....)<.c...\..".e...(.3...P.3..%vK....6N..8......9....DD,U %..d...=2.Z.......f..0...........R......\...h....H/F6....M3..m..&6..
..n.qc1...).3..\.+.r.4b.6j.3h...R&..+.D..1kW@DR,C..i.-.....+..Hu...D...."...y.v.6.8.X..R.J.A.B...h.{...H.......y.P..*.t.$z.!....t>....Sa?.....5.t.t..&.r.K5Zx:aI$/#.....N......4.D.k.....3...?x!8..&....|...v....hY
X..{Q.m...d.X.......R...OO"G..U........!.......3?.
...o......QT.NI..%.k. .^...[.*.'....,>.8..s.O....<.vd...O&..>.n.f..E..Q..]..9.&w.....5..`4.e..0......Q8d.O..p.I..8.j.r..?...$
a....^.5...&.|.....4......&..z.&I&2.c]L..(.........~....8..a.xc.....7t...O.z.............m....\.~{............}I..i?.M.H.a....[/m]..:.P....B.y........;...m5..|..ZEau.....hBC}0..D..a....... .....Y.....^...6.
...../m...V=Ob.D./c..)..G...hH..h...1.$].@k........{......8...?..... ..
thanks.
Your curl command is confusing. Specially the URL. So I used a dummy url here in my command.
curl -X POST -H "dataType: html" -d "/notam/p ib" -H "indicador: TRUE" http://url/
I have replaced your single quotes with double quote, also adjusted the space before -H.
i've solved my problem using this
curl -X POST -H "X-Requested-With: XMLHttpRequest" -d "indicador=TRE"http://186.153.175.229/notam/pib
My problem was that I was sending the url part of the ajax query as -d and really was a relative path from the ip of the host. also found this http://www.laktek.com/2012/03/12/curl-tips-for-daily-use/ it helped a lot.
I Could not compile curl under windows so I used what i've learned from curl to make the http POST with Qt c++ with the following code.
QNetworkAccessManager *nwam = new QNetworkAccessManager;
QNetworkRequest request(QUrl("http://186.153.175.229/notam/pib"));
QUrl URL;
QByteArray postData;
URL="http://186.153.175.229/portal/notam";
request.setRawHeader( "X-Requested-With"," XMLHttpRequest" );
QString postKey = "indicador=TRE";
postData.append(postKey);
QNetworkReply *reply = nwam->post(request,postData);