I'm trying to access remote url using Curl. When curl is executed i got this error
<html style="height:100%">
<head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
</head>
<body style="margin:0px;height:100%">
<iframe src="/_Incapsula_Resource ?CWUDNSAI=24&xinfo=4-13149215-0 0NNN RT(1490255810449 195) q(0 -1 -1 -1) r(0 -1) B12(4,315,0) U18&incident_id=439000400034489515-104043319326605956&edet=12&cinfo=04000000" frameborder=0 width="100%" height="100 %" marginheight="0px" marginwidth="0px">
Request unsuccessful. Incapsula incident ID: 439000400034489515-104043319326605956
</iframe>
</body>
</html>
Of course Incapsula rejects your request because curl behavior looks like a bot. If you are the admin of the site you can try adding a specific header to curl using -H option and whitelist that header in the Incapsula console.
Just go to the website console and inside the security settings you can find the option to whitelist with following fields:
▪ IP - Single IP (1.2.3.4), Range (1.2.3.4 - 1.3.3.4), Subnet (1.2.3.4/16).
▪ URL - exact URL (/admin); URL which contains all sub-folders (/exmaple/*). Note that it is recommended to seperate between wildcard exceptions and exceptions for exact paths.
▪ Client app ID - Each of the known client app IDs by Incapsula (Such as Qualys Scanner).
▪ Country - Specific country or continent.
▪ User-Agent - Each of the known user agents, such Chrome, FireFox, etc. Wildcard (*) can NOT be set here.
▪ HTTP Parameter - Specific HTTP parameter (not a value). Wildcard (*) can NOT be set here.
The website is protected by Incapsula.
You can use incapsula-cracker for bypassing incapsula
thanks Blender for his answer here