i wrote a site crawler and i use while loop to crawl whole web site but my loop stop after about 660 seconds .
i set set_time_limit ( 0 ) and use flush to echo output and i use sleep function,i thought my code is wrong but i test a simple while loop in my code :
while ( 1 ) {
sleep ( 30 );
echo "Crawling on the go ..." . time ();
echo "<br />";
echo str_pad ( " ", 4096 );
flush ();
}
But this simple while stop too after about 660 seconds!i don't know what's my problem ! I checked WHM Process Manager on my server and i saw that my process killed ! i wanna know server firewall kill my process because of something like CPU or RAM usage ? plz help me!
You need to write set_time_limit(0);
at top of the page after PHP opening tag. This has worked for me .. I hope this will also work for you too..
set_time_limit() may be denyed (it has no effect if PHP is in safe_mode). On a side note, the sleep() time is not included in the time limit, so if you remove the sleep and the time script is working is changed, probably it's time limit. To check if this is a time limit (other option is CPU-time limit), register shutdown function and check connection status (value of 2 means time out)
You can also put the following line in your .htaccess php_value max_execution_time <timeinseconds>