I am currently developing a website that has to perform a task every 10 seconds. The website checks a database every ten seconds, finds rows that are to be processed, and then run a short script on each row.
The service is working great right now, but I just want to make sure that once the database gets very large, the service will still work efficiently.
I am using a cron job that runs every minute, with sleep functions every 10 seconds.
Thank you for your help!
Just make sure your tasks don't overlap, i.e. wait till the previous task is over before launching a new one. Otherwise, if your tasks take more than a minute (or whatever timer you've set for them), you'll end up with tons of those running in parallel and your system will run out of available resources.