脚本运行时间太长

I made a program that works like so:

  1. grabs a list of existing members from the database
  2. goes through each members "link" which points to a forum thread, and it uses some xpath comments to grab a "date"; this date refers to when it was last updated
  3. it then compares this date to the one stored under the member, to see if an update for the member is required; if an update is required it will trigger a boolean for another script to run

Now I need to run this script every hour, as it was designed to check a members forum thread for specific information.

However as currently there are over 80 users, this script takes a long time to run; this number is likely to increase as time goes on so this is a problem, as it never finishes the script completely (I even set the exec time to 0, but this hasn't helped).

I figured the best way to deal with this was to run this script multiple times an hour with some way to check 30 unique members at a time.

But I am struggling to think of a clean method of doing this. Can anyone help me or suggest a better method of handling so many members at a time?

I added a new column of data known as update_status. This can be marked as 'na', 'ur' or 'nu' (not applicable, update required or no update).

I timed the script at checking 30 members and it took an average of 1 minute 40 seconds, so I have this script run every 4 minutes to be safe.

It only grabs members with the update_status of 'na', so it works its' way through the members until there are no more left, and then eventually the updater runs.

It marks members that need updating with 'ur', and members that don't require it get marked with 'nu'.

Then when the updater runs once on the hour, it only checks members that required the updates, and then runs a global update to reset all members back to 'na'.

It's not the perfect answer, but it did give me an idea for one in the future and for now will work fine :)