I have a question about queuing in php. so far I am making a web based app to manage social media accounts like facebook, twitter, instagram. my question has been asked before in this question by some other people
My app may gain thousands of users and I want to make it fast and fair. so here is the scenario:
users come to my app and register. they enter their social media user/pass. make some scheduling tasks or activity (eg. scheduling posts to be send or follow/unfollow people who followed their accounts)
some of these tasks can be done with a cron job like checking every minute to send scheduled posts. BUT question is how much?
how much post can I process in a cronjob? 10? 100? 1000? what if I have more than 10000 posts scheduled to be send at the same time in the worst case?
and about jobs that should run every x times in hour? how about them? if there is a limitation, for example if I cant follow/unfollow more than x people per hour? how can I schedule this kind of tasks to run fair and fast? should I run a permanent running job to check and do these tasks each moment?
as I said each user can add thousands of tasks in this list. for example each user can set it to follow 5000,000 people so how should I process it to be fair? there are more than one user who are in the queue so if one person needs to do 5000,000 follow the next one should wait till the end?
as mentioned in the other question I linked above the solution was aws services but I think that could be a FIFO queue isn't it? I know it IS fair but It is not fast. how should I accomplish this goal?