I have two Amazon SQS queues that are monitored by two workers. I use it to send e-mails, send some worker requests. The typical back-end stuff. Must be around a few 100 jobs this month.
However, I am getting email from Amazon that I am already at at 887,457 Requests of the free tier 1,000,000 Requests of Amazon Simple Queue Service.
I am wondering how I am getting at this number? Do the workers poll the queue which is considered a request? If so, can we increase this interval?
There is a --sleep option for queue workers:
php artisan queue:work --help
The default is 3 seconds, so it seems like a request is being made from each of your workers every 3 seconds. With 2 workers, that's 57,600 requests per day. If it's been about 2 weeks since you started this, that would be ~800k requests.
Additionally I have in mind that there is something like a maximum message size in amazon SQS messages. The way how they handle bigger messages is, that they count one message as multiple requests. Some time ago that was the reason why I reached the free tier limit. An example for big messages are jobs, that have big Objects (eloquent models) in the constructor. It seems like these objects are serialized and thats the reason for the huge size of a message.
The solution is to only give simple data types to constructors of jobs (e.g. the user id and not the user object), and then get the whole object within the handle method of the job.
You may want an immediate response if there is a message and at the same time reduce the number of requests to SQS when there are no messages. The correct configuration for this is called "long-polling". This is done by setting the Receive Message Wait Time to 20 seconds on the ReceiveMessage call or for the queue by default. As Laravel does not have an option to pass parameters to the queue call, you will have to do this in the Queue Options.
What this means is that if there are no messages in the queue, SQS will not respond for up to 20 seconds. That is 20 seconds, if there are still no messages then or shorter, if any message enters the queue. This dramatically reduces your SQS requests by factor 20, plus the requests that actually have jobs to process.