I have an application which keeps looping while calling lpop. Using the top command, I can see that redis is using 64% of CPU, while my application uses 101%.
I'm using redis to create a queue and worker. My worker is in an infinite loop, calling lpop and waiting for the next job to come in.
For this, I'm using the machinery package. There is an issue for this here, where the problem is said to be from lpop. However, since the comments are confusing, I'm at a loss as to what the difference is between LPOP and BLPOP, apart from the fact that one doesn't block and the other does.
Using timed BLPOP instead of LPOP to avoid massive cpu usage
committed 7 days ago
commit 54315dd9fe56a13b8aba2d2a8868fc48dfbb5795
machinery/v1/brokers/redis.go - itemBytes, err := conn.Do("LPOP", redisBroker.config.DefaultQueue) + itemBytes, err := conn.Do("BLPOP", redisBroker.config.DefaultQueue, "1")
Use the latest version of machinery/v1/brokers/redis.go which changes LPOP to BLPOP.
Reference: Redis commands: BLPOP