I just finished writing a simple proxy application in go: The code gets UDP packet from one interface, encrypt the data and then send it to another interface using TCP.
Currently, I'm using three goroutines: one for receiving, one for encrypting and one for sending the data. I just started to try and find ways to improve the efficiency and speed of the code.
First, I thought about creating a new goroutine for each packet needed to be encrypted but after reading the following two blog posts I understood that this isn't the right thing to do:
After reading those two I found myself with two questions about thread-pool:
Thank
The purpose of a worker pool is to limit the concurrency of tasks executed by the workers. This is useful when performing tasks that require sufficient resources (CPU, memory, etc.), and running too many tasks at the same time would exhaust resources.
With a worker pool, tasks are dispatched to available workers. If separate tasks (in your case packets to encrypt) are dispatched to separate workers, then you cannot determine which worker (goroutine) is executed first, and will not know the order that tasks are completed.
If you want to parallelize the encryption tasks, but need to maintain order, then an index can be assigned to each packet/task as input is received. When the worker pool is done with each task, the output can be put into a sorted container such as a heap where sorting is done according to index. As soon as the encrypted packet with the next index in the sequence is at the top of the heap, it is popped from the heap and sent.