I want to create a worker pool in golang, to divide a big task in multiple jobs.
The problem is that the routines may produce multiple output as a result for the computation, so I cannot know in advance how many times I shall iterate over the results channel.
I know that I can use WaitGroup, but I was wondering if there is another pattern I can use, in order to let the main thread advanced on the computation and iterate over the results instead of waiting first every routine to be done.
Another solution I thought, which is not very elegant to me, is to create a Result
struct or return an array of results. Since I can count the number of works, I know how many results arrays I'm gonna receive.
You can do something like this:
go func() {
wg.Wait()
close(results)
}()
for res := range results {
// Handle results
}
When all your workers finish processing, close the results channel. The process that's reading results will finish and then exit the loop.