Here is what I'm doing: a few workers and queue with job for them.
I have a simple worker:
func worker(jobs <-chan int, wg *sync.WaitGroup) {
for work := range jobs {
perform(work)
}
wg.Done()
}
Start them:
jobs := make(chan int)
var wg sync.WaitGroup
for w := 1; w <= parallel; w++ {
wg.Add(1)
go worker(jobs, &wg)
}
Also I have a queue, and here goes problem, how to check if there is available worker and write only then?
func queue(jobs chan int, stop chan bool) {
for {
select {
case jobs<-?: // How to check what worker is available?
work := receiveWork()
jobs <- work
continue
case <-stop:
return
}
}
}
}
Start queue:
stop := make(chan bool)
go queue(jobs, stop)
Note what receiveWork()
should be called only if there is an available worker.
One option to use two channels, one for ready and one for the actual work:
func worker(ready chan struct{}, work chan int, wg *sync.WaitGroup) {
for range ready {
w, ok := <-work
if !ok {
break
}
perform(w)
}
wg.Done()
}
func queue(ready chan struct{}, work chan int, stop chan struct{}) {
defer close(ready)
defer close(work)
for {
select {
case ready <- struct{}{}:
work <- receiveWork()
case <-stop:
return
}
}
}