Here is the problem I am trying to solve:
package main
import "fmt"
func workerA(work_in_chan <-chan int,work_out_chan chan<- int){
for d := range work_in_chan {
fmt.Println("A ",d)
work_out_chan <- d
}
}
func workerB(work_in_chan <-chan int,work_out_chan chan<- int){
for d := range work_in_chan {
fmt.Println("B ",d)
work_out_chan <- d
}
}
func account(account_chan <-chan int,final_chan chan<- int){
wa_in := make(chan int)
wa_out := make(chan int)
wb_in := make(chan int)
wb_out := make(chan int)
go workerA(wa_in,wa_out)
go workerB(wb_in,wb_out)
for d := range account_chan {
//TODO - dumb implementation starts here
wa_in <- d
<-wa_out
wb_in <- d
<-wb_out
//TODO - dumb implementation ends here
final_chan <- d
}
}
func main() {
account_chan := make(chan int, 100)
final_chan := make(chan int, 100)
go account(account_chan,final_chan)
account_chan <- 1
account_chan <- 2
account_chan <- 3
fmt.Println(<-final_chan)
fmt.Println(<-final_chan)
fmt.Println(<-final_chan)
}
The account goroutine receives incoming data on account_chan, executes some work on the data, and once complete sends the data to final_chan. The account work is done by workerA and workerB (order is not important),both must complete on the data before account sends it to final_data. There are a few requirements:
My pasted implementation is dumb since now workerA and workerB are never executing concurrently (as they could & should since they are completely independent of each other). So which concurrency pattern can I use to solve this problem?
With the restrictions you've provided there isn't much that can be done. Simply reordering the channel operation to allows concurrency might be all you're looking for.
for d := range account_chan {
wa_in <- d
wb_in <- d
<-wa_out
<-wb_out
final_chan <- d
}
play.golang.org/p/4d8hKyHTWq
The first time I saw this pattern, I worried "but what if B gets done first". It turns out the order doesn't really matter as both need to recv'd from.
An aside on style:
The provided snippet smells like it has too many channels and goroutines. But that may because this is a more complicated problem distilled down to a the essential parts. One thing that may actually be a problem is the out channel from the workers. Their output isn't used in the example and I can't see how it could be in a full listing. Either the values are copied in which case the out channel isn't needed (a sync.WaitGroup would be better) or they're not safe to share between the workers.
You pass the input for the workers and then block until you get their result separately.
// Give worker A work
wa_in <- d
// Wait until worker A finished
<-wa_out
// Give worker B work
wb_in <- d
// Wait until worker B finished
<-wb_out
Instead, use the select statement to wait for a result on one of two channels symultaneously:
func account(account_chan <-chan int,final_chan chan<- int){
wa_in := make(chan int)
wa_out := make(chan int)
wb_in := make(chan int)
wb_out := make(chan int)
go workerA(wa_in,wa_out)
go workerB(wb_in,wb_out)
for d := range account_chan {
wa_in <- d
wb_in <- d
for i := 0 ; i < 2; i++ {
select {
case <-wa_out:
case <-wb_out:
}
}
final_chan <- d
}
}
http://play.golang.org/p/U0fk1yiqWL
Now, the two workers will run concurrently but the program is still guaranteed to wait for all the workers to finish.
Also see the concurrency patterns go doc.