I made a small program to benchmark go channel throughput, however it always deadlocks, I tried very hard but cannot understand why:
package main
import (
"fmt"
"runtime"
)
const CONCURRENCY = 32
const WORK_PER_WORKER = 100
const TOTAL_WORK = CONCURRENCY * WORK_PER_WORKER
func work() {
sum := 0
for i := 0; i < 10000000; i++ {
sum *= i
}
}
type WorkItem struct {
Done chan int
}
func main() {
runtime.GOMAXPROCS(CONCURRENCY)
var workQueue [CONCURRENCY]chan *WorkItem
// initialize workers
for i := 0; i < CONCURRENCY; i++ {
workQueue[i] = make(chan *WorkItem)
}
// start workers
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
anItem := <-workQueue[i]
work()
anItem.Done <- 1
}(i)
}
completed := make(chan bool, TOTAL_WORK)
for i := 0; i < TOTAL_WORK; i++ {
go func(i int) {
// send work to queues
workToDo := &WorkItem{Done: make(chan int)}
workQueue[i/WORK_PER_WORKER] <- workToDo // !! DEADLOCK
// wait until the work is done
<-workToDo.Done
completed <- true
}(i)
}
fmt.Println("Waiting")
for i := 0; i < TOTAL_WORK; i++ {
<-completed
}
}
Your code go func(i int) { anItem := <-workQueue[i]; ... }
removes juste 1 item from workQueue[i]
but you are trying to stuff WORK_PER_WORKER items into it. You will work on CONCURRENCY many items and after that all reading goroutines have terminated and you have your deadlock.
Looping in the worker goroutines "solves" your deadlock: http://play.golang.org/p/j2pavqnBDv Just "solves" because these worker goroutines will never terminate. Maybe you can experiment with close
ing your channels to notify the worker goroutines when nothing will be sent.
Because your worker only process one task and then exit. Thus, only first CONCURRENCY
items proceed and then workQueue[i/WORK_PER_WORKER] <- workToDo
blocks indifinitely. Thus, completed
chan never receive enough values and main
also blocks forever.
Your worker should do work in loops, like this:
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
for anItem := range workQueue[i] {
work()
anItem.Done <- 1
}
}(i)
}