I'm trying to create a program that send strings to a pool of goroutines (through a channel). Once the goroutine have finish their job, they send some results (through an other channel).
The code is:
package main
import "fmt"
import "os"
import "sync"
import "bufio"
func worker(linkChan <-chan string, outChan chan<- string, wg *sync.WaitGroup, jobId int) {
defer wg.Done()
for url := range linkChan {
// ...
outChan <- url
}
}
func main() {
lCh := make(chan string)
wg := new(sync.WaitGroup)
outCh := make(chan string)
urls := []string{}
if len(os.Args) > 1 {
for _, link := range os.Args[1:] {
urls = append(urls, link)
}
} else {
s := bufio.NewScanner(os.Stdin)
for s.Scan() {
urls = append(urls, s.Text())
}
}
num_worker := 10
for i := 0; i < num_worker; i++ {
wg.Add(1)
go worker(lCh, outCh, wg, i)
}
for _, link := range urls {
lCh <- link
}
close(lCh)
for res := range outCh {
fmt.Printf("%s
", res)
}
close(outCh)
wg.Wait()
}
Running echo "something" | ./main
cause a deadlock.
From what I've understood, close(lCh)
should stop the for url := range linkChan
loop. Am I wrong (it seems so since the code deadlock) ?
How can I resolve this deadlock ?
Thank you for your answers.
You need to pump the urls in a goroutine, otherwise the outCh
will fill up which as you aren't emptying it. This will stall all the workers and it will deadlock.
So re-arrange the code to look like this
go func() {
for _, link := range urls {
lCh <- link
}
close(lCh)
wg.Wait()
close(outCh)
}()
for res := range outCh {
fmt.Printf("%s
", res)
}
And it will work fine
https://golang.org/ref/spec#For_range :
For channels, the iteration values produced are the successive values sent on the channel until the channel is closed. If the channel is nil, the range expression blocks forever.
You use range
before close outCh
. You have to close outCh
after wg.Wait()
, like in Nick`s answer.