This is homework and beginner question. I edited the question since I made a mistake that I found.
I am trying to do parallel frequency maps from a text and I get an error in the last operation (reducing maps)
The code seem to work up to this point.
If I close the channel I get error: "panic: send on closed channel"
If I do not close the channel I get: "fatal error: all goroutines are asleep - deadlock!"
func WordCount(text string) {
text = strings.ToLower(text)
re := regexp.MustCompile("\\w+")
sentence := re.FindAllString(text, -1)
numberOfGroups := 4
piece := len(sentence) / numberOfGroups
wordChannel := make(chan map[string]int)
wg := new(sync.WaitGroup)
wg.Add(numberOfGroups)
for i := 0; i < numberOfGroups; i ++ {
go processToCounting(sentence[i*piece:(i+1)*piece], wordChannel, wg)
}
wg.Wait()
fmt.Print(<-wordChannel)
fmt.Print("
")
finalMap := make(map[string]int)
close(wordChannel)
for i := 0; i < numberOfGroups; i++ {
for k, v := range <- wordChannel {
finalMap[k] += v
}
}
}
func processToCounting(textSlice []string, wordChannel chan map[string]int, wg *sync.WaitGroup) {
freq := make(map[string]int)
for _, v := range textSlice {
freq[v]++
}
wg.Done()
wordChannel <- freq
}
1. First question: panic
If I close the channel I get error: "panic: send on closed channel"
Why? One of you goroutines is trying to write to the channel which you already closed in the calling (main) goroutine. In your case in WordCount
function.
In the current version of your code the panic
is not reproducible with my test sentence, but you can easily cause this e.g. if you would call close(wordChannel)
before wg.Wait()
.
Lets look at the Bug in processToCounting
which may cause the panic
:
wg.Done() // tells to the WaitGroup that the gouroutine is Done (decrements the counter of goroutines)
wordChannel <- freq // trying to write to the channel
Here wg.Done()
signals to the WaitGroup
that the goroutine is Done
before actual writing to the channel has happened. The calling goroutine (WordCount
function) at some point thinks that all gouroutines are done (wg.Wait()
line) and closes the channel. But one of your goroutine which have not finished writing, will try to write to the closed channel. Then you will get the panic
.
How to fix:
Use defer
in processToCounting
function
defer wg.Done() // triggers wg.Done right before the function returns (but after writing to the channel in your case)
What to read:
See for beginners in A Tour of Go / Concurrency
Sending on a closed channel will cause a panic.
and documentation for: close
Sending to or closing a closed channel causes a run-time panic.
2. Second question: deadlock
If I do not close the channel I get: "fatal error: all goroutines are asleep - deadlock!"
You have a for-loop which reads from the channel. This for-loop is locked forever. Waiting for new values from the channel, but nobody will write there anymore.
See in A Tour of Go / Concurrency
The loop for i := range c receives values from the channel repeatedly until it is closed.
and see documentation for Channels
Receivers always block until there is data to receive
Q1: why close(wordChannel)
triggers a panic
A:
wg.Done()
before pushing result into the channelQ2: why there is a deadlock.
A: fmt.Print(<-wordChannel)
read a msg from the channel, so that the last loop can't read numberOfGroups
message. it waits for the last message forever.
for i := 0; i < numberOfGroups; i++ {
for k, v := range <- wordChannel {
finalMap[k] += v
}
}