How to handle crushes in a waitgroup gracefully?
In other words, in the following snippet of code, how to catch the panics/crushes of gorourines invoking method do()
?
func do(){
str := "abc"
fmt.Print(str[3])
defer func() {
if err := recover(); err != nil {
fmt.Print(err)
}
}()
}
func main() {
var wg sync.WaitGroup
for i := 0; i < 1; i++ {
wg.Add(1)
go do()
defer func() {
wg.Done()
if err := recover(); err != nil {
fmt.Print(err)
}
}()
}
wg.Wait()
fmt.Println("This line should be printed after all those invocations fail.")
}
First, registering a deferred function to recover should be the first line in the function, as since you do it last, it won't even be reached because the line / code before the defer
already panics and so the deferred function does not get registered which would restore the panicing state.
So change your do()
function to this:
func do() {
defer func() {
if err := recover(); err != nil {
fmt.Println("Restored:", err)
}
}()
str := "abc"
fmt.Print(str[3])
}
Second: this alone will not make your code work, as you call wg.Defer()
in a deferred function which would only run once main()
finishes - which is never because you call wg.Wait()
in your main()
. So wg.Wait()
waits for the wg.Done()
calls, but wg.Done()
calls will not be run until wg.Wait()
returnes. It's a deadlock.
You should call wg.Done()
from the do()
function, in the deferred function, something like this:
var wg sync.WaitGroup
func do() {
defer func() {
if err := recover(); err != nil {
fmt.Println(err)
}
wg.Done()
}()
str := "abc"
fmt.Print(str[3])
}
func main() {
for i := 0; i < 1; i++ {
wg.Add(1)
go do()
}
wg.Wait()
fmt.Println("This line should be printed after all those invocations fail.")
}
Output (try it on the Go Playground):
Restored: runtime error: index out of range
This line should be printed after all those invocations fail.
This of course needed to move the wg
variable to global scope. Another option would be to pass it to do()
as an argument. If you decide to go this way, note that you have to pass a pointer to WaitGroup
, else only a copy will be passed (WaitGroup
is a struct
type) and calling WaitGroup.Done()
on a copy will not have effect on the original.
With passing WaitGroup
to do()
:
func do(wg *sync.WaitGroup) {
defer func() {
if err := recover(); err != nil {
fmt.Println("Restored:", err)
}
wg.Done()
}()
str := "abc"
fmt.Print(str[3])
}
func main() {
var wg sync.WaitGroup
for i := 0; i < 1; i++ {
wg.Add(1)
go do(&wg)
}
wg.Wait()
fmt.Println("This line should be printed after all those invocations fail.")
}
Output is the same. Try this variant on the Go Playground.
@icza did a fantastic job explaining how to appropriately use WaitGroup
and its functions Wait
and Done
I like WaitGroup
simplicity. However, I do not like that we need to pass the reference to the goroutine because that would mean that the concurrency logic would be mixed with your business logic.
So I came up with this generic function to solve this problem for me:
// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
var waitGroup sync.WaitGroup
waitGroup.Add(len(functions))
defer waitGroup.Wait()
for _, function := range functions {
go func(copy func()) {
defer waitGroup.Done()
copy()
}(function)
}
}
So your example could be solved this way:
func do() {
defer func() {
if err := recover(); err != nil {
fmt.Println(err)
}
}()
str := "abc"
fmt.Print(str[3])
}
func main() {
Parallelize(do, do, do)
fmt.Println("This line should be printed after all those invocations fail.")
}
If you would like to use it, you can find it here https://github.com/shomali11/util