Creating multiple goroutines which will have nested goroutines while processing in a multilevel manner (Imagine a tree of goroutines each level can have many leaves).
What is the idiomatic way to gracefully shutdown these goroutines in order and wait for them to come back? Order is the bottom top (deepest child first) and also assume I dont know how many goroutines I will launch beforehand (dynamic).
the example below just gracefully shuts them down in an non ordered manner.
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
//level1
go func() {
fmt.Println("level1 started")
//level2
go func() {
fmt.Println("level2 started")
//level3
go func() {
fmt.Println("level3 started")
select {
case <-ctx.Done():
fmt.Println("Done called on level3")
case <-time.After(5* time.Second):
fmt.Println("After called on level3")
}
}()
select {
case <-ctx.Done():
fmt.Println("Done called on level2")
case <-time.After(7* time.Second):
fmt.Println("After called on level2")
}
}()
select {
case <-ctx.Done():
fmt.Println("Done called on level1")
case <-time.After(10* time.Second):
fmt.Println("After called on level1")
}
}()
time.Sleep(1*time.Second)
cancel()
time.Sleep(1 * time.Second)
}
To wait for a group of goroutines, sync.WaitGroup is the idiomatic solution. You can add 1 to its counter when you launch a new goroutine (WaitGroup.Add()), and the goroutine can signal that it's done with WaitGroup.Done(). The parent goroutine may call WaitGroup.Wait() to wait all its children to finish.
You may do the same on each level. Create a WaitGroup on each level where child goroutines are launched, and only return when Wait() of that goroutine returns.
Here's how it's applied on your example:
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
//level1
wg1 := &sync.WaitGroup{}
wg1.Add(1)
go func() {
defer wg1.Done()
fmt.Println("level1 started")
//level2
wg2 := &sync.WaitGroup{}
wg2.Add(1)
go func() {
defer wg2.Done()
fmt.Println("level2 started")
//level3
wg3 := &sync.WaitGroup{}
wg3.Add(1)
go func() {
defer wg3.Done()
fmt.Println("level3 started")
select {
case <-ctx.Done():
fmt.Println("Done called on level3")
case <-time.After(5 * time.Second):
fmt.Println("After called on level3")
}
fmt.Println("Level 3 ended.")
}()
select {
case <-ctx.Done():
fmt.Println("Done called on level2")
case <-time.After(7 * time.Second):
fmt.Println("After called on level2")
}
wg3.Wait()
fmt.Println("Level 2 ended.")
}()
select {
case <-ctx.Done():
fmt.Println("Done called on level1")
case <-time.After(10 * time.Second):
fmt.Println("After called on level1")
}
wg2.Wait()
fmt.Println("Level 1 ended.")
}()
time.Sleep(1 * time.Second)
cancel()
wg1.Wait()
fmt.Println("Main ended.")
This outputs (try it on the Go Playground):
level1 started
level2 started
level3 started
Done called on level1
Done called on level3
Level 3 ended.
Done called on level2
Level 2 ended.
Level 1 ended.
Parent ended.
What's important from the output:
Level 3 ended.
Level 2 ended.
Level 1 ended.
Main ended.
Levels end in descending level order (from bottom-up), closing with "Main ended.".
One possible, also I'd say idiomatic, way to do this is by passing a channel of strict{}. Whenever you'd like the said goroutine to terminate just write an empty struct to this channel: shutdown <- struct{}{}. This should do the job. Alternatively you could close the channel, you'll recognise this by having false as the second return value of <-, but I'd suggest using this only if you need to share this channel with multiple goroutines. In general I find this approach a bit shoddy and error prone.
On a side note: the way the shutdown of the goroutines is done in your example, once the context has been cancelled, all goroutines will return. Don't know whether this has much benefit in the general case. Maybe in your case it does.