I am very new to golang and I was trying out goroutine, while it's quite easy to run things concurrently, I am a bit surprised the way golang to "join the threads" using WaitGroup.
As far as I know, the goroutine needs to have reference to the WaitGroup object to call Done(), which means, I have to either make the goroutine to accept a WaitGroup object, or make WaitGroup object global to the goroutine.
But in other languages like Python, you call thread.join(), the "controlling" part sits outside of the thread code.
Like I said, I am very new to golang, I don't know why it was designed this way, could someone please shed some light on this aspect?
UPDATE: I hope the argument is not based on 'Goroutine vs Thread', at the end of the day they both try to achieve (some kind of) 'concurrency', my question is more about controlling the program flow.
No, it's just a different thing that does a different thing. They're not even really comparable, since a WaitGroup by its nature waits on multiple things (and can have things added to it during its lifetime) and a python thread's join always just waits on that one thing.
That said, Go's library is more about giving you the primitive things that you need to do more advanced things, while Python's has more of a "batteries included" philosophy. Using what Go gives you, you could create a type that acts quite a bit like a python Thread. It's probably not the best way to make use of Go, but you're given the tools to do it if you want. However the standard library isn't going to standardize on such a thing.
why it was designed this way
That's actually been explained many times by the golang team - why can't we kill goroutines, why doesn't they have an ID which we can read, why can't we wait for goroutine explicitly like with thread's Join.
It was explained multiple times but I could find only this. Basically, the authors didn't want you to depend on thread locality - to lock on a specific thread/goroutine, have a local storage only for it etc. When you don't have any means to know in which goroutine you're actually running you're forced to design your application in a truly concurent way. Your code is composed of truly independend pieces that run concurently and they don't care how exactly. You don't care which goroutine picks up your code, you don't care which OS thread is running your code. That's where channels, select and other primitives come in. They help you build your application in such a way. And I'm sure it doesn't stop there.
The answers from hobbs and creker especially are excellent, but I feel there's more to be said.
There's a very common notion that WaitGroup is the way to manage multiple goroutines - it certainly is commonly used and even idiomatic in a number of situations. And you know what? Being able to call thread.join() may indeed be superior to dealing with WaitGroups when just waiting on a bunch of threads/goroutines launched earlier.
But there's so much more to Go's concurrency model than that.
Goroutines were specifically designed to not have concepts of ownership or hierarchy or handles. They are independent, equal and responsible for ending their own execution. That, combined with strong concurrency primitives, gives Go's model almost unparalleled flexibility.
Therefore, if you find yourself using WaitGroups almost every time you're using goroutines, you're probably not taking advantage of concurrency in modeling and structuring your programs - it's more likely you're just using goroutines to parallelize computation.
To answer your question more directly, WaitGroups are rather primitive compared to stuff like thread.join(), but primitive, low-level building blocks are much more useful with Go's concurrency model. After all, goroutines are not threads, and they're not meant to be used exactly the same way.
As far as I know, the goroutine needs to have reference to the WaitGroup object to call Done(), which means, I have to either make the goroutine to accept a WaitGroup object, or make WaitGroup object global to the goroutine.
I disagree that WaitGroup is a step backward when compared to other languages. I really like its simplicity. However, I do agree that passing the WaitGroup object around would mean that the concurrency logic would be mixed with your business logic and I did not like that.
So I came up with this generic function to solve this problem for me:
// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
var waitGroup sync.WaitGroup
waitGroup.Add(len(functions))
defer waitGroup.Wait()
for _, function := range functions {
go func(copy func()) {
defer waitGroup.Done()
copy()
}(function)
}
}
Here is an example:
func1 := func() {
for char := 'a'; char < 'a' + 3; char++ {
fmt.Printf("%c ", char)
}
}
func2 := func() {
for number := 1; number < 4; number++ {
fmt.Printf("%d ", number)
}
}
Parallelize(func1, func2) // a 1 b 2 c 3
If you would like to use it, you can find it here https://github.com/shomali11/util