The code is like this:
func find(start int, end int){
for i := start; i < end ; i++ {
// Do something...
}
}
func main() {
// Do something...
// For example, I know: len = 1500
// and run max goroutine are 3
// will be
go find(0, 500)
go find(500, 1000)
go find(1000, 1500)
// Do something...
}
That is when I know in advance the maximum threads of goroutines and the length of "length". But if I do not know how many threads can run at goroutine, and the length of "length". Is there any way to divide "length" into equal sections for thread processing? For example: length = 10, max goroutine that can run is 2 and it will split length into 2 threads (10/2, each length is 5) to be able to handle simultaneously.
I know I've seen and read references on job scheduling strategies, but I can't find a good one right now. But you've got some basic options:
All of these work in Go. To find the number of real threads the Go runtime has available, look at https://golang.org/pkg/runtime/#GOMAXPROCS
Also, just from experience, channels are tempting. But for best performance don't send single work items in a channel. Send array / slices of work items in each message instead. Otherwise the channel's locking overhead can become more time consuming than the actual work.
Maximizing throughput is about getting rid of bottlenecks. First of all find where is most time is lost.
Sometimes running too many goroutines doesn’t make things faster but makes them slower, because goroutines start to compete for resources. Usually the fastest configuration is one that uses resources in the most effective way: if things are limited by computation then it's best to set the number of running goroutines equal to the number of CPU cores. But if it is limited by IO then choose the optimal number by measuring its performance.
Find the main thing that is slowing down your program - then make it faster.
The best way I can think of would be a fixed size worker pool of goroutines, as described in https://gobyexample.com/worker-pools