I am learning Go concurrency, and my expectation is that using goroutines and channels should increase the concurrency. The program takes a few milliseconds to complete. But as the load increases the execution time keeps increasing though there is good amount of CPU idle.
I am sending 1200 QPS/TPS to the program below to analyze the request to response times, and I see that overall execution time of the program increases over time. Also, the CPU usage is around 3-6%.
As I increase the QPS to 100,000, the execution time of the program increases to seconds (from milliseconds initially). But the CPU usage remains at 8-9%.
So why doesn't the program use the other 90-94% of the available CPU and complete execution of the program more quickly?
ulimit -n is 2000000.
package main
import (
"fmt"
"github.com/valyala/fasthttp"
"strings"
"sync"
)
func total(in chan int, out chan int) {
res := 0
for iter := range in {
res += iter
}
out <- res // sends back the result
}
func check() {
ch := make(chan int)
rch := make(chan int)
go total(ch, rch)
ch <- 1
ch <- 2
ch <- 3
close(ch) // this will end the loop in the total function
result := <-rch // waits for total to give the result
fmt.Println("Total is ", result)
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
m := func(ctx *fasthttp.RequestCtx) {
maingetpath := ctx.Path()
urlPart := strings.Split(string(maingetpath), "/")
func := urlPart[1]
switch string(func) {
case "white":
check()
default:
ctx.Error("not found", fasthttp.StatusNotFound)
}
}
fasthttp.ListenAndServe(":8080", m)
defer wg.Done()
}()
wg.Wait()
}
I am not sure exactly what you are trying to do in this little server.
1) You are creating a WaitGroup and adding 1 to it, calling a anonymous go routine, and waiting in main, this does nothing more then move your main to the anonymous function.
2) Lets look at what your are doing in the check and total functions:
func total(in chan int, out chan int) {
res := 0
// this will just pull the value, and then wait till the next value
// is pushed... till you close the "in" channel
for iter := range in {
res += iter
}
out <- res // sends back the result
}
func check() {
ch := make(chan int)
rch := make(chan int)
go total(ch, rch)
// we are pushing this value into a unbuffered channel...
ch <- 1 // this gets pushed and waits until it is pulled in the total function
ch <- 2 // this gets pushed and waits until it is pulled in the total function
ch <- 3 // this gets pushed and waits until it is pulled in the total function
close(ch) // this will end the loop in the total function
result := <-rch // waits for total to give the result
fmt.Println("Total is ", result)
}
Please help me understand how this would use any concurrency when it is completely synchronous?
Maybe if you put the call to check in a go routine it would be a little more efficient, but still really makes no sense to me.