UPD: Refactored code, nothing change
I'm have memory leak in this function but i don't know where.
func CheckProxySOCKS(prox string, c chan QR) (err error) {
//Sending request through proxy
dialer, _ := proxy.SOCKS5("tcp", prox, nil, proxy.Direct)
timeout := time.Duration(5 * time.Second)
httpClient := &http.Client{Timeout: timeout, Transport: &http.Transport{Dial: dialer.Dial}}
res, err := httpClient.Get("https://api.ipify.org?format=json")
if err != nil {
c <- QR{Addr: prox, Res: false}
return
}
_, err = ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
return
}
c <- QR{Addr: prox, Res: true}
return
}
Here i called it
for _, proxy := range splitedProxies {
go code.CheckProxySOCKS(proxy, respChan)
}
for range splitedProxies {
r := <-respChan
if r.Res {
checkedProxiesArray = append(checkedProxiesArray, r.Addr)
}
}
After 3-4 cycle i'v got more than 40k goroutines(i'v check it by runtime.NumGoroutine()
). After start app used around 100mb after 4 cycle more than 1GB
Github repo with all code
Your problem lies in not closing the response body, in this line:
_, err := httpClient.Get("https://api.ipify.org?format=json")
get the response variable and close the body in defer, like this:
r, err := httpClient.Get("https://api.ipify.org?format=json")
if err != nil {
c <- QR{Addr: prox, Res: false}
return
}
defer r.Body.Close()
c <- QR{Addr: prox, Res: true}
in the docs The client must close the response body when finished with it:
here
You're not always closing the connection, and you're using a new Client and Transport every request. If you discard the transport, you discard any connections in the idle pool, leaking those resources.
From the http.Transport
docs
By default, Transport caches connections for future re-use. This may leave many open connections when accessing many hosts. This behavior can be managed using Transport's CloseIdleConnections method and the MaxIdleConnsPerHost and DisableKeepAlives fields.
Transports should be reused instead of created as needed. Transports are safe for concurrent use by multiple goroutines.
Always close the response body when there is no error, and always reuse the transport.