I need to write a tcp server which can handle more than 500k connections.
I wrote a simple server on golang, but when connections more than 10k, the server crashed with the error message "runtime: program exceeds 10000-thread limit fatal error: thread exhaustion".
Server runs in last linux build. That go can use epoll for tcp connection, to make it async and use few fd. So why does the server exceed the thread limit?!
My simple server:
package main
import (
"strconv"
"net"
"log"
"time"
"bufio"
"io"
)
type Handler struct {
conn net.Conn
closed chan bool
}
func (h *Handler) Listen() { // listen connection for incomming data
defer h.conn.Close()
bf := bufio.NewReader(h.conn)
for {
line, _, err := bf.ReadLine()
if err != nil {
if err == io.EOF {
log.Println("End connection")
}
h.closed <- true // send to dispatcher, that connection is closed
return
}
// ... some business logic with data
}
}
type Dispatcher struct {
handlers map[string]*Handler `map[ip]*Handler`
}
func (d *Dispatcher) AddHandler(conn net.Conn) {
addr := conn.RemoteAddr().String()
handler := &Handler{conn, make(chan bool, 1)}
d.handlers[addr] = handler
go handler.Listen()
<-handler.closed // when connection closed, remove handler from handlers
delete(d.handlers, addr)
}
func (d *Dispatcher) ListenHandlers(port int) {
sport := strconv.Itoa(port)
ln, err := net.Listen("tcp", ":" + sport)
if err != nil {
log.Println(err)
return
}
defer ln.Close()
for {
conn, err := ln.Accept() // accept connection
if err != nil {
log.Println(err)
continue
}
tcpconn := conn.(*net.TCPConn)
tcpconn.SetKeepAlive(true)
tcpconn.SetKeepAlivePeriod(10 * time.Second)
go d.AddHandler(conn)
}
}
func main() {
dispatcher := &Dispatcher{make(map[string]*Handler)}
dispatcher.ListenHandlers(3000)
}
Update 11.10.2015
The problem was that https://github.com/felixge/tcpkeepalive library. Please do not use it :)
The problem was that https://github.com/felixge/tcpkeepalive library. This library get socket file descriptor (fd), copy then and worked with new fd, which running in blocked mode. Therefore, go created new thread on every connect. Go don't kill created thread (because this is native golang behavior), and after some times go failed with "Exceeds 10000-thread". That's it ! Please do not use it :)