使用systemd（无限无止境地）转到应用正常启动

I want to do real graceful restart for Go application, while still being able to control it with systemd. I'm using endless in the app. The code is trivial (endless.ListenAndServe(":80", nil)), and for the systemd config file, so far I managed to make it like this (goapp.service):

[Unit]
Description=Go Application
After=network.target
After=syslog.target

[Install]
WantedBy=multi-user.target

[Service]
User=root
Group=root
WorkingDirectory=/root/go/src/myproject
ExecStart=/root/go/src/myproject/goapp
KillSignal=SIGHUP
Restart=on-failure

Then I just start the app with systemctl start goapp, everything's fine, but when I issue systemctl restart goapp while giving the application tons of requests at the same time, the restart command seems to hang my terminal (and not kill/fork the process) until it's free from requests. Of course, when there are no requests, the service restarts successfully.

I'm testing with this simple code:

package main   

import "net/http"
import "github.com/fvbock/endless"

func handler(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello"))
}

func main() {
    http.HandleFunc("/", handler)
    endless.ListenAndServe(":80", nil)
}

Could anyone help with this? What am I doing wrong?