I am getting a response back as well as header responses and everything, but for some reason the body is empty:
https://repl.it/repls/HastyAggravatingArchitect
package main
import (
"fmt"
"net/http"
"io/ioutil"
)
const url = "http://comicbookdb.com/search.php"
func main() {
request, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
fmt.Println(err.Error())
return
}
q := request.URL.Query()
q.Add("form_search", "Captain America")
q.Add("form_searchtype", "Character")
// http://comicbookdb.com/search.php?form_search=captain%20america&form_searchtype=Character
request.URL.RawQuery = q.Encode()
client := http.DefaultClient
response, err := client.Do(request)
fmt.Println(response.Header.Get("Date"))
fmt.Println(response.StatusCode)
fmt.Println(response.Header.Get("Server"))
fmt.Println(response.Body)
r, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(string(r))
}
I commented in the full URL so that you can check it out yourself and see that the response body should not be empty.
The problem is not in the go program, you need provide additional header in this case it's a Cookie header:
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
const url = "http://comicbookdb.com/search.php"
func main() {
request, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
fmt.Println(err.Error())
return
}
// Here is requeired header
request.Header.Add("Cookie", "PHPSESSID=jmujtqjctuk1bv1g02ni88q9u5'")
q := request.URL.Query()
q.Add("form_search", "Captain America")
q.Add("form_searchtype", "Character")
request.URL.RawQuery = q.Encode()
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println(response.Header.Get("Date"))
fmt.Println(response.StatusCode)
fmt.Println(response.Header.Get("Server"))
fmt.Println(response.Body)
r, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(string(r))
}
The output:
Mon, 09 Jul 2018 06:13:35 GMT
200
Apache
&{0xc420060040 {0 0} false <nil> 0x5e2200 0x5e2190}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org ...
..... omitted ....
It is something related to the page. You can get away with setting an additional header. In this case, Connection header.
...
request.Header.Set("Connection", "keep-alive")
client := http.DefaultClient
response, err := client.Do(request)
...