Im trying to create goroutines for a http call. I have huge number of URL's.
I create a batch of 200 and send a batch request to the api.
Question:-
When I create the client object - client := &http.Client{}, I see a pointer variable is created, will it create a new one for each and every iteration of the for loop? How can I check if the client object created is unique or its pointing to the same one?
When I run this program with 250 batches I don't see a issues but once I increase to 270 it fails with an error. Error is given below, Not sure if this is memory error or program error.
for _, batch := range data_batch {
var str_join string = strings.Join(batch, ",")
// fmt.Println(str_join)
str_join = "{" + "\"inputs\":[" + str_join + "]}"
input_json_data := []byte(str_join)
var output core.Data
go http_call(respond, &wg, input_json_data, output)
}
func http_call(respond chan<- core.Data, wg *sync.WaitGroup, input_json_data []byte, output core.Data) {
defer wg.Done()
// clarifai http request URL
httpurl := os.Getenv("api_url")
//Create the client Object
client := &http.Client{}
//generating the HTTP GET request
request, err := http.NewRequest("POST", httpurl, bytes.NewBuffer(input_json_data))
if err != nil {
log.Println(err)
}
// setting authorization
api_key := "Key " + os.Getenv("api_key")
request.Header.Add("Authorization", api_key)
// setting content-type
request.Header.Add("Content-Type", "application/json")
//calling the URL
response, err := client.Do(request)
if response.StatusCode != 200 {
if err != nil {
log.Println(err)
}
log.Println("statuscode = ", response.StatusCode)
}
bodyBytes, _ := ioutil.ReadAll(response.Body)
bodyString := string(bodyBytes)
err = json.Unmarshal([]byte(bodyString), &output)
if err != nil {
print(err)
}
respond <- output
}
Error:-
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0xb5593]
goroutine 261 [running]:
github.com/p/go/utils.http_call(0xc820440d80, 0xc8204317a0, 0xc82052b730, 0x68, 0x70, 0x0, 0x0, 0x0, 0x0, 0x0, ...)