I'm trying to download only the first 1kb of large files using http.Client
in go, but it seems like response.Body
is always fully buffered. Is there a control over how much to buffer?
If so, how can this be used with app engine urlfetch service?
The following works fine with app engine urlfetch in python, and I'm trying to port this to go:
from urllib2 import urlopen
req = Request(url)
urlopen(req).read(1024) # Read the first 1kb.
Setting "Range" Request Header attribute to "bytes=0-1023" should work.
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET", "http://example.com", nil)
req.Header.Add("Range", "bytes=0-1023")
fmt.Println(req)
var client http.Client
resp, _ := client.Do(req)
fmt.Println(resp)
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(len(body))
}
Same thing should work for http.Client provieded in app engine.
There is also io.LimitReader() in io pkg, which can make limited reader
package main
import (
"io"
"io/ioutil"
"net/http"
)
func main() {
var client http.Client
responce, _:=client.Get("http://example.com")
body:=responce.Body
chunk:=io.LimitReader(body, 1024)
stuff, _:=ioutil.ReadAll(chunk)
//Do what you want with stuff
}