How do I convert the below code to use streams/pipes so that I don't need to read the full content into memory? Something like: http.Get("http://example.com/").Pipe("./data.txt")
package main
import ("net/http";"io/ioutil")
func main() {
resp, err := http.Get("http://example.com/")
check(err)
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
check(err)
err = ioutil.WriteFile("./data.txt", body, 0666)
check(err)
}
func check(e error) {
if e != nil {
panic(e)
}
}
How about io.Copy()? Its documentation can be found at: http://golang.org/pkg/io/#Copy
It's pretty simple, though. Give it an io.Reader and an io.Writer and it copies the data over, one small chunk at a time (e.g. not all in memory at once).
So you might try writing something like:
func main() {
resp, err := http.Get("...")
check(err)
defer resp.Body.Close()
out, err := os.Create("filename.ext")
if err != nil {
// panic?
}
defer out.Close()
io.Copy(out, resp.Body)
}
I haven't tested the above; I just hacked it together quickly from your above example, but it should be close if not on the money.