I need to get only a first few lines of a http.Get(<url>) request in go lang.Is there any way to do it.I don't need the entire response. I need to match a pattern which will be there only in the first few lines. Fetching the entire response and getting the regular expression is making it very very slow because the number of characters in the response are very huge.Which is the fastest way to do it. Currently i am doing
http.Get(<url>)
// returns a response
ioutil.readAll(response of the above url)
// returns a byte array
MyFunction(search for reg exp in the byte array)
Is there any method to fetch a few lines or avoid fetching the entire response and match the pattern.
Don't use ReadAll, just read a chunk you know will be large enough to capture the part of the request you want
// response must be in the first 2048 bytes
buf := make([]byte, 2048)
n, err := resp.Body.Read(buf)
MyFunction(buf[:n])
Or via an io.LimitedReader
buf, err := ioutil.ReadAll(io.LimitReader(resp.Body, 2048))
Or if you really need to read via lines, a bufio.Scanner
scanner := bufio.NewScanner(resp.Body)
for i := 0; i < fewLines && scanner.Scan(); i++ {
MyFunction(scanner.Bytes())
}
Note however if you don't finish reading the response Body, it will prevent your client from reusing connections, and if there is a large amount of incoming data the server may not like you frequently closing the connection early.
If you know how much you need to read from the response, you can use a LimitReader (https://golang.org/pkg/io/#LimitedReader), otherwise, the easiest way would be a make a read (or a Scanner) to consume your body until you finish your patterns.