I'm letting users upload a file using FormFile. At what point should I check if the file size is too large. When I do
file, header, fileErr := r.FormFile("file")
A file object is already created. So have I incurred the cost of reading in the entire file already?
You have r.ContentLength int64 field in request struct and r.Header.Get("Content-Length") string method. Maybe that can help.
Calling FormFile calls ParseMultiPartForm, which will parse the entire request body, using up to 32M by default before storing the contents in temporary files. You can call ParseMultiPartForm yourself before calling FormFile to determine how much memory to consume, but the body will still be parsed.
Th client may provide a Content-Length header in the multipart.FileHeader which you could use, but that is dependent on the client.
If you want to limit the incoming request size, wrap the request.Body with MaxBytesReader in your handler before parsing any of the Body.
Use MaxBytesReader to limit the size of the request. Before calling ParseMultiPartForm or FormFile, execute this line:
r.Body = http.MaxBytesReader(w, r.Body, max)
where r is the *http.Request and w is the http.Response.
This limits the size of the entire request body and not an individual file. If you are uploading a single file at a time, limiting the size of the request body should a good approximation of limiting the file size.
If you want to limit the amount of memory used instead of the request body size, then call r.ParseMultipartForm(maxMemory) before calling r.FormFile(). This will use up to maxMemory bytes for file parts, with the remainder stored in temporary files on disk.
Some people are suggesting to rely on Content-Length header and I have to warn you not to use it at all. This header can be any number because it can be changed by a client regardless of the actual file size.
Use MaxBytesReader because:
MaxBytesReader prevents clients from accidentally or maliciously sending a large request and wasting server resources.
Here is an example:
r.Body = http.MaxBytesReader(w, r.Body, 2 * 1024 * 1024) // 2 Mb
clientFile, handler, err := r.FormFile(formDataKey)
if err != nil {
log.Println(err)
return
}
If your request body is bigger than 2 Mb, you will see something like this: multipart: NextPart: http: request body too large