Sending big file with minimal memory in Golang

Most common way of uploading file(s) by http is splitting them in multiple parts (multipart/form-data), this structure helps greatly as we can also attach fields along and send them all in one single request.

A typical multipart request (example from Mozilla):

POST /foo HTTP/1.1
Content-Length: 68137
Content-Type: multipart/form-data; boundary=---------------------------974767299852498929531610575

-----------------------------974767299852498929531610575
Content-Disposition: form-data; name="description"

some text
-----------------------------974767299852498929531610575
Content-Disposition: form-data; name="myFile"; filename="foo.txt" 
Content-Type: text/plain

(content of the uploaded file foo.txt)
-----------------------------974767299852498929531610575--

We will start with this simple implementation, standard package mime/multipart got our back:

buf := new(bytes.Buffer)
writer := multipart.NewWriter(buf)
defer writer.Close()

part, err := writer.CreateFormFile("myFile", "foo.txt")
if err != nil {
    return err
}

file, err := os.Open(name)
if err != nil {
    return err
}
defer file.Close()

if _, err = io.Copy(part, file); err != nil {
    return err
}

http.Post(url, writer.FormDataContentType(), buf)

multipart.Writer will automatically enclose the parts (files, fields) in boundary markup before sending them ❤️ We don’t need to get our hands dirty.