mikesukmanowsky.com

File and Image Uploads with Express and Firebase Cloud Functions

March 29, 2019

I recently spent longer than I’d care to admit trying to figure out how to properly do file or image uploads with Express and Firebase Cloud Functions.

If after reading this you’re still stuck, grab an hour of my time and we’ll figure it out together:

book a meeting

TL;DR do not try and use Multer for this. As it turns out, Cloud Functions introduced a breaking middleware for Multer which parses the body of the request. The result is that Multer will always return a blank req.body, req.file and req.files.

So what are your options? Google documents them both here, but let me save you a click:

  1. Use busboy to directly parse multipart form-data and do something with it (sample code).
  2. Upload directly Google Cloud Storage directly using Signed URLs (sample code)

My solution was to create and use a middleware based off the code samples Google had that make things feel more Multer-y.

// middleware.js
exports.filesUpload = function(req, res, next) {
// See https://cloud.google.com/functions/docs/writing/http#multipart_data
const busboy = new Busboy({
headers: req.headers,
limits: {
// Cloud functions impose this restriction anyway
fileSize: 10 * 1024 * 1024,
}
});
const fields = {};
const files = [];
const fileWrites = [];
// Note: os.tmpdir() points to an in-memory file system on GCF
// Thus, any files in it must fit in the instance's memory.
const tmpdir = os.tmpdir();
busboy.on('field', (key, value) => {
// You could do additional deserialization logic here, values will just be
// strings
fields[key] = value;
});
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
const filepath = path.join(tmpdir, filename);
console.log(`Handling file upload field ${fieldname}: ${filename} (${filepath})`);
const writeStream = fs.createWriteStream(filepath);
file.pipe(writeStream);
fileWrites.push(new Promise((resolve, reject) => {
file.on('end', () => writeStream.end());
writeStream.on('finish', () => {
fs.readFile(filepath, (err, buffer) => {
const size = Buffer.byteLength(buffer);
console.log(`${filename} is ${size} bytes`);
if (err) {
return reject(err);
}
files.push({
fieldname,
originalname: filename,
encoding,
mimetype,
buffer,
size,
});
try {
fs.unlinkSync(filepath);
} catch (error) {
return reject(error);
}
resolve();
});
});
writeStream.on('error', reject);
}));
});
busboy.on('finish', () => {
Promise.all(fileWrites)
.then(() => {
req.body = fields;
req.files = files;
next();
})
.catch(next);
});
busboy.end(req.rawBody);
}

Similar to Multer, this middleware will ensure req.body are any text fields in form-data and req.files is an array of uploaded files.

const express = require('express');
const { filesUpload } = require('./middleware');
app = express();
app.post('/upload', filesUpload, function(req, res) {
// will contain all text fields
req.body
// will contain an array of file objects
/*
{
fieldname: 'image', String - name of the field used in the form
originalname, String - original filename of the uploaded image
encoding, String - encoding of the image (e.g. "7bit")
mimetype, String - MIME type of the file (e.g. "image/jpeg")
buffer, Buffer - buffer containing binary data
size, Number - size of buffer in bytes
}
*/
req.files
})
exports = app;
view raw usage.js hosted with ❤ by GitHub

Hope this helps but if you find that you’re still stuck by anything above, you can grab an hour of my time and we’ll figure it out together:

book a meeting


Mike Sukmanowsky

Written by Mike Sukmanowsky. A product manager and programmer who lives and works form home in Mississauga, Ontario, Canada. Follow him on Twitter