mikesukmanowsky.com

File and Image Uploads with Express and Firebase Cloud Functions

March 29, 2019·5 min read

I recently spent longer than I'd care to admit trying to figure out how to properly do file or image uploads with Express Firebase Cloud Functions.

TL;DR do not try and use Multer for this. As it turns out, Cloud Functions introduced a breaking middleware for Multer which parses the body of the request. The result is that Multer will always return a blank req.body, req.file and req.files.

So what are your options? Google documents them both here, but let me save you a click:

  1. Use busboy to directly parse multipart form-data and do something with it (sample code).
  2. Upload directly Google Cloud Storage directly using Signed URLs (sample code)

My solution was to create and use a middleware based off the code samples Google had that make things feel more Multer-y.

// middleware.js
exports.filesUpload = function(req, res, next) {
// See https://cloud.google.com/functions/docs/writing/http#multipart_data
const busboy = new Busboy({
headers: req.headers,
limits: {
// Cloud functions impose this restriction anyway
fileSize: 10 * 1024 * 1024,
}
});
const fields = {};
const files = [];
const fileWrites = [];
// Note: os.tmpdir() points to an in-memory file system on GCF
// Thus, any files in it must fit in the instance's memory.
const tmpdir = os.tmpdir();
busboy.on('field', (key, value) => {
// You could do additional deserialization logic here, values will just be
// strings
fields[key] = value;
});
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
const filepath = path.join(tmpdir, filename);
console.log(`Handling file upload field ${fieldname}: ${filename} (${filepath})`);
const writeStream = fs.createWriteStream(filepath);
file.pipe(writeStream);
fileWrites.push(new Promise((resolve, reject) => {
file.on('end', () => writeStream.end());
writeStream.on('finish', () => {
fs.readFile(filepath, (err, buffer) => {
const size = Buffer.byteLength(buffer);
console.log(`${filename} is ${size} bytes`);
if (err) {
return reject(err);
}
files.push({
fieldname,
originalname: filename,
encoding,
mimetype,
buffer,
size,
});
try {
fs.unlinkSync(filepath);
} catch (error) {
return reject(error);
}
resolve();
});
});
writeStream.on('error', reject);
}));
});
busboy.on('finish', () => {
Promise.all(fileWrites)
.then(() => {
req.body = fields;
req.files = files;
next();
})
.catch(next);
});
busboy.end(req.rawBody);
}

Similar to Multer, this middleware will ensure req.body are any text fields in form-data and req.files is an array of uploaded files.

const express = require('express');
const { filesUpload } = require('./middleware');
app = express();
app.post('/upload', filesUpload, function(req, res) {
// will contain all text fields
req.body
// will contain an array of file objects
/*
{
fieldname: 'image', String - name of the field used in the form
originalname, String - original filename of the uploaded image
encoding, String - encoding of the image (e.g. "7bit")
mimetype, String - MIME type of the file (e.g. "image/jpeg")
buffer, Buffer - buffer containing binary data
size, Number - size of buffer in bytes
}
*/
req.files
})
exports = app;
view raw usage.js hosted with ❤ by GitHub

Hope this helps someone!


Mike Sukmanowsky

Written by Mike Sukmanowsky, a product manager and programmer who lives and works from home in Mississauga, Ontario, Canada. Follow him on Twitter.