Note: Firefox Send was archived by Mozilla in September 2020, but as an open-source project, the source code was left available. We've based our integration on the Send fork maintained by Tim Visée.
In this tutorial, we're looking at how we integrated IBC S6 secure object storage as the back-end storage for the open-source file-sharing application, Firefox Send.
Firefox Send is a free, end-to-end encrypted file-sharing application developed by Mozilla, that allows users to easily and safely share files over the Web. The Send back-end is written in Node.js, allowing us to integrate with the Ionburst Cloud Node.js SDK.
Digging into the Send source code
From a cursory review of the source code and running the application locally, it looked like the focus of our integration would be the server directory, which contains the code for Send's back-end services.
Of particular interest was the storage sub-directory, which contains the functionality for integrating Send with the following:
- Local filesystem storage;
- Google Cloud Storage;
- Amazon S3;
A review of these files outlined common pieces of functionality expected from storage integrations:
length– returns the object size from the configured storage method;
getStream– retrieves the object from the configured storage method;
set– uploads or writes the object to the configured storage method;
del– removes the object from the configured storage method;
At time of integration, IBC S6 provided functionality for three out of four, as it did not expose the ability to query object size. Further digging suggested that the
length function was being used to set the
Content-Length header on the file download response to the user, so wasn't a functional requirement for the storage integration.
Since we did this original integration work, we've added a HEAD API method, which can be used to query the size of objects stored in IBC S6.
Exploring the storage sub-directory also confirmed how Send handles object metadata. In ‘development', Send uses a local, in-memory store to track each object, but is designed to use Redis in production. To gain a better understanding of the Send application, all of our integration work was carried out using Redis as the Send metadata store.
The final checks required were to see how the Send back-end handled storage configuration. The base back-end configuration is handled in the
config.js file found in the server directory, which defines the storage method selected by the
index.js file found in the storage sub-directory.
Integrating IBC S6 - Configuration
To begin Integrating IBC S6 with Send, we first had to add new configuration options to the Send project so it could use IBC S6 as the new storage method, along with the initial Ionburst Cloud SDK configuration.
The Ionburst Cloud SDK was added to the project using npm:
A local Redis instance was deployed to track Send metadata using Docker:
config.json file was added to the root of the Send project for the Ionburst Cloud SDK configuration file.
A new configuration item was then added to the Send
config.js file for IBC S6. Note: this configuration entry is only used to select IBC S6 as the chosen back-end storage, and does not perform any other configuration. The
redis_host entry was also adjusted to
127.0.0.1 to override the local memorystore:
A configuration option was added to the storage
index.js file, to ensure IBC S6 was selected as the storage method:
ionburst.js file was added to the storage sub-directory, and a constructor created for applicable configuration:
Integrating IBC S6 – File Operations
IBC S6 PUT
From the storage
index.js file, we can see how Send kicks off a file upload to its configured storage:
In this snippet, we can see that Send generates an identifier for each file stored, before passing it and the file to the configured storage method. As IBC S6 has no preference to how a given object is identified, we can simply pass this identifier to IBC S6 too.
To upload the file to IBC S6, the following function was created in
We encountered some issues passing the file object directly to the Ionburst Cloud SDK. To overcome this, we instead leveraged the existing filesystem functionality to write the file to a temporary directory, create a read stream for the Ionburst Cloud SDK, then remove the temporary file after successful upload.
This temporary file/directory leveraged functionality used by Send's file-system storage, and it was simply a matter of pulling the temporary directory configuration into the Ionburst storage constructor:
IBC S6 GET
Similar to the upload function, the main download functionality can be found in the storage
To keep things simple, we replicated the same temporary file functionality for the file download from IBC S6:
We first grab the file from IBC S6, write it to the temporary directory, then create and return a read stream.
IBC S6 DELETE
Send requires delete functionality from the configured storage method to remove uploaded files once they have reached their download limit, or expiry time.
The IBC S6 delete function was simple to implement:
At the time of integration, IBC S6 had no method of returning a stored object's size, nor did the Send metadata store it. As IBC S6 now has a
HEAD API method, this can be added to the implementation to pass the
Content-Length header on the file download reponse.
Depending on the deployment, Send can handle files up to 2.5GB. IBC S6 currently supports a maximum object size of 50MB, with larger objects requiring client-side processing before upload.
As a simple proof-of-concept, we've kept this 50MB limit in place for our fork of Send. However, since the time of integration, we've started to add our new SDK Manifests feature, which allows the Ionburst Cloud SDK to handle objects larger than 50MB. Once manifests have been added to our Node.js SDK, this functionality will be added to our Send fork.
All in all, integrating Firefox Send with IBC S6 was a relatively quick and simple process, providing us an opportunity to try out our Node.js SDK in an exisiting application. To try our Send fork for yourself, please check out our Getting started with Send and IBC S6 and Secure file-sharing with IBC S6, Firefox Send and the AWS free tier tutorials.
The full project source for our Send fork can be found here.