With our backend endpoints and user authentication in place, it’s now time to revisit S3. Amazon’s simple storage service provides developers to store whatever they need in the cloud.
Our project utilizes multiple resources with the hope of staying on the free tiers for each. The frontend is hosted on Vercel, while our backend is running on my personal server here at home. Although it’s handy to have our backend hosted locally, I don’t want to see major delays in communication with the frontend.
In my limited experience using S3, I didn’t need to worry about delays since my code was running on the same domain. There are a few options I’ve come across to handle file uploads to S3.
Use Multer to stream file to S3 bucket:
This is the traditional approach that I have used before. It requires reading the file contents, storing the data in a temporary location on the server, and streaming the file to S3. Before handling the file, you can easily check for a valid JWT in the request headers. If all of our code was running on my server, this is probably the solution I would use.
Pre-signed POST to upload directly from client-side:
With this option, the client sends a request for an “upload URL,” which is “pre-signed” with the necessary data (AWS credentials, file field requirements, user authentication, etc.) for the file to be uploaded directly to S3. With this approach, our backend would not be dealing with file streaming. This also allows for finer control of restricting the file size and type.
The downside to this approach is the higher level of complexity. Amazon Lambda functions and the use of Amazon’s API Gateway are often referenced in tutorials and guides. It may be possible to use our server to pre-sign a post for upload to S3, but with only a few more weeks left in the quarter, this option could become too time-consuming.
Pre-signed URLs???
This approach is very similar to a pre-signed POST in that the client has to request access to upload a file to S3. Pre-signed URLs use a PUT request to S3 but follow a similar pattern of requesting the URL from your own server or through API Gateway and Lambda functions. With Pre-signed URLs, you lose flexibility and some security. You can’t check the file size, and once the URL is generated, anyone can access it and use it multiple times until the URL expires. Implementing this option appears to be much simpler than the pre-signed POST.
I’ll need more time to research these options before implementing file upload routes. Our front end uses the NextJS framework, which I have not had as much time to work with so far. The built-in server functionality with NextJS could be the best way to handle file uploads to S3!