Category: Capstone Project

With our backend endpoints and user authentication in place, it’s now time to revisit S3. Amazon’s simple storage service provides developers to store whatever they need in the cloud.

Our project utilizes multiple resources with the hope of staying on the free tiers for each. The frontend is hosted on Vercel, while our backend is running on my personal server here at home. Although it’s handy to have our backend hosted locally, I don’t want to see major delays in communication with the frontend.

In my limited experience using S3, I didn’t need to worry about delays since my code was running on the same domain. There are a few options I’ve come across to handle file uploads to S3.

Use Multer to stream file to S3 bucket:

This is the traditional approach that I have used before. It requires reading the file contents, storing the data in a temporary location on the server, and streaming the file to S3. Before handling the file, you can easily check for a valid JWT in the request headers. If all of our code was running on my server, this is probably the solution I would use.

Pre-signed POST to upload directly from client-side:

With this option, the client sends a request for an “upload URL,” which is “pre-signed” with the necessary data (AWS credentials, file field requirements, user authentication, etc.) for the file to be uploaded directly to S3. With this approach, our backend would not be dealing with file streaming. This also allows for finer control of restricting the file size and type.

The downside to this approach is the higher level of complexity. Amazon Lambda functions and the use of Amazon’s API Gateway are often referenced in tutorials and guides. It may be possible to use our server to pre-sign a post for upload to S3, but with only a few more weeks left in the quarter, this option could become too time-consuming.

Pre-signed URLs???

This approach is very similar to a pre-signed POST in that the client has to request access to upload a file to S3. Pre-signed URLs use a PUT request to S3 but follow a similar pattern of requesting the URL from your own server or through API Gateway and Lambda functions. With Pre-signed URLs, you lose flexibility and some security. You can’t check the file size, and once the URL is generated, anyone can access it and use it multiple times until the URL expires. Implementing this option appears to be much simpler than the pre-signed POST.

I’ll need more time to research these options before implementing file upload routes. Our front end uses the NextJS framework, which I have not had as much time to work with so far. The built-in server functionality with NextJS could be the best way to handle file uploads to S3!

Last week our Capstone groups and project were assigned. We will be developing a cross-platform app to help improve the concept of Craigslist. As I’ve spent time researching possible technologies to use, I keep asking myself, “what happened to Craigslist?”

I wouldn’t consider myself an active user of Craigslist. Still, back in the mid-2000s, when I was working on my first bachelor’s degree, Craigslist was the best way for a broke college student to furnish their room. I also found and adopted, Zoey the first dog I raised as an adult, through a local shelter’s foster program. My personal experience using Craigslist has been positive! I felt it necessary to do a little research on Craiglist’s history and learn more about their competitors.

The Craigslist controversy:

Before I continue, let’s acknowledge a common sentiment regarding transactions on Craigslist; be careful! That was a valid concern even in Craigslist’s early days, given the anonymity of listings. Craigslist’s free and straightforward community service hoped to bring out the best from people. But with limited or no moderating, scammers could create listings and successfully deceive a buyer with no consequence.

One interesting fact that I don’t think is well known is that Craigslist is a private for-profit company.^[1] As they expanded to cover more cities and countries, they kept their same ideals and simplistic website design. They refused to profit from ads and other marketing strategies and dismissed outside developers looking to collaborate on new features.^[2] With millions of users on their site, it’s not surprising to learn that Craigslist has faced numerous legal battles. The most controversial was their personals category, which they shut down after the FOSTA-SESTA bill was passed into law in 2018.^[1]

Who is the competition?

Facebook Marketplace:

I admittedly don’t use Facebook often and have never used their marketplace. I can imagine the appeal of connecting to people through Facebook. Users can view the profiles of potential buyers or sellers, which takes away some of the concerns users of Craigslist face.

As I browsed through listings on Facebook Marketplace, it didn’t feel like a local community classifieds website. Even with my location filter set, I see more sponsored ads and listings from businesses than individuals. If I wanted to find something to buy, I wouldn’t enjoy sifting through ads, and I bet sellers have a difficult time connecting with buyers as well.

Shifting towards cross-platform solutions:

OfferUp and Mercari are Craigslist competitors that chose to pursue the rising popularity of mobile devices to create cross-platform apps.

Mercari is a Japanese company that became Japan’s largest community-powered marketplace.^[3] They have expanded to the U.S. and have gained a small foothold. They also have developed a shipping protocol to allow users to explore beyond their local community.

OfferUp, located in Bellevue, Washington, is similar to Mercari but focuses more on safety concerns. OfferUp created user profiles with detailed rating systems to help hesitant buyers. They also formed partnerships with local police stations and businesses to give their users well-monitored places to meet for transactions.^[4]

So how can I improve Craigslist?

If you visit Craigslist now, you won’t see anything new. Select a category or search for an item; you will encounter an unorganized mess of posts and come across obvious spam that hasn’t been flagged by a user or removed by a Craigslist employee. Developing a user profile with a rating system would significantly decrease spam and provide a more personal touch to users conducting transactions. Developing a more intelligent search algorithm and creating a modern UI will also be beneficial.

As Craigslist’s competitors have done, making a mobile app in addition to a website will appeal to a broader audience. However, those apps exist to profit from their user base. I think Craigslist has the right idea about not profiting from its users. I will need to research frameworks and free database and hosting services to keep my project free to use and free from ads.

The final quarter:

If someone told me I’d soon be on a trajectory to begin a new career in computer science back in 2018, I doubt I would have believed them. To be clear, the thought had definitely crossed my mind during my time working as a scientist. I had tried working through some beginner-level coding coursework over the years, but it was far more overwhelming than anything else I had learned.

By the end of spring in 2019, I had drafted up an academic plan with an advisor at my local community college and let my boss know I’d be resigning that summer. My initial plan was to take the few classes I needed to complete a transfer degree and apply to a local university’s computer science program. But the COVID-19 pandemic led me down a different path; an online computer science post-baccalaureate degree at Oregon State University.

Despite my initial concerns with learning complex material strictly online, I found myself progressing more quickly and with a deeper understanding of the material than I expected. I started exploring new technologies beyond the scope of my coursework and would constantly be thinking through possible solutions to problems in my head while taking breaks from classwork. For the first time, I felt confident that I was headed down the right career path.