Among the development projects in CS3216 so far, this is my most balanced team. I finally have a designer on my team and among the developers, we have someone familar with frontend work, instead of having only developers who are more familar with the backend as was the case in my first assignment group. Given the complementary skill sets of this team, I’m hopeful that I’ll finally have something that I’ll be proud of.

For the final project, we will be working with external partners on an idea that they proposed - an app that aggregrates posts from multiple social media sites of an online store and displays selected reviews from customers in a widget in the online store. I like the project for its potential “Machine Learning extension” - developing a classifier that could isolate reviews from the rest of the social media posts and classify them as either “positive” and “negative” reviews. At first glance, this looked like a project that was more demanding on the backend, especially if we go through with the implementation of the classifier. However, while discussing the features of our app with the team, I realised that designing a good dashboard for the display of our users’ aggregated social media posts is no trivial matter. For example, to avoid having a very cluttered inbox of posts where it’ll be very hard to find a post of interest as the number of aggregated posts grows, I proposed having user-defined labels so users can organise their aggregated posts. However, we soon ran into the problem of how we should display the ui components for that (the label selector/creator + the display of the labels and filter for labels themselves) without cluttering our existing ui.

For the tech stack that we will be working on, our external partners decided that we should use reactjs for the frontend, with the backend running on meteorjs. Our partners suggested using MongoDB as the initial database - I can see why, MongoDB is really easy to setup and meteorjs’ default database connection is to MongoDB. However I’m not really comfortable with having MongoDB as the database as it doesn’t scale well and will definitely have to be switched when the application’s user base grows. Also, it is a pain when you have data with more complicated relationships. MongoDB is awesome when you are just literally caching documents - you can store the documents and retrieve them as they are. But once you have relations between data, it quickly breaks down. Nested documents are a bad idea, you can’t query on fields in nested documents without running through all the documents in the collection then retrieving the nested documents. If you have documents in different collections referencing the same nested document type, then you would have to isolate the nested document as a collection of its own anyway. In the end, I forsee a schema that closely mirrors a relational one anyway, just that MongoDB is a huge pain to work with for relational data - there is no equivalent, so you can’t retrieve data from multiple collections in a single query. The only thing MongoDB has over a relational db is it can store arrays as they are, which might be convenient for us if we are to implement labels for our aggregated posts. So in our meeting with our external partners this week, I would suggest that we use Postgres instead. The downside is more work has to be done to setup the database, finding a reliable library for meteorjs that handles the connection to the database…

Regardless of the database that we settle on in the end, I propose that we use an ORM to convert our in-memory data to our actual database representation so it’s easier to switch the database in future, barring data migration. The problem is it seems like there doesn’t seem to be an ORM library for meteorjs. Our best bet seems to be to adapt an existing nodejs ORM library, like waterline, and manually wrap it with Fibers so it works with meteorjs…