System Design - an Engineer’s Art Form

What do you think of when you see a software engineer?

You will mostly likely think of them as the person implementing features in a mobile app or creating amazing UI interfaces or a wizard who is able to conjure software out of thin air. Something that isn’t talked about a lot in the popular culture of software engineering is System Design.

What is system design?

System design is the process of designing components for a specific purpose like a web app. You can imagine that it is similar to a building architect. Architects need to consider how all the components of a house work together, from the walls to the plumbing. All these decisions on how the house should be structured is what an architect thinks about. This process is similar to system design in software. A software architect needs to consider where the customers will use their app (web or mobile), how data is stored and how information is communicated between different components.

So why system design?

System design is the process in which software engineers build components to solve complex problems and it is not something that is often practiced. Coding interviews usually focus on the skill component of engineering like “do a binary search” or “find the shortest path between s-t”. It trains up the ability of an engineer to find an algorithm to a precise and well-defined problem.

Translating to the real world, not all problems are clearly defined. Even the “simplest” idea of building a fully functioning to do web app requires detailed planning from designing the UI to storing task data in a database. And most importantly, there is never a truly “perfect” way to do it. There are always trade offs like cost, speed, scalability and time to market.

This is what makes system design, an art form. Engineers need to creatively navigate through the fog of technical obstacles and unseen tools and come up with an elegantly designed system to service an app.

How to learn system design?

Like any skill, the more you train it, the better you get. System design is the exact same thing. Designing more systems and understanding the trade offs between each design will help you become better at it. 

Let’s walk through what it is like doing system design from the ground up.

So the goal here is to create a fully functioning to do app. Some basic requirements include ability to create and delete tasks and also remember what task I’ve done before and be able to see it at anytime.

Let’s look wide first. This app’s purpose really boils down to saving and showing information, thus, it must need a frontend layer, which contains all the nice buttons and card components to show our tasks. But we also need to save our information, thus, we need some kind of database to store our tasks. We’ll put this first iteration and see how it is.

First iteration of our to do app (but it is not the best)

This is a relatively simple architecture right now. Do you see any flaws?

Flaws of the first iteration. Components are too coupled together.

Simple is one of its key advantages but thinking ahead, any business logic that we put into this will be tightly coupled with the frontend. This will most definitely make your frontend unnecessarily complex and also supporting different platforms like mobile will require a whole rewrite of the business logic. Thus, we cannot couple business logic inside of the frontend. So how about we abstract that logic to another layer and call it the backend?

Now this looks a lot cleaner! The frontend will communicate with the backend and it will handle all the business logic like information retrieval and handling requests for task creation. Finally the database will store all the information required.

And that’s is an high level of how engineers approach system design. Obviously, there are many other factors to consider such as tech stack, cloud hosting, database design and so on. And that is the beauty of system design. There are so many moving components that does into making software work and when an engineer is able to make it work, it is truly a piece of art.

---

I would recommend taking a read at 12 factor app. This should plant some ideas on what are some aspects of system design to consider and new concepts to explore such as CI/CD.