Day 23 | #100DaysOfCode
Learning System Design
To Build scalable systems
Complexity kills! It sucks the life out of developers, it makes products difficult to plan, build, and test!
Example 1: Can a single person/body be responsible for everything? The cities have various bodies who are responsible for managing various parts of the city. Some focus on the details, while others focus on a high level.
Cities work because they have evolved appropriate levels of abstraction and modularity that make it possible for individuals and the “components” they manage to work effectively, even without understanding the big picture.
Example 2: A single CEO alone is not responsible for running an established company, he cannot be responsible for marketing, building the website, reaching out to customers, taking care of transportation, logistics, accounting. He is responsible for taking high-level decisions that will be fruitful for the company, manage the shareholder’s interests. He will be delegating various managers to get the job done in various domains.
The separation of concerns is one of the oldest and most important design techniques in our craft.
The software systems should separate the startup process: when the application’s objects are created and dependencies are wired together from the runtime logic that takes over after startup.
Most applications don’t separate the two logics in the rush of coding and hence mix both of the logics. A classic example,
public Service getService() {
if (service == null)
service = new CustomServiceClass(...);
return service
}We call this lazy instantiation which has a lot of benefits of its own like avoiding the overhead of instantiating some expensive objects over and over again and just reuse the existing ones. We are also ensuring that null is never returned.
We now have a hard-coded dependency on CustomServiceClass and if we don’t have its implementation, we can’t compile the code even though, we might not be using getService() at all.
Testing can be a problem as well if your required object is heavy. You will need to make sure that the expensive object is instantiated before calling the test methods on the core logic.
Note here you have to worry about both the instantiation logic and the runtime logic violating the Single Responsibility Principle. Also, you don’t know whether the object implementation is correct if you haven’t tested it before.
We should modularize this process separately from the normal runtime logic and we should make sure that we have a global, consistent strategy for resolving our major dependencies.
That’s where Dependency Injection comes in for your help! Do check out this story to learn more about Dependency Injection. I am not gonna cover that here.
Scalability
Back to the example of constructing a city, when your city is actually a small town the roads would have been small and as the city grows road improvement projects have to be put in place and we all think why couldn’t they have made a 6 lane highway earlier? Because that’s the only way things happen. You don’t expect and want to build a 6 lane highway through a small town initially. That’s just not required and is overkill.
Similarly, In software system design, you cannot get the architecture right the first time. You should implement whatever are your requirements today along with maintaining good separation of concerns.
Then as the requirement grows, we refactor and expand the system to implement new stories tomorrow. This is the essence of iterative and incremental agility. We achieve this by TDD, Refactoring and Clean Code as they keep our codebase easily scalable.
Building Big Designs upfront is harmful because you design a system that is not very adaptive to changes. You have already made a lot of effort finalizing things for the “Bigger” picture hence when the small-small changes are needed to be added over time, to accumulate to a big picture, it will be hard for you to do so.
All the big websites in today’s world started small, completing a user-story at a time while maintaining decoupled classes made those systems scalable and performant.
Hence, don’t go into the wildlife exploration of your software system without a compass thinking that it would scale itself in the future. You have to be responsible for maintaining the ability of your system to change.
Decision Making
Its best to give responsibility to the most qualified person, we all know that! But it is also best to postpone decisions until the last possible moment. This isn’t lazy or irresponsible.
- It allows us to have more time to make informed choices
- With the best possible information as we have most of the time to collect and process information
- We give more time to get and understand customer requirements and feedbacks
Remember, a premature decision is made with sub-optimal knowledge.
How the construction industry build amazing infrastructures in all sorts of weather and geographic conditions? They have matured over time and have a standardized set of methods and tools which they use for building which have evolved over time.
Similarly, defining standards for building your software systems is a good thing. But don’t get so obsessed with some standards so that you forget to deliver value to your customer.
Standards are made to help make it easy to do a job like working on an idea, recruitment of good talent, wiring different components of the system together.
However, the process of creating standards can take too long for the industry to wait, and some standards get outdated as per the latest need of the industry while the adopters might still be trying to adopt.
