Introducing Modularity and Creating Componentization

Problem

What is the holy grail of software development? In my opinion, it is modularity and componentization. Although modularization as a framework has been widely discussed and debated for the past 30 years, unfortunately, we, as practising software developers in big enterprises, still do not have the discipline to implement it. The biggest problem with code is that it rots. As the software community trailblazer with inventions and improvements, what is “new & innovative” today becomes “old & outdated” tomorrow. So, what ends up happening are nuclear rewrites: we abandon our existing codebases to adopt a better, newer system and spend a long time replicating existing functionalities under new frameworks. But this is extremely expensive both from a time and effort perspective.

Modularity allows for a more gradual and natural process to update code. Instead of massive rewrites and redesigns, we can rewrite only one module at a time and do it fast. We can do it without interfering with the current core functionality of systems. Modularity asl allows “safe experiments” - instead of blindly adopting a new library/framework, modularization allows us to first take it on a test ride with just a couple of modules and to explore advantages and promised benefits.

It sounds like a great vision, but executing this in real life is plain difficult. Many have failed. It requires a lot of time, high investment, long-term commitment and diligent discipline.

Actions taken

While conceptually "modularized approach to writing code" makes sense, from my experience, team capacity and existing roadmap commitments do not allow any bandwidth to even contemplate it as a building standard. For the past 2 years, my team has been gradually pursuing modularization, gradually breaking monoliths. While we have made some good headway, in my opinion, we still have a long way to go. Here is how we have approached it.

• We separated all our front-end code and created a component library with standard shared components.
• We separated all of the APIs and converged them into a cross-app API layer
• We have started converting the remaining backends into microservices
  

Separating “battle for modularization” into 3 separate initiatives allowed us to fight 3 different battles instead of one big “war for modularization”. Change is incremental. When organizing initiatives people try to tackle everything at once. Pacing ourselves on various projects had allowed us to gradually make improvements incorporating and adjusting to learnings over time.

Every initiative above also got its own name. Branding was important since it made the initiative more tangible and had allowed us to negotiate with our product team for small percentages of time to be allocated to these initiatives, instead of blocking large time frames and calling it “tech debt”.

Finally, having an evangelist for every initiative and having the team buy-in was critical for the success of every initiative. Top-down projects where people do not have emotional buy-in tend to die down. Making “modularization” a year goal did not get us anywhere. However, having a lead on every initiative and requesting people to contribute ensures that component libraries and API layers get built.

Lessons learned

• Break up big critical initiatives like “tech-debts” & “componentization” into specific projects to make slower but incremental and gradual progress over time.
• Change is inevitable. Modularity is just the benevolent way to embrace it.
• Nothing is permanent. Rebuilds will happen. Set up a mentality that whatever you have in hand may need to be thrown away. With that in mind, architect accordingly.
• For every initiative get a leader and evangelists and ask for teams to pitch in. People will support an initiative only when they are contributing to it.