The Evolution of an Enterprise’s Architecture

Problem

Over the past decade, I've worked on many enterprise software architectures. Many engineering architects I've worked with or managed struggle with how to prioritize their time. Should I focus on the scale? Cost efficiency? Or execution speed? Long-Term maintainability or testing? There isn't a perfect answer to these questions. In my personal experience, the right architecture becomes evident when teams focus on customer experience. Customers ultimately leverage your service, and it's important to consider this when thinking about time to value for the customer versus long-term technical debt impacts. No engineer will ever build the right solution for all phases of the company's growth. As a result, architecture must evolve with the customer and the company. A typical startup goes through many phases, sometimes repeatedly,   before it becomes a Megacorp. Below I have identified four critical phases that impacts how architectural decisions are made.

Actions taken

-> Phase 1 — No product

Business strategy at this point is delivered by market research, some intuition, and intimate interactions with potential customers.

Architecture — it's all Proof of Concept or "POC":

• Small development teams — mainly founders.
• The technology stack is natural to code languages like Python, Ruby, or JavaScript.
• Public cloud with easy-to-use APIs are leveraged.
• To narrate your mission/story, it's essential to have the right UX architecture.
• Move fast. The backend can be driven by google sheets and manual email flows!

-> Phase 2 — Minimum Viable Product or "MVP"

It's in the best interests of the company to get out of POC as fast as possible and launch an MVP to capture real/more customer feedback in production. Customer feedback will allow you to drive a longer-term strategy and grow your customer base. Architecture 

• The team should be composed of more generalists than specialists.
•  Move fast. Collecting customer feedback is more important than fixing all P2/P3 bugs. Most of the code from this stage gets rewritten in future phases.
• Don't code/maintain anything that is not the USP of the business. Stay away from native integrations (like Salesforce). Use out of the box solutions like MuleSoft when possible.
• Use Looker and Mixpanel to capture customer interaction/journey.

-> Phase 3 — Product-market fit

In this phase, your sales team should be showing signs of repeatability and customers are driving 60% of your new development. Your customers are renewing, and you are adding new logos at a more than sustainable rate. Architecture

• Agility over scalability. Forex: Using DB for batching and queuing is faster than using Kafka; Python over spark.
• Iterations should be quicker.
• Collecting customer feedback on the feature-by-feature basis remains the priority.
• Identify the platform's weaknesses with out of the box monitoring and alerting. Vital at this stage is getting to the realization of what's not scaling or working so that you can build the future roadmap. Some tools to start leveraging:  Splunk, New Relic, Sentry, PagerDuty. I find it helpful to have Slack integration along with the above tools.
•  DevOps process  - different environment for Prod and QA. Tools like Ansible provide proper tooling to automate task around code and config deployment.

-> Phase 4 — Growth and Scale

A product with the right market fit now needs to meet the scale of hundreds or thousands of customers. Customers expect a reliable, always-on product with robust SLAs and performance. Sales & marketing are driving key company decisions, existing customer feedback is building a backlog of features, and continued innovation in a defined market are driving your development. You wish you had infinite engineers to satisfy customer demand. Engineering teams usually consist of multiple remote locations for healthy recruiting and efficient operations. Architecture 

• Reliable and always-on. Stress on the scalable, resilient, and cost-effective platform. Frameworks like serverless and microservices are good options to consider. Netflix has a good post on designing Microservices.
• The horizontal scale should be the norm for any service,  including databases. Both POD and Shard architecture works well, with some exceptions.
• Utilize scalable public cloud features for 99.999 reliability. Try not to reinvent the wheel. Engineers should spend more time with business than worrying about the scale. At Radius we adapted AWS Lambda and Kinesis which yields good results so far concerning reliability and scalability.
•  Start using code quality tools like SonarQube for continuous software quality and security. Integrate with Git flow for fast feedback to developers.
• E2E test with CI/CD process becomes critical for bug-free releases and good development velocity. It guarantees good uptime for customers.
•  Formal RnD org structure with right ownership and expectations set to make sure customers are always well supported.
• Mature monitoring with custom alerts  -  new relic insights or DB queries. Alarms and monitors get triaged and prioritized against the existing backlog.
• DevOps gets mature with fully automatic code deployments, rollbacks, and disaster recovery. A thoughtful strategy needs to be put in place to avoid longer maintenance windows or unplanned downtime. For example, Blue Green DB deployments, Elastic search snapshots, etc.
• Agile should be the development norm. All work should be captured by Jira epics, stories and tasks. Project scoping and backlog grooming should be done consistently to focus on the right priorities. Robust remote culture:  demo days, Slack channels (over email) and wiki documentation.

Lessons learned

• Architecture is not just about the choices between the different tools available in the market-  Kinesis vs. Kafka- or the need of any one customer. The right architecture should be a reflection of the overall company mission and more importantly the realization of at which stage your company's life you are in.
• Most enterprise software companies misinterpret the Scale stage for MVP or Product-Market-Fit stage. Coming to understanding where you are in your lifecycle will allow you and your team to justify the architectural decisions that need to be made. Source: https://medium.com/@abambha/ingredients-for-a-successful-architecture-ef24a7a8786b