Adapting Scrum to Continuous Deployment
17 February, 2021
Several years ago, I joined a development organization that operated on a very inefficient one-month sprint cycle and manual deployment process. This quickly made it the top of my list of things to improve. However, the existing leadership team was very risk-averse, felt the existing processes were good enough and did not want change.
Gathering data to support the changes
Undeterred, I observed the cycle for several months and documented the many pain points: urgent mid-sprint deploys for bug fixes, wasted developer downtime time at end of or in-between sprints, giant monolithic deploys that were hard to debug and rollback, error-prone four-day long manual QA regression, the pressure to hit sprint deadlines leading to painful last-minute shortcuts, a two hour, 25 people, cross-department pre-deploy “go/no-go” meeting and a manual, late-night deploy that took the product out of service. I took this laundry list of issues to the leadership team and got a conceptual agreement that the process needed to change, but no resources or official support.
Guerilla effort to demonstrate early results
Again, I was not deterred. I set a goal of enabling fully automated deploys at least once per day and created a roadmap to get there. I developed the first several milestones of the automated deploy pipeline myself on nights and weekends. At the same time, I worked with members of the QA team during their downtime at the start of each sprint to develop an automated QA regression test. I also developed and rolled out a distributed risk management process to eliminate the need for the 25 people “go/no-go” meeting.
The demonstrated value of the partially complete deploy pipeline and automated regression tests allowed me to elevate the automation project to an officially sponsored initiative. With the ability to commit dev teams to work on the project I was able to complete the deploy pipeline, extend the regression test suite and make the significant application changes required to allow a zero-downtime deploy.
Small experimentation and iteration
After achieving the goal of daily fully automated deployments, the heavy sprint cycle development process was the next bottleneck slowing us down. Over the course of a year, I experimented with process tweaks on a single team, rolling out each successful change to all the other teams. Eventually, the vast majority of the existing process was stripped away and replaced with a lighter weight process. This process, which we still use today, allows teams to deliver a predictable flow of work while retaining the ability to quickly react to production incidents and changes in project scope.
- Use data to get buy-in to change entrenched practices. While the solutions seemed obvious to me, data and demonstrated improvements were what really convinced leadership to back the project.
- Don’t settle when you believe you are right. Despite the lack of management support, I continued pushing the project because I believed in it. If I had given up, the organization would likely still be doing things the same, inefficient way.
- Start Small. When changing processes/tooling, test it out with the smallest unit you can (probably a team.) Iterate at that level and tune it before fanning out to a larger group.
Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader
Mike Weaver, VP of Engineering at Invoca, tells of his “guerilla” efforts to migrate from a traditional Scrum to a process more suitable to continuous deployment.
VP of Engineering at Invoca
Michael Smith, VP of Innovation at Hadean Supercomputing, meticulously dissects the history of agile methodology and critically examines its current, uncritical use.
VP of Innovation at Hadean
Graham McNicoll, CEO of Growth Book and former VP of Product and Engineering at Education.com, recalls how he introduced data-driven decision-making and AB testing to drive growth, and the unexpected effects it had on development speed, team alignment, intellectual humility, and collaboration.
VP Product and Engineering at Education.com
Peter Berg, Founder and CTO at Forward, describes how he helped ramp up a slow-moving team by applying his simple, yet expert approach.
Founder / CTO at Forward
Brian Guthrie, VP of Engineering at Meetup, shares how he learned the hard way that many Agile processes he eliminated had their second-order effects that the team was benefiting from greatly.
VP of Engineering at Meetup
You're a great engineer.
Become a great engineering leader.
Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.