The Hidden Value of Agile Processes

Problem

Some years ago, it became increasingly popular to deconstruct Agile and not use Scrum or XP anymore and go for more streamlined, slimmed down, lean style process innovations. It was becoming more accepted to skip stand-ups or sprint planning and apply the continuous planning process. I was hoping that in doing that, I could keep my team processes as lightweight as possible. However, by doing so I failed to acknowledge the hidden benefits of Agile processes.

Actions taken

I walked into the team and told them that we wouldn’t do regular spring planning every morning anymore; instead, every single day at stand-up we would continually re-plan and reprioritize and take whatever time we needed to ensure that we were doing the right thing. I told the team that we wouldn’t be bothered by estimating stories anymore because it was a waste of time.

I sat down and compared the cycle time for a story and anticipated the future work based both looking at estimates and just by counting stories and I discovered that I got the same result. I could just count stories and get the same level of predictability that I used to from asking my engineers to add estimates to the stories. If my stories were sufficiently small I could dump the estimates.

We intentionally deconstructed and moved away from both of these processes. That would help our team streamline and move faster and we would spend more time on hands-on work and less time on meetings.

Over time our cycle times got longer and longer and stories would drag in unexpected ways. We would walk into a stand-up and someone would say that they were still working on the same thing they were working on the week before. Because everyone was always busy I was not as careful as I should have been about check-in and things would drag on and on until I realized one day that an initiative we budgeted for two weeks had taken two and a half months. Though it was only one person and we were still making progress elsewhere I felt terrible. I was not managing the situation as closely as I should have. That person didn’t want to do a bad job, they kept on finding more work that had to get done.

What I found was that meetings and processes I had eliminated were serving a purpose that was not clear to me. I felt I was not doing an effective job and I decided to bring back an estimation meeting. Our estimates didn’t get any better or more accurate, but everything else got better -- our stories got better, people started to talk about architecture more, etc. The value that the estimates had for my team wasn’t that they were helping me with predictability, but that they were giving engineers a venue for pushing back on stories, discussing architecture or connecting over what the future approach for a particular project should be. It wasn’t about the estimates -- it was a technical alignment meeting.

The value that estimates provide is to quantify something fuzzy and what I wanted was for people to go through the quantification process to critically engage with something fuzzy.

Now I ask my teams to do the estimates, but I make it very clear to them what I am using them for. I don’t expect them to stick to an estimate, I am not judging a team on the basis of velocity, but I want us to run through that process. I don’t want them to overthink it, but if our numbers are different I want to discuss it because it is a discussion that is valuable, not a number itself. In addition, it provides a forcing function for us to go back to our stakeholders in the business and product and have a valid argumentation.

We finally brought back sprint planning meetings as well. It was not that we couldn’t replan and reprioritize day-to-day, but we needed a cadence for asking ourselves not only if we were planning a week correctly, but if we were working on the most important thing. We had to take a step back and look at things more strategically. Sprint planning is more about understanding why you need something, establishing a cadence and being honest about the value of the process, than about the planning itself.

Lessons learned

• Understand the second-order effects of the processes that you are using and factor those into the work that you do. Before you eliminate a process, understand what are the side effects involved. If you can gain those side effects without burning people with meetings, do so.
• Do experiment with processes. As with any other experiment, limit the scope of those experiments, try to find a way to limit the cycle time of your experiment. My mistake was that I led my experiments for too long before thinking critically if they were working.
• As with any other process, don’t follow it blindly and understand that the value a process is adding may not be immediately clear. I am all for stripping down the process to its core and understanding what makes it tick, but if you are following a process without an understanding of what is its hidden value, you will always be annoyed that you don’t know where you are getting that value from.