Structuring Using Tools, Measurements, and a Spreadsheet of Expectations

Problem

The company that I am currently working for grew very quickly. We went from 150 employees to 600 in a short time and so we did not have in place a solid structure. About a year and a half ago, though, there became a need for a better structure, better guidelines, and better ways to measure our team members.

Actions taken

It was a complete team effort to standardize the structure. I contributed specific pieces but I certainly cannot take credit for the whole thing. We began by implementing more tools to get more metrics. For example, we measure technical debt, the amount of time it takes developers to code, the quality and efficiency of the code, etc. By having these tools and concrete metrics, we now had a solid way to track our engineers performance.

Based off of those measurements we created a spreadsheet that included the expectations for each of the various levels of engineering. We wrote out, level by level, what we were looking for if you were in the junior, middle, or senior level categories. We listed out soft skills, technical skills, communication, and strategy. Now, it was clear what we were asking for and what steps to take in order to move on to the next level. Additionally, we can see the health of our engineers in terms of efficiency, activity, how much technical debt they have introduced, how many tickets they've worked on, and the average estimate for those tickets. All of these tools and processes fit together to give structure and confidence in articulating where the engineer stands, the directions they can go, and areas that need improvement.

All of this was part of Phase I. Our next phase, Phase II, will have more to do with items that are not measurable using tools but aspects that our company definitely puts emphasis on. These aspects include collaboration, communication, attitude, and reliability.

Lessons learned

• First of all, you need some kind of structure. It's impossible to simply say that you think an engineer is doing great. This is a non-effective subjective view. Thus, you need some way of measuring so that there is congruence throughout your organization.
• Next, be careful of how and when you rollout your new structure. It takes time to put these new frameworks in place and to be sure that they are running properly. You can't just roll it out and set it in stone. You need first to get feedback, do a couple of test runs, and then open it up company-wide. Even then it may not be perfect. In fact a testament to this is that we currently are still modifying and tweaking our structure.
• Lastly, the numbers that we measure are used to tell a story. There are no right or wrong numbers but they do give a narrative to telling the truth. For example, if I see my top engineer go from 80% efficiency to 40% efficiency during a release, then I can ask what happened. What did they work on? And really get down to the root cause of the fluctuation in numbers. The numbers provide background and present dialogue to a story. It's very useful for us to know those stories and use them to adjust so that we can do better overall. Better not just for the individual engineers, but also for planning and testing as well. It gives us a 360 degree view.