There’s quite a moment happening right now in a certain quarter of the internet. For those who care, there is a lot of focus right now on “measuring developer productivity” McKinsey started the discussion with their paper, “Yes, you can measure software developer productivity.” Lots of folks are weighing in with takes on the topic, including this interesting one from The Pragmatic Engineer. Alas, once again, they all fall victim to the same fallacy as the drunk who keeps looking for his keys under the streetlamp because “that’s where the light is.”
The issue with the McKinsey paper is its failure to reckon with Goodhart’s law - whenever a measure becomes a target, the measure loses value. This law is especially true in the context of software development. If you haven’t ever noticed, software developers tend to be exceptional at solving puzzles to maximize a certain value metric. I’m stereotyping a little bit here, most of the time they have been trained to do so with thousands of hours of game playing since they were kids. Jumping straight to developer productivity within an opaque and poorly understood SDLC workflow promises to create a Hall of Fame example of Goodhart’s law in action.
This observation becomes particularly impactful when we think of the impact of Taylorism on the business process of software development. Frederick Taylor taught us to measure every part of a process if we want to understand the output of the process. In essence, by rooting out waste at each step of a process, we could optimize the process as a whole.
Taylorism (and later relatives like Lean and Six-Sigma) worked great in a manufacturing process, the specifications for the widget we want to build are well known and uncontroversial, and the challenge focuses around the question of “did we build the widget right?”
Software doesn’t work that way. In contrast to traditional manufacturing, when building software, building the same widget over and over again (rather than reusing objects) is about the worst form of waste imaginable. Each unit of work is fundamentally novel, with at best hazily defined parameters.
It will simply never work to take a Taylor inspired approach and fixate on “are we building the widget right” in software. Compounding that with a violation of Goodhart’s law to fixate on intermediate measures that lose value when they become targets only makes it worse. It leads you to a place where everyone is arguing over story points, DORA and SPACE metrics while 78% of software is late, over budget, or doesn’t ship at all.
Yes, the SDLC is a business process. Yes, it can absolutely be measured and improved. But, we must do so not from the starting premise of “are we building the widget right”... we have to adjust our thinking to focus the process optimization on the question of “are we building the right widget?”
The vast majority of waste comes upstream and downstream of engineering in the SDLC. An amazing development team downstream of poorly written requirements will rarely produce a work product that creates real business value.
So how do we improve the process?
Here's a step-by-step approach to improving the process holistically:
- Focus on the Entire Process Flow: Avoid the temptation to concentrate on specific functional areas. Isolate the inefficiencies by examining the entire process.
- Ensure Process Observability: Ensure your team comprehends how work is prioritized, requirements are gathered, designs are validated, and edge cases are tested in QA. Visibility into the entire process is crucial.
- Measure Process Adherence: Evaluate how well the team adheres to the established process. The output of your process is often a reflection of how well it's followed.
- Assess Process Predictability: Once there's adherence to the process, assess its predictability. How often does the team meet their commitments to the business, and vice versa? Predictability is key to effective collaboration.
- Optimize for Efficiency: After achieving observability, adherence, and predictability, you can start optimizing for efficiency. Focus on eliminating issues like discovering design flaws only after development or, worse, during production, as these are significantly more costly to fix than during the design phase
