James Shore: A Useful Productivity Measure?

🌈 Abstract

The article discusses the challenges of measuring productivity in software engineering and the author's approach to addressing this issue as a VP of Engineering. It explores the concept of "value-add capacity" as a potential productivity metric and the risks associated with using such a metric.

🙋 Q&A

[01] A Useful Productivity Measure?

1. What is the author's main challenge as the VP of Engineering regarding measuring productivity? The author is dreading the question "How are you measuring productivity?" as software productivity is famously unmeasurable, with experts like Martin Fowler and Kent Beck stating that it is not possible to measure developer productivity.

2. How did the author approach the question of measuring productivity with the CEO and other leadership team members? The author led an exercise with the CEO, CTO, and CPO to imagine what the best product engineering organization would look like and what indicators could help understand how they are getting closer to those ideals. The author avoided using "productivity" as one of the categories.

3. What was the company-wide OKR (Objectives and Key Results) related to productivity, and how did the author have to present his productivity metrics? The company had established an OKR to define and improve productivity metrics for each department, and the author had to present his productivity metrics to the full leadership team at the end of April.

[02] The Productivity OKR

1. What were the three indicators the author's team defined for the "profitability" aspect of being the greatest product engineering company? The three indicators were actual ROI, estimated ROI, and value-add capacity.

2. How does the author explain the concept of "Product Bets" and how it could provide a true measure of productivity? The author explains that the "Product Bets" process, where each major initiative needs a proposal with details on what will be accomplished, the estimated value, and how it will be measured, could provide a true measure of productivity by knowing the value of a bet and the cost to implement it.

3. Why was the author unable to use the ROI-based metrics initially, and what metric did the author end up using instead? The author couldn't use the ROI-based metrics initially because they were still rolling out the "Product Bets" process and didn't have any data yet. Instead, the author used the "value-add capacity" metric, which measures the percentage of engineering time spent on value-add activities.

[03] A Better Measure of Productivity... For Now

1. How did the author present the "value-add capacity" metric to the leadership team, and how did it change the conversation? The author presented the value-add capacity metric as a stacked bar chart, showing the breakdown of time spent on muda (non-value-add activities) versus value-add work. This shifted the conversation from "how can we get the stuff we want sooner" to "how can we decrease muda and spend more time on value-add work?"

2. What is the author's goal for improving the value-add capacity metric, and why does the author consider it to be a "fatal flaw" of the metric? The author's goal is to double the value-add capacity over the next three years. However, the author recognizes that the metric has a "fatal flaw" because when a measure becomes a target, it ceases to be a good measure, as it can be easily manipulated or cheated.

3. What does the author consider to be the most important things to focus on, rather than just measuring productivity? The author emphasizes that the real focus should be on delivering valuable software often and writing it well, rather than just obsessing over productivity metrics.