3 metrics to help product managers function in an engineer’s world

Nidhi Sapra | Mar 02 2016

Product managers can easily piss off their team's engineers by breathing down their necks to manage product releases without listening and understanding the world where the code lives. It's easy to turn into one of those nagging product managers without realizing it. So, how do you become an empathetic product manager running a cross-functional team - without letting anything fall through the cracks? Data and metrics can come to your rescue - if used well.

Product managers often perform analytics on user data and track metrics for their products. It is equally important to have metrics to track your team's performance. Without real data, you run the risk of setting unrealistic deadlines and expectations, thus damaging team morale and productivity. I've found the following three metrics are the most useful for keeping track of the speed of development and quality of software code.

Team velocity

Tracking team velocity can help you understand how fast your engineers can develop and deliver features, turn around times for your designer to develop and deliver wireframes, and how well your designers and engineers can collaborate with each other. My team runs in an agile environment, and we follow an iterative development methodology. We use Pivotal Tracker to write user stories, attach wireframes, define non-functional requirements, assess engineering efforts, and add bugs. We use Tuesdays as our iteration/sprint planning day and our sprints are planned weekly. We spend 15-20 minutes daily, grooming our stories so that each team member inputs their edits before the day of the iteration planning meeting (IPM). At the end of the week, we track our velocity and compare it to previous sprints. If the velocity drops, it could be because of various reasons, such as:

  1. Your engineers could not spend the usual hours on development because of external factors. These external factors may include time spent in too many meetings or stress in their personal lives.
  2. Wireframes attached by UX did not have all functional and non -functional requirements, and several changes had to be made after the iteration/sprint planning meeting
  3. Engineers underestimated the effort required to develop each story correctly or may have had to spend a lot of time squashing bugs.

Additional note on bugs: My team does not have a quality assurance tester, so as a product manager, I test each story before accepting it. At the same time, I constantly check the software for bugs. If too many bugs resurface every week (on an average, more than 5), I have a conversation with my team about why older functionalities, styling, and so on, are breaking with release of new features.

Test coverage

My team strives for healthy levels of test coverage without getting insanely obsessive about it. For the uninitiated, Test Driven Development (or TDD) is a methodology of development where you write and run a set of tests before you write code. Then, your code is simply functioning to get all the tests to pass. Ideally, having all your tests pass is the metric that gives you confidence in the code's quality. Code Climate is a great platform to track test coverage.

Much has been written about advantages and disadvantages of TDD. It may be tough to implement in the beginning since these tests are hard to write and it slows down development, thus not always yielding best results for rapidly iterative startup environments. However, on the plus side, TDD makes documentation of code and collaboration easier. Team members and new engineers can edit each other's code with confidence because they edits are informed by tests. In the long run, development is faster since a code written in the past can be refactored easily.

We set up regular test and code reviews with senior engineers to make sure the tests are well-written, that we are testing something worthwhile, and the less-experienced engineers are kept up to date on best practices. I use Code Climate to keep track of our coverage. When it falls below our goal, I consult with the senior engineer and the team to understand if it can be brought back up with reasonable effort and without slowing down development.

Grade Point Average (GPA)

Code Climate also calculates another score called GPA. Much like college GPA, Code Climate uses four-point GPA system as a way to indicate the overall code quality of a repository. If your team's GPA is lower than average, it may be because of bad practices in code writing such as:

  1. DRY (Do not Repeat Yourself) - if the engineers are repeating the same function, it should be wrapped into one thing, or else it results in confusing code.
  2. Cyclomatic Complexity - imagine a flowchart with many correlations when the same chart can be represented in a much simple fashion. Similarly, it is best to keep the code simple and non-complex.


Ronald Coase, a famous economist once said, "If you torture the data long enough, it will confess to anything." With a deeper insight into what's going on with your software code, you can have a better conversation with your engineers. It is most effective to track these weekly. I usually bring them up at the end of our sprint cycles. There are weeks when our productivity is at a high and calling out these moments makes the team feel appreciated. Similarly, there are days when we fall behind and a team discussion helps us understand why.

Refrain from hanging deadlines in front of your engineers. Instead learn more about the roadblocks, ask good questions, and find solutions that can work consistently.

The five whys analysis is always a good tool to get to the core of the issues and prevents teams from coming up with superficial solutions. As Six Sigma states it - "By repeatedly asking the questions 'why' (five is a good rule of thumb), you can peel away the layers of symptoms which can lead to the root cause of a problem".

I'm curious to know what other metrics product managers track to understand their team performance. If you have any that are not included in this list, please leave comments here.

Until then - get these in place and enjoy not being THAT product manager!

full story