TL;DR: Good and Great PMs differentiate themselves by focusing on improving the rate of various aspects in product development. Good PMs improve rate of execution and rate experimentation. Great PMs also improve the rate of idea validation. 

Recently a few founders I work with wanted to discuss what a product person does and how to tell a good one from a great one. After thinking through it I decided to share what I’d come to believe makes a good vs. great PM after time at Facebook, Microsoft and my own (mostly failed) startups. More importantly, I’m very curious to hear how others identify great PMs.

Skill One: Rate of Execution

Fundamentally, a PM helps a company more quickly define, build and launch new products. Entry level PMs start off by writing specs, triaging bugs / issues and helping project manage, with engineering management, specific releases. More experienced PMs take responsibility for successfully launching new products which requires coordination and decision making across a wide constituency such as executives, marketing, legal, finance, etc all while never letting the product’s soul be compromised. While every company has its own specific approach to product development a skilled PMs ensures that every successive release happens faster with better results.

This is really hard and the number of PMs that can do this is surprising small given the number of people with PM titles. Why is it hard? Because often the key to becoming faster involves changing the way things are done in areas a PM has no authority over. Specifically, sometimes it can mean changing composition of the engineering team and others times it means persuading various stakeholders that the plan is the right one so work doesn’t have to be redone. It can even mean getting the CEO to change his mind or even just flat out telling the CEO “no”.

Not surprisingly it is often not until a PM has had 5-10 years of experience building & shipping products that they can reliably accomplish this level of performance. In a startup this is the minimum level of competence that should be hired as an individual contributor PM. Needless to say when evaluating PMs measuring their impact on rate of execution of their team is critical.

Skill Two: Rate of Experimentation

With software product delivery happening continuously via the web it has become critical for the product development process to include rapid experimentation to validate assumptions & features before committing resources necessary to launch them. PMs start building this skill by learning the mechanics of experimentation that includes multi-variant testing and how to construct good experiments. Experienced PMs shift their focus to increasing the throughput of experiments their company can run as well as the number of experiments that can be run in parallel.

Throughput is surprisingly hard to affect and is even harder to be disciplined about since it is much more fun (and easy) to argue about which experiment to run next. Naturally, everyone thinks their own idea for an experiment is high priority when in reality it is more important to just get the company to run more experiments faster. To really affect the rate of experimentation good PMs have to focus on technical infrastructure that enables experiments, cultural norms around experiments and prioritization frameworks for evaluating experiments. Almost none of these have direct ties to end user facing features so they can be hard to get a team interested in prioritizing them.

Typically this skill set only gets developed and internalized by a PM who is member of either a user acquisition or performance marketing team. Especially if they have been a PM for a dating, gaming, affiliate marketing, ecommerce or SAAS company. But once a PM has developed skill 2 they can join any organization and almost immediately increase the rate of experimentation.

Skill Three: Rate of Idea Validation

Every company needs multiple PMs that have both skills above. In fact large economic value can be created by only having PMs with above levels of skills. But iconic product companies need to have at least one PM who can drive the rate of idea validation faster over time. The rate of idea validation is a measure how quickly new ideas are evaluated to be killed or further resourced. Some companies call this innovation while others call it R&D. Regardless of what they call it, all companies are a function of how well they allocate their resources (I know, so very clinical) as well as how big the market is for their products. Thus finding large new market opportunities as efficiently as possible is straightest path to create outsized value.

Great PMs consistently find great new markets to build new products for as well as totally rethink existing markets to challenge status quo. It is almost magical to see this process in action because it turns out that there is no conscious way to learn how to do it. Great PMs are great because they have “taste” in picking problems & coming up with solutions. Really fucking good taste. This almost intuitive sense of taste enables them to allocate resources more efficiently than everyone else. They make great decisions with partial information while under great pressure while avoiding dead ends.

I believe great taste can be developed but not in a linear manner that is predictable or time bound. The best, and perhaps only way, to develop great taste is to be interdisciplinary and to gather a large variety of life experiences to draw upon. This is why Steve Jobs’ focus on the intersection of technology and liberal arts has always made a lot of sense to me.

So, in a nutshell, Great PMs improve the following three things:

1. Rate of execution
2. Rate of experimentation
3. Rate of idea validation

When you find a great PM that has developed all of the skills above give them whatever resources they need to launch a product. It will likely succeed beyond anyone’s wildest expectations.

Thanks to Andrew Chen and Semil Shah for reviewing an early version of this.

Update: Cleaned up some formating and improved the TL;DR based on feedback.