TL;DR: Good and Great PMs differentiate themselves by focusing on improving the rate of various aspects in product development. Good PMs improve rate of execution and rate experimentation. Great PMs also improve the rate of idea validation. 

Recently a few founders I work with wanted to discuss what a product person does and how to tell a good one from a great one. After thinking through it I decided to share what I’d come to believe makes a good vs. great PM after time at Facebook, Microsoft and my own (mostly failed) startups. More importantly, I’m very curious to hear how others identify great PMs.

Skill One: Rate of Execution

Fundamentally, a PM helps a company more quickly define, build and launch new products. Entry level PMs start off by writing specs, triaging bugs / issues and helping project manage, with engineering management, specific releases. More experienced PMs take responsibility for successfully launching new products which requires coordination and decision making across a wide constituency such as executives, marketing, legal, finance, etc all while never letting the product’s soul be compromised. While every company has its own specific approach to product development a skilled PMs ensures that every successive release happens faster with better results.

This is really hard and the number of PMs that can do this is surprising small given the number of people with PM titles. Why is it hard? Because often the key to becoming faster involves changing the way things are done in areas a PM has no authority over. Specifically, sometimes it can mean changing composition of the engineering team and others times it means persuading various stakeholders that the plan is the right one so work doesn’t have to be redone. It can even mean getting the CEO to change his mind or even just flat out telling the CEO “no”.

Not surprisingly it is often not until a PM has had 5-10 years of experience building & shipping products that they can reliably accomplish this level of performance. In a startup this is the minimum level of competence that should be hired as an individual contributor PM. Needless to say when evaluating PMs measuring their impact on rate of execution of their team is critical.

Skill Two: Rate of Experimentation

With software product delivery happening continuously via the web it has become critical for the product development process to include rapid experimentation to validate assumptions & features before committing resources necessary to launch them. PMs start building this skill by learning the mechanics of experimentation that includes multi-variant testing and how to construct good experiments. Experienced PMs shift their focus to increasing the throughput of experiments their company can run as well as the number of experiments that can be run in parallel.

Throughput is surprisingly hard to affect and is even harder to be disciplined about since it is much more fun (and easy) to argue about which experiment to run next. Naturally, everyone thinks their own idea for an experiment is high priority when in reality it is more important to just get the company to run more experiments faster. To really affect the rate of experimentation good PMs have to focus on technical infrastructure that enables experiments, cultural norms around experiments and prioritization frameworks for evaluating experiments. Almost none of these have direct ties to end user facing features so they can be hard to get a team interested in prioritizing them.

Typically this skill set only gets developed and internalized by a PM who is member of either a user acquisition or performance marketing team. Especially if they have been a PM for a dating, gaming, affiliate marketing, ecommerce or SAAS company. But once a PM has developed skill 2 they can join any organization and almost immediately increase the rate of experimentation.

Skill Three: Rate of Idea Validation

Every company needs multiple PMs that have both skills above. In fact large economic value can be created by only having PMs with above levels of skills. But iconic product companies need to have at least one PM who can drive the rate of idea validation faster over time. The rate of idea validation is a measure how quickly new ideas are evaluated to be killed or further resourced. Some companies call this innovation while others call it R&D. Regardless of what they call it, all companies are a function of how well they allocate their resources (I know, so very clinical) as well as how big the market is for their products. Thus finding large new market opportunities as efficiently as possible is straightest path to create outsized value.

Great PMs consistently find great new markets to build new products for as well as totally rethink existing markets to challenge status quo. It is almost magical to see this process in action because it turns out that there is no conscious way to learn how to do it. Great PMs are great because they have “taste” in picking problems & coming up with solutions. Really fucking good taste. This almost intuitive sense of taste enables them to allocate resources more efficiently than everyone else. They make great decisions with partial information while under great pressure while avoiding dead ends.

I believe great taste can be developed but not in a linear manner that is predictable or time bound. The best, and perhaps only way, to develop great taste is to be interdisciplinary and to gather a large variety of life experiences to draw upon. This is why Steve Jobs’ focus on the intersection of technology and liberal arts has always made a lot of sense to me.

So, in a nutshell, Great PMs improve the following three things:

1. Rate of execution
2. Rate of experimentation
3. Rate of idea validation

When you find a great PM that has developed all of the skills above give them whatever resources they need to launch a product. It will likely succeed beyond anyone’s wildest expectations.

Thanks to Andrew Chen and Semil Shah for reviewing an early version of this.

Update: Cleaned up some formating and improved the TL;DR based on feedback.

Photo credit: Post Memes via Flickr

Talking with founders is one of my favorite parts of being an early stage investor. Recently a founder casually mentioned there were three types of push notifications. I pounced on the comment as I had thought the same thing independently and was I excited to hear someone else’s definitions. Turns out we were in completely different places: the founder was thinking about notification technology while I was thinking notification use cases.

The three technical types she outlined were push, local, and in-app. In reviewing the iOS dev docs, one could also frame three as badge, message, and sound. The reality is that on either dominant mobile platform, notifications have a lot of unexplored potential in terms of tech and UX.  And as the sophistication of notifications design grows in support of user engagement (not new user growth as is commonly discussed) the need for specific language to analyze the different types of notifications grows since not all of them are made equal!

With that in mind I’m going to enumerate the key notification use cases given their importance to mobile application design, engagement, and growth. Personally, I’ve come to believe there are three common types of use cases for notifications*: (1) user-generated, (2) context generated, and  (3) system generated. Diving into each a bit more:

1. User-generated notifications: These notifications contain content created by a human using the app to other humans. Generally, these are the most engaging but especially so when the content they contain is private and directed to specific people. Mobile messaging is the highest volume example of this type of notification but other examples include comments / likes / favorites on posted content or @ mentions. My current favorite example of this notification is getting a new photo of my son from my wife.
2. Context-generated notifications. These notifications are generated by an application based on the permission of its users. This is the fast growing category of notifications because the amount of machine readable data mobile devices create: location, contacts, calendars, and much much more. The norms around context-generated notifications are still be worked out between developers and users. Location-based notifications currently dominate this category but other examples include information about your next meeting (time relevance) or updates about your favorite sports teams (interest relevance). My current favorite example of this notification is when I get notified there is a designer nearby via Highlight (disclosure: I’m an investor there).
3. System-generated notifications. These notifications are generated by an app based on the needs of the app. This type of notification can usually be called re-engagement at best or spam at worst.  Sometimes these can create value for the end user like letting you know a friend has started using the app or that there is a sale on in app purchases.  Actually I can’t say that with a straight face as I don’t think they’ve ever created any value for me. Wait, I got one: security alert that someone has requested to reset my password. Well, until those started coming every few days anyhow. 😦

I’d like to pause here to solicit feedback on this classification and specifically to find examples that break it. After digesting feedback I’ll follow up with some more ideas on notification design.

*While I’m framing this in the context of mobile push notifications I think these categories work fine to classify notifications delivered via other channels including SMS, email and soon to arrive widely web browsers notifications.