The Three Skills of a Great PM

TL;DR: Good and Great PMs differentiate themselves by focusing on improving the rate of various aspects in product development. Good PMs improve rate of execution and rate experimentation. Great PMs also improve the rate of idea validation. 

Recently a few founders I work with wanted to discuss what a product person does and how to tell a good one from a great one. After thinking through it I decided to share what I’d come to believe makes a good vs. great PM after time at Facebook, Microsoft and my own (mostly failed) startups. More importantly, I’m very curious to hear how others identify great PMs.

5142145393_48a8baa876_bSkill One: Rate of Execution

Fundamentally, a PM helps a company more quickly define, build and launch new products. Entry level PMs start off by writing specs, triaging bugs / issues and helping project manage, with engineering management, specific releases. More experienced PMs take responsibility for successfully launching new products which requires coordination and decision making across a wide constituency such as executives, marketing, legal, finance, etc all while never letting the product’s soul be compromised. While every company has its own specific approach to product development a skilled PMs ensures that every successive release happens faster with better results.

This is really hard and the number of PMs that can do this is surprising small given the number of people with PM titles. Why is it hard? Because often the key to becoming faster involves changing the way things are done in areas a PM has no authority over. Specifically, sometimes it can mean changing composition of the engineering team and others times it means persuading various stakeholders that the plan is the right one so work doesn’t have to be redone. It can even mean getting the CEO to change his mind or even just flat out telling the CEO “no”.

Not surprisingly it is often not until a PM has had 5-10 years of experience building & shipping products that they can reliably accomplish this level of performance. In a startup this is the minimum level of competence that should be hired as an individual contributor PM. Needless to say when evaluating PMs measuring their impact on rate of execution of their team is critical.

4393914288_68c10a576c_oSkill Two: Rate of Experimentation

With software product delivery happening continuously via the web it has become critical for the product development process to include rapid experimentation to validate assumptions & features before committing resources necessary to launch them. PMs start building this skill by learning the mechanics of experimentation that includes multi-variant testing and how to construct good experiments. Experienced PMs shift their focus to increasing the throughput of experiments their company can run as well as the number of experiments that can be run in parallel.

Throughput is surprisingly hard to affect and is even harder to be disciplined about since it is much more fun (and easy) to argue about which experiment to run next. Naturally, everyone thinks their own idea for an experiment is high priority when in reality it is more important to just get the company to run more experiments faster. To really affect the rate of experimentation good PMs have to focus on technical infrastructure that enables experiments, cultural norms around experiments and prioritization frameworks for evaluating experiments. Almost none of these have direct ties to end user facing features so they can be hard to get a team interested in prioritizing them.

Typically this skill set only gets developed and internalized by a PM who is member of either a user acquisition or performance marketing team. Especially if they have been a PM for a dating, gaming, affiliate marketing, ecommerce or SAAS company. But once a PM has developed skill 2 they can join any organization and almost immediately increase the rate of experimentation.

libtechSkill Three: Rate of Idea Validation

Every company needs multiple PMs that have both skills above. In fact large economic value can be created by only having PMs with above levels of skills. But iconic product companies need to have at least one PM who can drive the rate of idea validation faster over time. The rate of idea validation is a measure how quickly new ideas are evaluated to be killed or further resourced. Some companies call this innovation while others call it R&D. Regardless of what they call it, all companies are a function of how well they allocate their resources (I know, so very clinical) as well as how big the market is for their products. Thus finding large new market opportunities as efficiently as possible is straightest path to create outsized value.

Great PMs consistently find great new markets to build new products for as well as totally rethink existing markets to challenge status quo. It is almost magical to see this process in action because it turns out that there is no conscious way to learn how to do it. Great PMs are great because they have “taste” in picking problems & coming up with solutions. Really fucking good taste. This almost intuitive sense of taste enables them to allocate resources more efficiently than everyone else. They make great decisions with partial information while under great pressure while avoiding dead ends.

I believe great taste can be developed but not in a linear manner that is predictable or time bound. The best, and perhaps only way, to develop great taste is to be interdisciplinary and to gather a large variety of life experiences to draw upon. This is why Steve Jobs’ focus on the intersection of technology and liberal arts has always made a lot of sense to me.

So, in a nutshell, Great PMs improve the following three things:

  1. Rate of execution
  2. Rate of experimentation
  3. Rate of idea validation

When you find a great PM that has developed all of the skills above give them whatever resources they need to launch a product. It will likely succeed beyond anyone’s wildest expectations.

Thanks to Andrew Chen and Semil Shah for reviewing an early version of this.

Update: Cleaned up some formating and improved the TL;DR based on feedback.

3 Types of Notifications

https://www.flickr.com/photos/postmemes/14876404921/in/photolist-oEzsU2-6ySueJ-6yNpCp-o5SmJA-9aqABV-awo33u-4UHdYz-9AM9vi-yP8Ni-4UHeiF-9QWcmV-9atiAk-9fpeaj-9fyWAN-8iXQ5k-9fvP22-9fyWzJ-a69WFk-68thin-9n4wdv-aVrJBr-7XiLD3-7aXSx1-7xk83j-bVShhj-6s3Lk3-efr8xX-dFCvDi-fLur7b-a5CpcV-dTaciK-gYy1G4-btt8yn-6YwU6j-fieNMs-6TACCc-77WX54-6UCabD-7tKmy9-fqgG5K-b2Kmn6-fGJdaU-74ySFz-6U4jSe-efr9pK-efr8Vn-7zZWk2-efqUmD-efqTvD-6FNCt1/

Photo credit: Post Memes via Flickr

Talking with founders is one of my favorite parts of being an early stage investor. Recently a founder casually mentioned there were three types of push notifications. I pounced on the comment as I had thought the same thing independently and was I excited to hear someone else’s definitions. Turns out we were in completely different places: the founder was thinking about notification technology while I was thinking notification use cases.

The three technical types she outlined were push, local, and in-app. In reviewing the iOS dev docs, one could also frame three as badge, message, and sound. The reality is that on either dominant mobile platform, notifications have a lot of unexplored potential in terms of tech and UX.  And as the sophistication of notifications design grows in support of user engagement (not new user growth as is commonly discussed) the need for specific language to analyze the different types of notifications grows since not all of them are made equal!

With that in mind I’m going to enumerate the key notification use cases given their importance to mobile application design, engagement, and growth. Personally, I’ve come to believe there are three common types of use cases for notifications*: (1) user-generated, (2) context generated, and  (3) system generated. Diving into each a bit more:

  1. User-generated notifications: These notifications contain content created by a human using the app to other humans. Generally, these are the most engaging but especially so when the content they contain is private and directed to specific people. Mobile messaging is the highest volume example of this type of notification but other examples include comments / likes / favorites on posted content or @ mentions. My current favorite example of this notification is getting a new photo of my son from my wife.
  2. Context-generated notifications. These notifications are generated by an application based on the permission of its users. This is the fast growing category of notifications because the amount of machine readable data mobile devices create: location, contacts, calendars, and much much more. The norms around context-generated notifications are still be worked out between developers and users. Location-based notifications currently dominate this category but other examples include information about your next meeting (time relevance) or updates about your favorite sports teams (interest relevance). My current favorite example of this notification is when I get notified there is a designer nearby via Highlight (disclosure: I’m an investor there).
  3. System-generated notifications. These notifications are generated by an app based on the needs of the app. This type of notification can usually be called re-engagement at best or spam at worst.  Sometimes these can create value for the end user like letting you know a friend has started using the app or that there is a sale on in app purchases.  Actually I can’t say that with a straight face as I don’t think they’ve ever created any value for me. Wait, I got one: security alert that someone has requested to reset my password. Well, until those started coming every few days anyhow. :(

I’d like to pause here to solicit feedback on this classification and specifically to find examples that break it. After digesting feedback I’ll follow up with some more ideas on notification design.

*While I’m framing this in the context of mobile push notifications I think these categories work fine to classify notifications delivered via other channels including SMS, email and soon to arrive widely web browsers notifications.

Kiwi

Getting to use Facebook for work, while I was part of the company for five years, was hard to top but I found a way – playing mobile games is now part of my DFJ job description.

I’m thrilled to share my third investment for DFJ and my first out of our eleventh fund, into Kiwi’s Series B led by Northgate Capital.  Kiwi is an AAA mobile gaming company co-founded by Omar Siddiqui and Shvetank Jain and initially funded by Omar Hamoui and Sequoia Capital. This team is one of the rare Android-first developers that have gotten five of their titles into the top twenty-five grossing of the Google Play Store. You can read more about Kiwi’s Series B funding here, here and here.

Now, as has become a bit of a personal tradition, I’d like to share the story behind my investment into Kiwi.

Since joining DFJ just over a year ago I’ve been talking about the large opportunities available on Android.  As a result I’ve been lucky enough to see some of the most exciting Android-first focused startups that are pushing the mobile experience further than most folks in the Silicon Valley think is possible. Nevertheless, I had trouble finding right team, concept and investment opportunity to get behind and wasn’t sure if I’d find the right Android-first company.

It turns out, sometimes as an investor, you have to be found instead of doing the finding.  Lucky for me Omar Siddiqui found some of my blog posts about Android so he asked for an introduction via our mutual friend Matt Wyndowe.  Even though Matt spells his last name so wildly, he is anything but wild when it comes to making introductions. He is one of the most thoughtful super nodes I know!

In our first conversation, Omar captured my imagination with his vision for building an amazing Android-first company focused on AAA games and distribution. As I spent more time with him and the team I started to learn new things about mobile gaming and Android that floored me.  Especially around how Android distribution and monetization were working in the wild.  The scale of Kiwi along with the number of Android product experiments they’d made was astounding.  Just as importantly, Omar and I had a very natural communication rhythm that led to several deeply engaging conversations around the future of Android and mobile in general that profoundly affected my thinking. Needless to say, I quickly knew I wanted to invest.

The only rub with investing was that Kiwi already had an incredibly strong investment syndicate with Northgate committed to leading the Series B with participation of Sequoia and Omar Hamoui (the founder of Admob) who led the Series A and Seed rounds, respectively.  Not such a little rub, in truth. Luckily, Omar and his existing investors were equally excited about having DFJ participate so we came up with a way to make it work for all parties. This was also a particularly good learning experience for me since Kiwi was my first series B investment in addition to being a multiparty negotiation.

Right now, the Kiwi team is hard at work on its next generation of AAA games. And in seeing the internal demos I am confident that mobile gamers are going to be floored with what they deliver.  If you love producing world class AAA games please reach out to me as the Kiwi team is planning to hire another executive producer now that the series B money is “in the bank”.

Thoughts on the Amazon Fire Phone

Benedict Evans has a thoughtful post up on the Amazon Fire phone. I agree it is going to be a fascinating first experiment to see how an Android fork without Google Mobile Services (GMS) fares outside of China and a while back wrote about other ways to compete with GMS. Ben also raises a great question around how any company’s motivation to build an Android fork that does not run GMS can be at odds with consumers needs and desires for a phone. It is worth quoting his point directly before going further:

“All of this takes us to the elemental question – why, exactly, are you forking Android? What important problem do you solve that’s worth reinventing the wheel, while taking on the risk of building on someone else’s platform, open-source or not? Why are you asking people to buy a phone with second-rate maps and a second-rate app store? Are you offering them something you couldn’t otherwise do in return, or just addressing your own strategic concerns? Are you solving a user problem or your own problem?”

This certainly got my attention given my experience leading Android products at Facebook. Let’s list out most likely motivations driving Amazon, and any other company, when they decide to fork Android:

  • Lock-in current ecosystem customers;
  • Capture a valuable data asset for Maps, App Stores & more;
  • Long term market pressures on Google and Apple to innovate.

What follows below is how these three motivations address both Amazon’s and consumers’ problems. The rub, however, is the number of consumers who face problems starts off small but grows dramatically over time. In writing this, I’ll make a very conservative assumption that Fire Phone will attach to 5% Amazon Prime users, who are the most loyal to the company, in the first 3 years in the market.  Amazon has never released the total Prime subscriber count but recent articles suggest it is at least 20 million (holy crap, Amazon has a $2 billion dollar subscription service).  Lets look at why Amazon would invest so many of its resources to build a phone for just 1 million customers and how many customers it could ultimately impact.

In the near term, current Amazon customers are delighted by amazing end-to-end scenarios across Amazon’s web, tablet, TV, Kindle and now, finally, smartphones. Their digital content works seamlessly across all of their experiences.  This is the Apple “vertical playbook” that has created hundreds of billions of value. However, Amazon is attaching a very different business model to this vertical play by approaching the hardware as near zero margin business*.

This means, in the near term, Amazon will view success of Fire Phone not in total units sold, but rather in increase in LTV of current ecosystem customers. This seems like solid business motivation given reports that Amazon Prime members spend almost twice as much as non-prime members.  Doubly so when seeing comps like eBay share they GMV sales of $35 billion on Mobile devices in 2013.

In the mid term, all smartphone customers** benefit by increased competition across Maps and App Stores. I have no doubt that Apple will make Google Maps & Play Store up their game over time because of the data they collect & use to improve their versions of those services via iOS devices in the wild. My statement might be controversial and hard to believe but I’d cite Google’s aggressive protection of its location data collection on Android as support for the argument (to be continued in the comments :).  And now Amazon will be able to do that too without being constrained by the limits placed on third party applications***.

However, what is really fascinating to watch is how Amazon’s business model will allow them to target specific geographies where they can profit via a long-term e-commerce relationship instead of upfront hardware purchase or long-term advertising revenue – this is a major differentiated advantage. Amazon can outspend competitors in specific geographies where their model works to build the best in class Maps & App Store solutions. While hard to quantify numbers it is clear that customers on other smartphone platforms will benefit from this competition and choice.

Finally in the long-term, the dominant platform owner’s interests diverge from customers and application developers. Unfortunately, there is oftentimes little that customers can do directly when things get to this state, so government regulation or market-driven disruption have to solve the problem. Large developers, like Amazon, fund both approaches via lobbying and projects like Fire Phone.

However, at some point, the interests of head and long tail of app developers result in open standards and source to commoditize the dominant platforms (web browsers vs. windows, linux vs. *nix). Then, the disruptive platforms, and frankly the platforms that lost, implement the new open standards first to put pressure on the winning platforms to also adopt them. If you think this is too much of a stretch, I’d suggest looking at many of iOS8’s features and imagine if they’d been built if Android didn’t exist. Cue, “the circle of life”.

Taking it all together, I’d argue Amazon has customer problems firmly in mind with the launch of Fire Phone. It is just over a much longer timeframe than most companies can conceive. Not surprising, given everything I read about the long term thinking of Jeff Bezos.  This Bezo’s quote is striking given all of the above:

“If everything you do needs to work on a three-year time horizon, then you’re competing against a lot of people, But if you’re willing to invest on a seven-year time horizon, you’re now competing against a fraction of those people, because very few companies are willing to do that.”

* I suspect the Fire Phone is not deviating from the low-margin hardware business model, but given the US phone distribution dynamics, they chose to offer more features at the same price as their competition (e.g. 32GB of memory for $199 price point instead of 16GB).

** I’m being too subtle here probably, but when I say smartphone customers I actually include not just end users but also mobile operators as they are key customers as well. All smartphones are sold to both at the start of a new mobile platform given the complexity of the mobile value chain. Longer post on this someday.

*** I’m not sure how to value having 1M devices giving me all of their sensor and application usage data 24 hours a day, but given Waze was worth a billion dollars for a small fraction of the data in question, I believe it to be worth quite a bit.

 

The Three Phases Of Mobile Advertising

We all know mobile advertising is big today and will only get bigger (read Mary Meeker’s internet trends if you need to be convinced). As consumer time continues to shift to mobile devices, more dollars will shift to mobile advertising. While mobile ad spend lags behind today, it will eventually catch up and surpass desktop and television if you believe, like I do, that Facebook’s 59% of Q1’14 earnings from mobile are the bellwether. Best of all, this rising tide will only get higher as mobile ads evolve in their format and targeting abilities.

Because of all of the above I view the mobile advertising landscape as having three distinct phases:

Phase I (before my time to 2012)

 

  • Traditional IAB units shrunk to fit inside a small screen, resulting in a bad user experience
  • Targeting mobile ads was difficult because every app was its own silo with no cookies, re-targeting
  • Incentivized advertising dominated because of volume of installs they could drive and the nascent understanding in app purchasing

Phase II (2012 to 2015)

  • Native ads took off with a better user experience, but largely the same content and goals as previous ad formats
  • Targeting improved to match levels of web-based ad targeting (including cookie equivalents, re-targeting, and real-time bidding).
  • Facebook — and soon, others — created world class self service ad tools to attract large amounts of direct marketing / user acquisition marketing spend.

Phase III (2016 to beyond)

  • A class of mobile-only advertising units such as sponsored push notifications, try before you buy app previews, and other to-be-determined formats emerge and take off.
  • Mobile-only targeting data where the data mined originates from our device sources, such as current location, current apps installed, calendar, and address book data deliver massive value to marketers.
  • All the big players (Facebook, Alibaba, Yandex, etc) make piles of money through direct monetization and via mature ad networks that share 90% of revenue with publishers.

So, where are we today, mid-way through 2014? I believe we are firmly into Phase II and starting to see some evidence that Phase III is ~24 months out. As an investor, I am trying to find key opportunities around products that will support large businesses once Phase III is in full swing. Like the web, large communities of users and communication applications are the early winners, but there will be more variety as mobile devices grow and our time spent on them increase.  Can you think of others? If you have ideas in this area, of course, please drop me a line.

Note: Tomorrow I’m on a panel talking about Mobile Native Ads at Grow.co so decided to prep for the discussion by blogging my current thoughts on Mobile advertising. Feedback before I go on stage tomorrow at 2:15pm PST is much appreciated.  :)

Silicon Valley Is Underinvested In Android

I know what you’re thinking…”Oh, it’s Bubba again, blogging about Android.” There he goes again….But, it’s true. The Silicon Valley ecosystem, as a whole, is wholly underinvested in Android. In fact there even seems to be a rising sentiment that building on top of Android is a poor strategy.  There are a probably many reasons for this but here are a few reasons that seem particularly large.

First, the number of folks in the Bay Area startup and technology scene with Android devices as primary phones seems to be inversely proportional to the worldwide distribution of Android. How will we all be able to build for a platform we don’t natively use? Yes, many of us will adapt quickly if forced, but technology is a speed game — will other regions already steeped in the ins and outs of Android development and distribution beat us at our own game? It’s possible. Just look at the messaging giants spreading across Asia and the Pacific.

Second, the intellectual horsepower and curiosity in the Bay Area toward mobile is incredibly rich and varied. Yet, the majority of this attention is focused on iOS. I understand why, all the hot consumer apps start on Apple’s platform. But, one side effect of this is that we aren’t able to form equally deep opinions on Android as a platform, or its forks, or apps on top of each. We are more than prepared for the next iterations inside iOS and advancements to the iPhone hardware, yet we don’t seem to have a similar level of curiosity and exploration for the Android platform. Every now and then, a mobile startup will survive long enough to go “cross-platform,” and those teams get to experience the wondrous joy of Android (it ain’t easy, folks!), but those are often isolated experiences.

I believe most mobile observers and investors would have a difficult time trying to name 10 Android-first or Android-only startups in Silicon Valley. How is it possible that so few of these exist, and how is the next mobile unicorn going to be created if we don’t have large numbers of experiments running on Android? It’s cliche to say “we are living in a very special time,” but…we are! Look at all the new platforms emerging around us — virtual reality software (like Oculus VR and Morpheus), the Bitcoin protocol, drones, 3-D printing, and more. Android is one of these platforms, and perhaps one of the biggest in terms of size, reach, and potential market value.

Android is literally a freight train, an unstoppable force and a computing platform that will only increase in power. To that end, I want to see and fund the true Android-first startups out there, the teams who will run with the blank canvas, who won’t be constrained by conventions of a sister platform, who will think originally and critically about the next innovations in mobile computing, technology, and consumer experiences. If this describes you, please get in touch.

Bear Hugging Android

From a consumer perspective, a mobile operating system has to offer certain experiences to be considered a viable option. Many of the challengers to iOS & Android have painfully learned this. The most important mobile services are maps and app stores, so consumers can use these devices to navigate, search around them based on location, and to leverage all of the sensors and capacities of the phone. Beyond these, there are also a variety of secondary and tertiary mobile services including communication, games, media, cloud backup, and more. All of these have one thing in common, however: they are massive web services at their core.

Controlling the critical mobile operating system services creates outsized value for companies — independent of who owns the underlying platform. Google arguably has the biggest advantage through its Google Mobile Services (GMS) suite which are anchored by Maps & the Play Store. On Android it uses this advantage to require inclusion of secondary and tertiary Google services as well as to dictate product design requirements in its interest in exchange for giving partners a license to the core GMS suite. This is a tremendous leverage except for places like China, where there has been a proliferation of alternatives for key mobile operating system services.

While there have been a handful of investments in new mobile operating systems (startups such as CyanogenMod and Xiaomi which are based on Android while Windows Phone, Firefox OS and Tizen have all been built from scratch by large companies), there hasn’t been as much focus on competing with GMS. It is hard to imagine a startup viably competing with Google’s offering in maps or app store. Larger companies such as Amazon, Yandex & Microsoft (with Nokia), are starting to do this, but it’s very expensive to directly compete in these core mobile operating system services.

For a while I’ve been curious if there were other ways to compete with GMS but never found anything convincing. That is, until Facebook bought Whatsapp. I’ll get the disclaimer out of the way: I own Facebook stock, I’m a former Facebook employee, and I’m a current happy user of Facebook.

Back to Whatsapp and Facebook…it turns out that a great way to compete with GMS is not to compete head on with Google, but to control other critical points of the mobile operating system experience. With the shift to over the top communication apps a new critical mobile operating system service is emerging. Specifically, Facebook is now in a position to offer a preload of “Facebook Mobile Services” (FMS) that includes Whatsapp, Instagram, Paper, Messenger, and other Facebook apps and services. Combined, these apps represent an emerging mobile communication suite that is so core to the mobile experience, it puts Facebook on par with Google in terms of leverage on the Android ecosystem.

Further, it appears that Google is not in a position to limit Facebook’s influence on the Android ecosystem via its key control point of GMS. Facebook might be the only non-Chinese company in this position right now. Over time Facebook could require  inclusion of their own social app store, zero rating for their applications or perhaps even require a revenue share per subscriber in exchange for a license to “FMS”. So while Google bear hugs Android via GMS, Facebook can bear hug Android+GMS through its own mobile services suite.

This is a big deal.