Customer interviews - My learnings

As a product manager, there’s nothing more important than having a single source of truth for discovering problems or opportunities. That source of truth is the customer. This isn’t rocket surgery to be honest but with this post I do intend to emphasise on why speaking with customers is an essential part of the product development lifecycle. Over the past year, I’ve averaged at least one customer interview per week and I still feel I should’ve done more. These interviews were in the form of job-shadowing, discovery, UX research and prototype testing. While each of these types of interviews are different, the end goal is always to ensure that you are solving a real problem for the customer and that when you go to market with a product, you ensure some amount of predictability for its success.

We’re all familiar with the product lifecycle development stages, right? Well, here’s a simplified version that I have I created where I mention the various touch-points where customer interviews play a crucial role.

Untitled+drawing.jpg

Like any interview you undertake, preparation is important. Here are some of my learnings that I think anyone new or experienced at customer interviews might find useful.

Learnings

Segmentation


Before talking to the customer/user, it’s important to get as much context about who she is. You want your segmentation research to give you an idea on who she is, how long has she been using your product, how active is she and how much is she paying for it. This will help you make assumptions on what drives her to use (or not use) your product. 


Have a script ready


It’s always handy to have a script prepared on what questions you plan to ask the user. This not only helps you bring a structure to the interview but as you interview many users, a script ensures that the same research methodology is applied in the entire set of interviewees. 


Avoid leading questions

Leading questions would probably form an entire chapter in a book about doing good customer interviews. However, if I were to extract the gist of it, it could be simply put as - “Avoid questions that can be answered with a yes or a no”. Leading questions do not add value to the research process as you fail to understand factors influencing users’ decisions or their emotional state. The answers you seek in interviews should not be biased by the questions but rather you want customers to open up about their experiences. For example, if you ask a user - “Are you having problems with our product X?” The answer could be a curt ‘no’ or a ‘yes’. But if you frame the question as “Can you walk me through how you use our product X?” The latter helps pinpoint the journey of the user and the “problems” that you seek will be revealed in due course of time with follow-up questions. Refer to the next point below on the five whys. 


The five whys


This is a simple yet effective technique to get to the bottom of a problem. It was first conceived in the Toyota production facility for identifying bugs. In the interview setting, it is brutal and unrelenting when used in the right manner. The short version of how this is done is when the interviewer persistently quizzes the user with a “why?” and follows that with another “why?”. At times, insights are derived before the fifth why is asked but it’s the process that matters more. This could be an example of asking the five whys:

Me: You’ve been using our product X for 2 years now. Can you walk me through how you interact with feature Z.

User: I actually don’t use feature Z.

Me: Why don’t you use feature Z?

User: I don’t know, it looks complicated.

Me: Why do you think it is complicated? 

User: I visit the page and I see a lot of text that I don’t understand. 

Me: Alright, why do you feel that way? 

User: English is not my first language and I also prefer to see a tutorial video to understand how to use a new feature

.
…and the process continues but I hope you catch my drift.


Don’t listen to the user’s solutions


It’s easy to fall prey to what the user/customer says about your product. The caveat here is that the customer is not always right. It is imperative as an interviewer to understand what problem the user is facing rather than accepting the solution the user instructs you to build. This is a fairly common occurrence where users lay out feature-flows and solutions down to the tiniest details. But as a product manager, you are solving problems for an entire customer/user base, not just one person who gives her version of the solution. 


Record, record, record


Record your interviews in one way or the other. If it’s an interview that is happening over video call, then you can ask the user’s permission to record the session. Or else if you have a co-interviewer who can observe and take notes during the interview, that is another good option. If all else fails, you need to brush up on your note-taking abilities. You wouldn’t want to miss out on key insights during the interview process that you can revisit at a later time. 


When do you stop interviewing


This is quite a common question that comes up. Just how many customer interviews is enough? The unfortunate answer is that it’s never enough. But my general thumb rule is to interview as many people within a particular time period and you should stop interviewing when you see a pattern developing in your responses. In short, stop interviewing when you stop seeing variability for that set of customer interviews.

On a closing note, these learnings and observations have happened over a course of time and I hope you found them useful. While researching this topic on the interwebz to see other people’s opinions, I also came across a very useful chart on conducting interviews at the Product Tribe which is quite share-worthy.

Until my next post, happy interviewing!

Podcasts for Product Managers

2016 saw me really dive into the world of podcasts. I felt this urge of making my commute times more productive, so I turned to podcasts. With some advice from fellow podcast listening friends, I downloaded the Podcast Addict app (I now use Pocket Casts and it's 100x better) and I was off to the races!

Coincidentally, the period of 2016-17 was the time when I was sinking my teeth further into the field of product management. Here's a joke - "How does one know if a person is a Product Manager?". "S/he'll tell you about it, blog about it, podcast about it and create a PPT about it". Okay I made that up. But the point I'm trying to make here is that there's a lot of material on podcasts directed at new and experienced product managers alike.

Keeping my cynicism aside, I have learnt a lot from these podcasts. They give you a sense that the playing fields are levelled, you can up-skill hard and soft skills, and eventually, you feel at par with accomplished product managers in marquee companies. I'm going to use this opportunity to mention a bunch of podcasts that have helped me grow as a product manager over the past two years. Please note that the list is not in any order of preference. 

1. This is Product Management
This is arguably the most popular podcast for product managers out there. Their amazing catalog of podcasts covers every imaginable topic a product manager would want to listen to. TIPM is produced by the team at Alpha, a tool for generating user insights. While TIPM is vast, I find the tone of the podcasts fairly monotonous. It's less conversational and more of doling out advice or product experiences. Nevertheless, it's a great place to start or revisit for certain key topics. 
Listen: https://www.thisisproductmanagement.com/

2. Rocketship.fm
Rocketship.fm does a great job of creating stories for each of their topics, which makes it a very enjoyable podcast. They usually have a recurring theme across a series of podcasts. It's a podcast that helps a layman understand the goings-on in startup life. While the podcast covers general topics, of late, they have been talking a lot about product related topics. There's a very interesting episode on the origin of product management which talks about the 'Brand Men' at P&G (circa 1930s). For the compelling stories, structure and conversations, I rate this podcast highly.
Listen: http://rocketship.fm/

3. Inside Intercom
Intercom is a tool used for customer support and in-app customer engagement. While I'm not completely sold on their product, I must say that they're doing a fantastic job in getting inbound traffic via their website. They've created a blog and a podcast which comprises of a knowledge base spanning topics on product management and design. It's done well enough that it doesn't feel like "content for the sake of content". Des Traynor, the founder of Intercom, usually hosts the podcast and the show has had respectable guests from the startup/design/product world. There are some great conversations in the podcasts, especially the one on the Jobs-To-Be-Done framework. 
Listen: https://blog.intercom.com/category/podcast/

4. Exponent
It doesn't get more credible than Exponent when it comes to the people behind the podcast. Exponent is run by Ben Thompson, author/founder of Stratechery, and James Allworth, author and Harvard Business Review writer. Exponent isn't directed at product managers but they often talk about over-arching problems, trends, innovations in the technology space. Whether it's AI/ML/Bitcoin, Amazon's dominance or Facebook's newsfeed algorithms, there is always something useful for the listener. The hosts (and guests) pick each other's brains on several topics every week. Prepare yourself for some great analysis and deep-dives into interesting subjects on the Exponent.
Listen: http://exponent.fm/

5. Product Popcorn
Product Popcorn is the Aubrey Plaza of product podcasts. By that I mean, it's quirky and funny. Product Popcorn is hosted by Kimberly and Adam, who will give their light-hearted take on product management interspersed with elephant trumpet sounds. What I love about this podcast is that you'll feel like it's a roundtable discussion with friends. Another aspect of the podcast that I love is that the hosts share updates on their own professional lives, something like a stand-up meeting update. It helps you connect with the hosts and what they encounter on a daily/weekly basis while PMing. I would 12/10 recommend a listen. 
Listen: https://www.productpopcorn.com/

6. The Startup Chat
With close to 300 episodes, the Startup Chat is an important podcast for entrepreneurs and startup folk. It's hosted by Hiten Shah (founder of Kissmetrics) and Steli Efti (founder of close.io), who have been hustling for over 10 years. This podcast pretty much covers all the bases on advice related to starting a startup, the modalities of running one and the specifics on key topics like hiring, processes and more. The episodes usually don't run over 20-25min, which makes it very easily consumable and retainable. I also love how Hiten and Steli can be at loggerheads with each other during debates on "controversial" topics. This podcast becomes more relevant if you're a product manager at a startup because you see these different flavors play out around you. 
Listen: https://thestartupchat.com/

The Hard Thing About Hard Things - Book Review

If there's one feeling that you will be left with after reading Ben Horowitz's 'The Hard Thing About Hard Things' is that you'd be kicking yourself for not having read it earlier. In my opinion, it is the definitive guide to doing business whether you're an entrepreneur, a sales guy, a product manager or a team leader in any department. 

Image picked up from Google image search

Image picked up from Google image search

One of the primary motivations for me to pick up 'The Hard Thing about Hard Things' is the essay titled 'Good Product Manager, Bad Product Manager'. This was first featured in the book and has been widely touted as a 101 reading for all aspiring product managers. And rightfully so. I had read that essay a couple of years ago and it outlines the qualities of good product managers and how most fall into the trap of certain bad habits. It's essentially a list of dos and don'ts. A good product manager makes a team a winning one, whereas a bad product manager creates negative value for the team. The essay can be read here.

But moving on to the book on the whole, it was an easy read. I suspect this would even be entertaining for a reader if s/he is not well versed with business jargon. Each chapter comprises of the choicest anecdotes from Ben's expansive career from Netscape to Loudcloud to Opsware to HP and finally a16z. All of these stories, if not relatable, will evoke a sense of empathy with Ben's narration. I love the honesty and the struggle behind each of the big decisions Ben, as an entrepreneur, had to make.

'The Hard Thing  About Hard Things' succeeds like no other business book because it embraces failures while instilling advice and leaving you with memorable takeaways. Sure, what you may face as a future entrepreneur may or may not mirror Ben's or Marc's (Andreessen) struggles but the learnings are timeless and many of them can be applied in different scenarios. For example, no matter what team you lead, your team members will have the base expectation of you not bullshitting them. Many team leads and CEOs fall prey to this. This is one of the basic principles that is hammered to the reader with many of the stories. Some of the other key lessons that Ben talks about is going against the grain of the executive management or board members. Important decisions often get swayed by group-think or risk-averseness. It's your job as a CEO/manager to champion the tough decisions especially when you have corroborating evidence of a positive hypothesis, whether data leads you that way or it's customer feedback or market research, what have you.

As you progress through the book, you will realize that Ben's narration pretty much covers the major events in the life cycle of a company. Some of those are:

  • Working in a startup and scaling it 
  • Product - Market fit
  • Fending off competition from incumbents
  • Excelling at sales
  • Knowing the true value of your company/product
  • People management as your company scales
  • Knowing when to go public with your company
  • Knowing when to get acquired

One criticism that this book faces is that most of the stories predate the current millennial-tech-ecosystem. Okay that's not really a thing (or is it?). But I hope you get my drift. The companies mentioned in examples by Ben aren't glamorous but they're hardcore businesses which survived the 2000 dot-com bubble, made hundreds of millions of dollars and one even being valued at $1.6 billion dollars in the mid 2000s. If those aren't great benchmarks, I don't know what are. Just because a Snapchat may have a completely different model, doesn't mean that their teams don't struggle with sales or people problems. A lot of the learning in this book stems from principles of doing business and handling situations.

The Hard Thing About Hard Things is not an entrepreneur's playbook to success but it's a warning to that lot that...well...shit happens! And when you realize that you've been dealt a bad hand, Ben proves that there are still ways to maneuver through them and come out standing. It's a book to be re-read multiple times! 

I'm going to wrap this up by quoting some of my favorite lines from the book. There are many to choose from but these quotes were an instant home-run for me and they left me nodding profusely in agreement. 

"Note to self: It’s a good idea to ask, “What am I not doing?"

"Take care of the People, the Products and the Profits - In that order."

"If you're going to eat shit, don't nibble!"

"A healthy company culture encourages people to share bad news. A company that discusses its problems freely and openly can quickly solve them. A company that covers up its problems frustrates everyone involved."

"Having dogs at work and yoga aren't culture!"

 

The 2nd and 3rd Ps of being a PM

I guess I’m ultimately calling this the 3Ps of being a PM. Very Kotler. Much jargon. In the off chance that you missed the first post where I talk about Prioritization being the first and foremost attribute of a product manager, you can read it here

At the outset, a product manager is required to have a very good understanding of the market, competition, business models, future trends apart from knowing the user thoroughly. The insights gained from the above and channelized into better decision making is what, I reckon, constitutes the second important P of being a product manager — Pulse

I may be oversimplifying it but with my limited experience and reading about product failures and successes, it is very important for the product manager to have her ears close to the ground. This enables you to not only understand the environment in which your product is being used/consumed but it also helps you predict/anticipate for the foreseeable future to a certain degree. 

  • Getting a pulse of your customers’ problems, needs, aspirations, behavior helps you build better products. Clearly, a no-brainer. 
  • Getting a pulse of your competition helps you identify your competitive advantage. Go a step further and talk to your competitor’s customers. It helps you beyond the realm of product management, say if, Company X is kick ass at marketing. Can those serve as inputs to your internal stakeholders? Or can there be possible synergies that can be explored to benefit you and the competitor? 
  • Getting a pulse on trends helps you build better products. Trends would encompass everything from market/economy to consumer/behavioral to technology. How would you incorporate these trends to make your product better? 
  • Acting on impulse helps you build products effectively and efficiently. Here’s a really neat graph published by Hacker Noon which hits the spot about what I’m trying to get at. This point transcends the scientific aspect of being a product manager and makes it seem like product management is also an art form. Unfortunately, this can be learnt only through experience, but here’s a heads up nonetheless. 

All in all, pulse, is a very fluffy word to describe what I just wrote but I believe the best product managers have this as an unsaid trait. It comes naturally to the curious and the ambitious. Personally, I’ve been trying to better myself at getting a pulse of the aforementioned things. I try to read more, I listen to relevant podcasts and I avoid any blinders restricting my vision lest I miss out on anything. 

So…great, you’ve nailed your product-market fit, you’ve validated your ideas, you’ve prioritized your backlog…what next? 

The third important P of being a product manager is Project Execution. Notice, how I don’t say project management. How a product manager brings design and technology together is what constitutes execution to me. Project execution would certainly vary from company to company. In more established companies like an Amazon or Facebook, a product manager isn’t alone in execution. One works with technical product managers, program managers, engineering managers, tech/dev leads and so on. But if you’re in a seed-funded or series A funded startup, product managers end up playing a larger role in execution. The scope of responsibilities vary, but I’m going to talk from my own personal experience. 

  • People — The success of project execution solely depends on your team. It is important for the entire team (designers, engineers, QA testers) to be on the same page. But hey, you’re a product manager. You have no authority over everyone else. This is where as a product manager, you need to play your ‘people skills’ cards well. I strongly believing that motivating the team before and after the project, irrespective of the size of the release, is key to having the team stick together. You show them impact, the potential of a product they’re building and how important each of their contributions are. A product manager must in no way take credit away from the team, after all they’re doing all the heavy lifting. Sure, you “conceptualized” the idea but make no mistake about it — your team will lose faith in you. 
  • Timelines — You will need to approximate timelines pre-development while sprint planning. You will need to get finer timelines from design and development. You will need to give out timelines with additional buffer, wherever necessary, to your stakeholders. Timelines are sacrosanct. There will definitely be times where you don’t meet these deadlines but it’s crucial to inform your stakeholders about any delay and possible alternative timelines. 
  • Development — If you’re a non-technical product manager, you would be solely reliant on your engineer leads or senior engineers. Invariably, engineers do come back to product managers with doubts, cases missed, cases where they’re block. It’s your role as a product manager to ensure that any doubts/blockers are swiftly resolved. You gain the respect of an engineer if you clear her blockers promptly. 
  • Testing — One of my strengths as a product manager is that I’m very hands-on with my product during development and testing. The more rigorous the testing, the fewer chances of your product failing in the user’s hands. Once the engineer(s) hand over the product to the QA team for testing, I step in between to do a pre-UAT (user acceptance test) where I try to ensure that the product is behaving as it should for the users’ major cases. I say ‘major cases’ because you are generally constrained for time. The pre-UAT testing phase significantly reduces the back-and-forth exchanges between QA and engineers. Once the QA team gives a sign off on product testing, I usually perform a UAT thoroughly to ensure that the product is ready to be shipped. 
  • Shipping — Here you have it, your feature/product moved from a staging test bed to your production servers. I prefer to ship out new releases on Tuesday, Wednesday, Thursdays or when the next day is not a holiday. That’s because in the slightest chance that there is a bug, you have the team available to quickly re-ship a bug-free version as soon as possible. 
  • Post-shipping — Is a product manager’s job done after shipping? Hell no. Sure you may have dusted your hands and got that awesome product released. While this doesn’t fall under the purview of ‘execution’, how else will you measure how successful it is? Are your users using the product as intended? Whether it’s a beta release or a full fledged roll-out, it is essential to choose the right metrics and have those metrics closely monitored either via data pulled from the database or analytics tools like Google Analytics or Mixpanel. A topic like measuring success metrics would require its own blog post. 

So there we have it — Pulse and Project Execution, the two remaining Ps in what makes my three essential Ps of being a product manager. While I can’t cover everything with this 3Ps post because, let’s face it, product management is a fluid field, part science and part art. 

I would love any feedback on this post. Do write in at me@vcent.in if you would like to talk further about product management. 

The first P of being a PM

The idea for this series of posts stemmed from a completely different discussion I had with a colleague on the differences between product management and program management. I won’t delve into that but what got my mind racing was me trying to introspect on a product manager’s role. After running through a whole host of responsibilities and attributes, I finally narrowed it down on Prioritization. I believe this is the first and foremost task a product manager has to be responsible for. I did cover this briefly in my introductory post of my first month as a Product Manager. 

Why is prioritization important?
As a product manager, you are measured against metrics related to your user’s goals. These metrics could range from Gross Margin Value (GMV) to Average Order Value (AOV) to Engagement (DAUs, Cohorts) and so on. There is absolutely no room for anything else if your features/releases aren’t impacting the metrics the most. It’s the holy grail. Intercom published a really informative read on prioritization. They subscribe to a model called the RICE framework — Reach, Impact, Confidence and Effort. Each of these parameters are scored and the overall RICE score determines the priority of a feature. Honestly, it cannot get simpler than this. But I personally also like to leave some room for judgement and intuition as well. What follows from prioritization is the product roadmap. 

Roadmap vs Ad hoc Requests
A product roadmap is created after you are done prioritizing tasks and features/releases. Depending on the stage of your company, you may have a roadmap for a quarter, a year or possibly even more than that. I’ve only worked for small to medium sized startups so my product roadmaps haven’t gone beyond 6 months. I believe that once you’ve gotten the items in your roadmap prioritized, half your job is done. But wait…

…and just when you’re mid-way feeling good about your roadmap going on as planned, someone from the marketing team makes a feature-request. Someone from Finance asks for alterations in certain dashboards. Long story short, it’s a balancing act between handling ad hoc requests and ensuring you’re still on course with the roadmap. On receiving an ad hoc request, I ask myself (and the stakeholder) these high-level questions:

  1. What is the impact on my user and the aforementioned metrics?
  2. Can I measure this impact? Short term gain or long term gain?
  3. How much of a pain point is it to the stakeholder/user? 
  4. Can I quantify or measure the pain point or criticality of #3? 
  5. Can it be resolved or shipped with a complementary feature which is already in the product roadmap?
  6. What is the effort (quantified by time) involved? 

There are more questions but I’ll stick to the above six. Based on these, I take a hard call on whether to pursue ad hoc requests at that time or post-pone working on it or shoot it down all together. 

What certainly helps is organizing your engineering team into developers who can work on long term features and shorter term features separately. I have seen this through experience as I’ve faced resource crunch working with small teams of about 5–6 developers. I may have groups of 2 or 3 developers working on the big ticket items which typically have a development effort of more than 1-2 weeks. On the side, I will reserve one developer who will work on relatively smaller features (development effort of ~3 days or less). If you can draw up a mental picture of a Gantt chart, you will realize that I am constantly shipping small features and/or bug fixes on a regular basis without having the major chunk of my roadmap affected. In light of this segregation of resource, I would then assign the ad hoc requests (post scrutinizing on whether to take them up, of course) to the developer working on short term features. In the worst case scenario, I would be able to get to those ad-hoc requests in a few days. There you have it, my approach on handling a roadmap with ad hoc requests. (Vincent’s secret sauce much?)

Dealing with stake-holders
By nature of the PM role, you will be dealing with multiple stakeholders and inevitably, everyone wants their requests as soon as possible. To be specific, I’m talking about internal stakeholders of the company such as marketing, finance, sales, operations and others. Prioritization becomes important more than any where else. As I had described earlier, there will be times you have to shoot down proposals or feature requests by nature of failing the ad hoc test. I wish there was a surefire way of coming out with a win-win outcome. It’s subjective and depends on the context of the discussion. I have learnt though, that being empathetic with your fellow peers is a step in the right direction. Back up your reasons with data, guesstimates or strong hypothesis to ensure that your stance defensible. Without these, it’s all about people management skills. On the flip side, do take into account that these stakeholders are equally important to you and the success of your product roadmap. One such example would be at the time of product launch, you’d have to coordinate with the sales and marketing teams to conduct demos with customers/sales representatives. I guess you get my drift, right? Product managers need to develop people skills to handle such situations. 

If there was a punk rock song about product management, it would be ‘Prioritization Über Alles which translates to ‘prioritization above everything else’. It determines the success or failure of your product. 

Stay tune for the next post — The second P of being a PM.