I am talking to Peter Dwyer, a product manager from Nielsen. Their approach, product structure, and product lifecycles are somewhat different from your regular software products. We explore that non-standard approach and things Peter and his team do in their context.
As always, if you have any questions or want to subscribe to our podcast – ask a question here
Transcript of the episode 4 (courtesy of otter.ai)
Vlad G 0:07
This is Real World Product Management.
Hello, and welcome to the next episode of the real world product management. My guest today is Peter Dwyer. Peter. Hi, how are you? I’m well
Peter D 0:25
I’m good. Thanks, Vlad. And how are you doing?
Vlad G 0:27
Thank you. I’m great. Hopefully we’re all staying healthy and sane in this new reality and new world. So can you please tell us more about you and your product management experience? What is it that you do when how you do it?
Peter D 0:40
Yeah. So my name is Peter Dwyer. I’m a product manager at Nielsen. So working in there. I’ve been a in product leadership for about three years now. But I’ve been a product manager for a little over a year. I kind of entered the role kind of worked my way through. I think everyone who listens here who is on Got a background in product management knows from experience that sometimes the product management role can kind of be a product of circumstance, if you will, I don’t know that many people naturally find themselves in the product management track, I don’t think there is a real well defined one it kind of finds you and to a certain extent. So I started as an analyst at Nielsen in what’s called the Nielsen bases division or the innovation practice, which kind of helps manufacturers around the world, evaluate new product ideas and bring them to market and kind of make some volumetric impact assessments or sales forecasting for these new innovations I go to market. So I started as an analyst work my way through into a sales operation role. So bidding and scoping one off projects. That’s what kind of what our business model was based on is individual projects, with deliverables and defined deliverables that are delivered to clients. And then I transition to product leadership and began as more of a business analyst role Before assuming more of a junior product management role about two and a half years ago, and then saving that role through the launch of a solution and service and kind of maintaining support. Getting a lot more know how and experience from the Support Portal or the help desk. I kind of felt like I was ready to make a move into product management kind of got elevated into the product manager role of that service. A little over a year ago, as I said,
Vlad G 2:28
Great, by the way, since you mentioned you that you participate in a lot of product ownership ceremonies, participate in sprint planning, and so on, as well as solving some high level problems leveraging delicious without the teams working on strategy. That’s more of a product manager rather than product owner responsibilities. In my mind, product owner, Product Manager completely different roles on two different levels. I understand that in your case, there’s some sort of symbiosis. You guys kind of merge them somewhere. Together, maybe you could elaborate a little bit on this for us.
Peter D 3:05
Yeah, that’s a that’s a great point to add. We’re pretty lean organization. So as I was speaking to, we were talking about earlier, our role, product manager role is a little different than probably what other people in the industry are used to, in my role as a product manager, not only am I responsible for the backlog in the roadmap of a product that we’re doing, that we’re working on, or that we’re assigned to or that we kind of pitch and get assignment towards, or do we win the resource battle. We also do maintain, we run stand ups. We also run through the grooming. So I’m actively writing stories, writing requirements, and at the same time, I’m going through and grooming them with my engineers to understand exactly what that they’re going through that size and estimate tagging for release tagging into sprint and managing the queue managing the Agile board with my team directly one to one while also being responsible for product health metrics and being able to assign or assign identified needs in the roadmap, so kind of keeping in touch with my commercial stakeholders in a regular cadence to understand what needs there are in the solution or in the platform, in my case, a little bit more on my my recent assignment. So it really is kind of touching go a little bit all over the place May, adding a little bit more challenge to the mix is that we’re in a global business. So kind of accustomed to the early mornings with Europe or Russia in some cases, when we got to be a little bit more Russia specific or less him. So working around a lot of time zones to manage that relationship, in addition to to everything else, so And lastly, kind of going hand in hand with the commercial relationship is also working closely with product marketing to align on like, something that we can do to get communication out, get the word out, bring new news to clients, when we have new features available. ton of working to establish that sales playbook or pitch deck when we need to as well understanding those market segments, we have a direct role on that. Almost like a vertically integrated role for the entire product experience a scrum manager, or Scrum Master product owner and Product Manager kind of all wrapped up in one roll. Again, we’re we’re pretty limited when it comes to that, and very similar to a lot of other organizations out there. But it’s just makes it a little bit more rewarding. I think it lacks
Vlad G 5:31
definitely makes it more challenging. So you saying you were watching product metrics? obvious question is what are the metrics that you’re watching? But before we dive deeper into that, what are the products that you’re working on? So we can understand why those metrics Make sense? So let’s break it down into two the four questions. What are the products that you’re working on? And which metrics are you watching and why they make sense?
Peter D 5:55
Yeah, so what do you do here? That’s the question. What work. So our business unit within Nielsen has a discrete offering of a multitude of market research products, or solutions as we’ve kind of packaged them up. So it’s a survey based research methodology with numerous applications. So we have discrete offerings, where there’s one format here and other format there. They have very specific individualized deliverables for that solution. And then they’re kind of sold as a project. One off basis. It’s not like not like a lot of typical SAS out there, where it’s based on licensing fees. These are kind of more of commodity that are sold or condition, their project engagements that are commissioned by the clients, and then they’re run, leveraging the platform that we kind of managing where we host these services and solutions. So that can be anything from qualifying a new, soft drink idea to a new bath wash, to helping clients understand the shelf dynamics in terms of like managing and integration pipelines. Let’s say you’ve got Multiple flavors of a soft drink, you need to define or decide which one is the best. We have a we have an offering in the platform that allows you to do that and kind of understand the revenue trade off for having an optimal line. And understanding what’s going to happen in the market when you swap out a new stock keeping unit or askew in what price points how to maximize volume, what what which price points maximize revenue, as well. So that’s, that’s a little bit more of what we do. So when it comes to product metrics, or product health metrics. Personally, myself, I’m pretty commercial oriented when it comes to that. So I’m really looking at unit volume, price per unit and total revenue or total revenue generated for the for the solution that we’re managing are under management at the time
Vlad G 7:50
going through throughout, but I just have to ask for my own sanity sake. What is the unit here is that a single survey is that a contract is that something else?
Peter D 7:59
You can think of it as a Single proposal for a project. So single project standalone as project manager, I’m looking at support burden. So we doesn’t matter if we’re generating a lot of revenue, if we have a lot of helpdesk tickets we’re looking at, that’s a support burden. We need to kind of reassess what the solutions got or what what what’s holding back the platform from delivering that without support burden. because it adds up means that it’s more man hours spent, we’re we’re pretty lean team. So if we’re getting pulled into tickets, a lot to kind of helping guide the conversation, and troubleshooting or trying to find a resolution for something. It’s less time spent on other work that we would like to ideally spend our time on.
Vlad G 8:42
So you effectively have dollar value or other direct dollar cost per ticket.
Peter D 8:49
Yeah, I mean, you can, you can pretty much come back to that if you do simple math a little bit like just assuming that no standard ticket for a project. It was going to be four hours of labor from our team. Maybe a little bit less than that kind of depends on where you kind of put the baby in the middle, so to speak, or draw the line. And then kind of averaging that out, you know, each project, kind of applying that equation that like, we got 500 tickets this month, we sold 250. That means we had two tickets for projects. that’s a that’s a sport, right. And it has a tangible monetary value that we could have. We could be doing with our we shouldn’t be working to reduce. So it’s just as important almost because a lot of the largest user base we have within our organization is internal. That’s not necessarily the clients that are executing these projects. We sell them the vision of having results within the within the platform for these projects, the commission, but the execution is mostly internal. So when we run into user issues a lot that means when we have internal associates that can’t use the tool correctly, or they struggle to use the tool or maybe the feature just isn’t there, which you know, to be honest with We’ve all I think you’ve kind of run into that issue where you define a feature, you kind of go through that discovery process, you ship. And turns out you miss a requirement. And depending on how important that requirement was, the blast radius can be pretty significant. And it can result in a pretty huge support, but and unintentionally. So we do keep an eye on that from, from our perspective, for from my perspective, at least. So kind of hand in hand, making sure we have operational excellence internally, but also making sure we got commercial excellence to when it comes to the product being in market or, you know, commercial viability, rather sorry. Making sure that we are where we kind of want to be from, from a commercial point of view.
Vlad G 10:46
Right, making sure you’re hitting certain financial KPIs.
Peter D 10:49
Yeah.
Vlad G 10:51
I like the commercial excellence as a term. I think I’m gonna use it in my conversations moving forward. Thank you. That was interesting. So you mentioned discovery and collecting the requirements process. So let’s, let’s look at this from a slightly different angle. As a matter of fact, let’s get to a higher point of view. And there’s probably a discovery process across the whole product lifecycle, you’re collecting requirements, and then do analysis and do other stuff. So what is this process? What are the challenges? Or what are your war stories around each step of this discovery? Or what are you guys dealing with? As the discovery progresses across different points in the product lifecycle?
Peter D 11:34
Yeah, that sounds good. Um, so So in our framework, for our software development lifecycle, we usually kind of work within discovery, development deployment, and that was kind of break out into a few different things, which I’ll kind of go into a bit more detail there. But another thing to note about our business unit is it’s kind of in transition So we’ve got this software application that we use to kind of service a lot of these solutions. But we’re also, we have a lot of legacy technology that we’re doing. Or we are replac, forming a lot of services that were kind of standalone consulting engagements without being like a specific product. We have to kind of draw the line and minimize variation and identify those minimal requirements to re platform that solution as an actual software offering. So we have to understand what is the survey variation? And what are the deliverables that are usually expected with that service and where the reporting needs. I mean, for anyone that’s kind of working in a business area or kind of a commercial services or consulting services, everything’s about a PowerPoint report. So we have to kind of have that in the back of their mind at all times when we’re forming these services. How do we get this down to a PowerPoint? Or what’s the expected delivery mechanism to the client So that’s something we have to consider. So we have to our discovery process is almost forensic. In that we have to understand the legacy application, the legacy processes, the legacy tools and teams that are that are leveraged throughout the execution and in the old world. And then we kind of have to size to fit into this new application that leverages the automation. So we think about a survey being programmed in a legacy world that would be programmed by a team or a vendor, and it would be published, it would go live in a production instance and collect the data. And then afterwards, we would have another team manage the tabulation of that data by hand. In the New World, what we would do is we would say, Okay, here’s the minimum survey requirements. We’ve defined these as what they’re going to be. We’re going to automate the survey creation and then we’re going to automate the tabulation of the data itself. But we have to kind of figure that out. A lot of ways where we have to kind of get down To the most granular level of the lowest possible level, the highest level of detail, the lowest common denominator, understand every process that’s going on in these legacy tools in teams. And then we use that to guide a little bit more of the software development, at least for the minimum viable product.
Vlad G 14:19
And that’s kind of like a race against the clock in our book.
Peter D 14:24
When it comes to defining the MVP, you kind of break it up into two platforms of discovery, I guess you got for your MVP discovery, like, just trying to like cut Jews and kind of picking, picking and choosing who’s gonna live who’s gonna die in terms of features. Now, and then that second part is okay, this didn’t make the initial cut. Why? Okay, I understand why. Now we’re going to put it in the backlog. We need a little bit more time to figure out what the actual value to the user for this. Okay, we got a value for the user. Is there a monetary value to me? Can we upsell for delivery or inclusion of this feature? Yes or no? Or is this an operational efficiency? Is this a margin play? Or is this just simply the cost of entry like something that we’re going to have to eat the time in terms of dev in terms of time and resources is it just going to be that minimum cost of entry and get our foot in the door to get our clients to adopt this new solution, because it kind of goes back a little bit more to our business. Like we’re business that’s in transition. So when we are replac, forming these services, it’s as much as winning over our internal users as it is our external clients. So we’ve got the name of the game does become a little bit of conversion. So it’s kind of our discovery processes. I wouldn’t say tainted, but it’s definitely informed a lot by that goal of conversion of adoption, away from a legacy service.
Vlad G 15:50
Interesting. And looks like you guys are doing great job given how small the team is. And listen, while listening to all of this. I still have this one question and this may sound like a interview. But please don’t treat it as such. It’s just the Curiosity because every organization kind of has it defined in a different way. So what is how do you define the MVP? What is an MVP for you?
Peter D 16:18
I think for us it becomes it’s kind of trying to harmonize what we think is the the MVP and what commercial stakeholders think is the MVP. And it’s just trying to get to a compromise somewhere in the middle where it’s, it’s commercially acceptable for what we want to push out to market. And that that’s always where the challenge comes in. Like you always have to manage those commercial expectations. And you have to you have to have a valid reason and you almost make it into a trade off situation. You try to give them enough of the carrot to say hey, this is what we think we can get done in we can get X amount of deliverables out and y amount of time the the z over Hear that’s that’s gonna have to come a little bit after, is that palatable? Or is that something that like, we need to spend an additional three to four months of dev work to get out the door? So it kind of comes down to getting getting the right stakeholders in the room to understand, Hey, can you get this in front of a client? Could you close the sale with just this in the spec of what this solution is offering? Now, it’s easier said than done. But sometimes it kind of comes down to it really is as much as like, do you think you can sell this? Okay, why not? If you have this in this or this, would you be able to sell it then. But it’s really, we have like, we kind of have a cheat sheet I guess in our organization since right now. We’re still on the transition of re platforming and transitioning a lot of this business to a product, focus mindset or product oriented delivery where we kind of have Way good established service Rolodex, if you will to pull from where it makes that a little bit easier. But for newer solutions that we have that are like, kind of a bit of a ways out, but kind of like the next suite of solutions that we want to deliver, or get onto the platform, it’s kind of like, I don’t want to use the term but like virgin products, it’s just like completely new in the market stuff that we want to build. That is going to be a little bit tougher, and that’s a situation haven’t been in quite yet. But I know that like, from like an MVP standpoint, we have a little bit easier, but it’s definitely it can definitely like pulling teeth sometimes with commercial folks to get it in there in the kindest way possible.
Vlad G 18:42
It’s understandable, at least from my professional perspective, from our professional experience. Sometimes we this is how I can relate to this. Sometimes we would show the business and we generally call them to business, could be sales, could be a PMO. could be anybody. We would show the business there. map and roadmaps are usually broken down into quarters so they can, they can deal with them easier. And everybody would get very excited. And we’ll we’ll show them the piece of nationalities that are coming down the road on the roadmap. And there’s always this one person, there’s always this one particular person that would look at the roadmap that was pointed to a single feature or capability or functionality on the roadmap somewhere in the fourth quarter, almost, at the end of a at the end of the roadmap, and it would say, get very excited about and said, this is this is the best seller, I can I can totally sell someone today, if we would have that feature. Can we have it now? Like what would it take to have it now. And by sheer luck, or I don’t know by an accident or, or or whatever? Yeah, that would be the only feature they requires this almost the waterfall sequence of events. So you have to make certain things happen. And that’s the only feature, that’s the only capability that actually requires four quarters of development. But that’s the only one they would care about. And that’s, that’s my next question. Really? How much? Sure, would you say your stakeholders are if they’re asking for this feature, this capability? How well do they understand that certain things have to happen in like, almost like a waterfall fashion? So, you know, they realize that there’s one thing that enables another enables another and that’s when the magic happens, or they they believe everything can be done by Friday?
Peter D 20:36
That’s a great question when, and it’s actually very prescient. So, five years ago, I don’t know if I would have been able to answer that question. Honestly. Just because it was a different setting. We hadn’t even launched the first real standalone solution. So the understanding at the time was very different from what it is today, whereas today we have the business unit Or president of the business that understands what the product focus is. And what it is even though they may not have exposure to this, the software development lifecycle, they at least understand it like, okay, there’s there’s trade offs to be made here. And it’s incredibly tight, like good timing on part of this podcast. But we been kind of on a full court press in 2020, from our organization to go out. And not just it’s not just the commercial stakeholders that need to be aware of the software development process. It’s it’s the users themselves that need to like, understand and internalize how how the sausage gets made, like everyone wants to make their client happy, and everyone wants to like their own private like mansion in terms of like what the software has offered for them. But you have to educate them a little bit more along the way in terms of what it is so we recently did an exercise. We kind of run this organization within the orange Are we kind of run this group of stakeholders within the organization we can refer to as our champions. They kind of disseminate some information via release notes or upcoming features we’ve got, you can never have too many points of contact with the org. So we just try to use them to facilitate that, that those channels as much as possible. But we recently walked them through a few features. And just like for compare and contrast, we walk them through a simple checkbox. Now what you or I would probably call like, a UI feature. But that UI feature that started out as three stories expanded scope into taking 16 stories and we kind of had to walk them through why that happened. And what that was and why that ended up being you know, something that would have taken a sprint to develop these three stories into why took four Sprint’s to develop it. Now, you know, Dev effort or dev approach is a completely different conversation, one that they don’t need to know about But under like making them aware of like, what the blast radius is of introducing something as simple as a checkbox into a UI that does something elsewhere, is pretty big. And then the second example we did was breaking down a recently released add on deliverable or a diagnostic add on that we we, we offer with a few products, kind of breaking that down and saying, Hey, this is like how a feature gets made, it starts out as one thing. So start from like, just generating a CSV file. And then from there, we work it up into 65 different other requirements that come together holistically to be generating a CSV file that is reflective of like, we produce a CSV file that’s read by this. It’s ingested by the platform, and it renders in a completely different fashion to kind of display the data that is produced for the client in this way. Kind of going from one to 65 and illustrating that journey has been huge. So I think now we’ve kind of also we’ve got the benefit of reaching that critical mass where the stakeholders, they’re bought into this, they they understand that software does come with a trade off, there is necessary tax to pay some time in terms of like, it’s better to bite the bullet now get, get something that’s acceptable to the client, and then wait for some of the better stuff to come along the way. Like it’s almost more important to prove to the client that we have a commitment to the software mindset than it is to deliver everything all at once and kind of that old school waterfall fashion.
Vlad G 24:38
You just think I love the part where you’re educating your users, your users or clients who you’re doing this for? Are your internal sales, right?
Peter D 24:46
Yeah, yeah. So we we call it we refer to them as champions. So it can be Client Services, operations, and sales and service are pretty closely aligned but operations is a is another pillar in the organization. Which kind of stands alone and kind of has their own priorities, similar like product leadership and sales and service?
Vlad G 25:06
This is pretty cool. I don’t hear about this pretty often. Usually, what I see in here is someone on the executive level saying things like, hey, I’ve been a developer 25 years ago, I know things down. You don’t need to teach me that. I don’t need to tell me that. So yeah, I am impressed. So okay, let’s move forward. You said that there are three steps that you guys are using in a delivery is a discovery, development deployment. And my understanding is, deployment includes the go to market activities. And again, if I understand your industry correctly, you’re developing things for specific engagements, then that’s when you give them to the customer, which is exactly what got the market is
Peter D 25:48
that that’s kind of right on the money. When we’re deploying something. We usually we know that the MVP for the service or at least the MVP for the add on that we want to make available. For the service, we’ve deemed market ready, and we deem it ready for client use and internal use at that point. So
Vlad G 26:07
let me ask you this and I understand that it’s a little fuzzy, this whole thing but for your deployment since you’re deploying solutions based on what you’ve been asked to do, rather than building things for the market, so there’s really not that much of trying to get to the market getting market share, you know, fighting for your place under the sun. I failed to capture that I failed to get the idea that you guys have a marketing strategy. Now we’re going to market strategy as a whole based on what a year based on what you guys doing, how you’re doing it.
Peter D 26:43
It’s it’s a bit of a dance. So I mean, in our in our portfolio, we’ve got a pretty expansive portfolio, but we’re kind of trying to streamline and standardize so when we have these software oriented solutions, they’re designed to kind of right now we’re kind of Tackling solutions are like the majority of our portfolio and then there’s like a pretty extensive long tail that we’ll need to address eventually. So when we’re doing these deployments, they’re kind of these these solutions that are designed to meet in at scale the needs of multiple clients on a global level, to answer one problem, or offer a suite of diagnostics to, to offer analytics for one problem. So thankfully, we’re kind of exempt from, you know, kind of building these one off pieces of software for clients. But when when it comes to deploying these new services to market, it becomes a bit of a full court press internally and externally. So we’re working with Product Marketing, to identify the account segments that we need to go after. So that’s kind of hand in hand with with the commercial stakeholders. We’ve got to develop a pitch deck, we go through an exercise with our sales organization to understand make sure you know how to sell it. So we kind of use this program that we we sit on and we help it and we kind of poke and prod when the sales team goes through it. We call it Pitch Perfect. It’s been something that’s been working pretty well within our organization. So we just coach them through 15 minutes or they get an opportunity to pitch for 15 minutes, we give them some pointers and kind of try to poke holes and see how well they actually know and understand what the solution capabilities are. We do a pretty big mailing or we do a pretty big communication Blitz in what product marketing is able to do, and leveraging them in that way. And then, on the internal side of things, were going through that, making sure that we’re doing end to end testing. With the internal users, we kind of do all day training sessions around the world for our associates that are getting trained up on how to execute these new solutions. So the client service users or operations, users weren’t really doing a full court press right off the bat for you know, up to a month until we go to market. That way when we turn the switch on. We’re ready to go. We have leads generated from the marketing blitz that kind of Pre precedes the internal training. So we get that critical mass we get the salespeople trained First they go out, they start pitching. By the time we finish up training with our internal organization, we’re ready to kind of go. And then we’re kind of in that deployment phase where we got a lot of round the clock production monitoring, make sure that nothing’s going wrong with a solution when it when it’s going to execution. And we’re also just making sure that we’re getting enough leads through the door that we got critical, like critical mass critical lion’s share of our internal teams to make sure that they, they’re, they’re doing their part what they can to make a commercial impact for it. And then that deployment kind of tapers off after we get through a certain amount of reps. Like there’s going to be that support spike. And then the user base kind of gets used to things and they get educated, they kind of learn from mistakes, or we kind of learn from our training mistakes. We make it a point to like re educate as needed. And then we kind of see that support burn and kind of taper off a little bit and then a little spike when we get a new hire class and or it’ll spike when we get a new market or a new methodology supported for some of the solutions. So the deployment can definitely be the most fun part, like discovery and deployment are definitely really enjoyable deployment, it’s so rewarding to get it out there. Because in organization, it’s just kind of like a big event. It’s like kind of a, you get to be the belle of the ball kind of traveling around you. Help people get excited about the product as much as you can. Gives us that’s kind of our role too. We we’ve got to be evangelists for what we what we’re working on, we can’t just be doom and gloom or kind of take our lumps a little bit in the back. So that’s a bit one bit of it. When it comes down to as far as deployment, it’s a lot of education, making sure documentation is up to snuff. or in our case, sometimes you have to develop internal tools or like an Excel document to kind of guide some things. Now that may be a little counterintuitive, what product management but when it’s engagement that runs all the time Mind, sometimes you have to make those sacrifices to ensure that like you’re delivering on time to a client, you got to develop a tool to make sure that your teams know what that timeline is put into a proposal. So it’s a lot of developing those like software tools that we we will use as well. And making sure those are up to snuff during the deployment phase.
Vlad G 31:19
Makes sense, and you said you have to do the door and you have to do some other stuff. Is that a part of your product management responsibilities, or is that something else
Peter D 31:28
I was reflecting on my past experience. So we we have a pretty unique structure again. But we have this role that we refer to as product operations. And they are kind of like Junior PM, but they’re doing a lot of this documentation, internal documentation, or they’re doing a lot of a little bit of the legwork to kind of help the pm out. So in my case, when I was a in product operations, my role was a little bit more oriented to preparing things for certainly automation as much as it was contributing to discovery Where I could, it’s doing a little bit of this, like pre wiring for the organization to be ready to go. That, but the product manager in my experience, when we’ve done trainings, I’ll definitely do a training here and there, it’s a great way to become more visible to the organization, again, like education is a huge priority for us in the past year or so. So being able to be visible to the org and making sure that they know that we have a product management team, and that they can, that you can they can kind of put a face to a name, so to speak, is invaluable too. So those training sessions are a really good vehicle to build that credibility within the organization.
Vlad G 32:42
And since we’re talking about responsibilities, and specifically about product management responsibilities, I wanted to ask you as it pertains to organization, about the product management role. I keep seeing it in almost every episode, that product manager is a thankless job. If the product is successful, then it was a team effort, but if the product is failed, that’s it. Your failure, your individual failure. So what does it look like from your organization? Well, where do you guys stand on that?
Peter D 33:08
uh, it’s a little bit different. I mean, part of the pitch that my managers kind of made is like you get, you kind of get a target on your back. So you like you. the good and the bad is that you become very visible in the organization very quickly. As much as it can be a painful, thankless task, like there’s definitely that there are definitely those days where you just feel like no one knows anything that you’re working on and you feel underappreciated. But more often than not, you do become the go to person for everything. So you when there’s a cset that comes through for your product, but people will forward that to you and be like, this is great. This is just what the client needed, like, like your work is invaluable. Kudos to you for working on this but when something goes wrong, conversely, you are also the first person you’re kind of Johnny on the spot to fix everything. So if you’ve got a production issue related to your product, you better be on call, you better be able to figure it out and you better be able to triage and get through this and come up with a solution and a resolution to move beyond that. And salvage the engagement if you if it comes to that those dire circumstances, but we do a lot of communication where I kind of work, we put our name on things so people know that when something goes well, they can reach out to us and they do reach out to us. Or when something goes wrong, they know exactly who to go to, to kind of chill out a little bit too. So you get as much as the glory you get all everything else that comes along with it to you become. I’m trying to think of the word to describe it, but I think I might have had a better app description for it but you get a lot of as much as the guts you get the glory too. So it’s kind of take it as it comes. It’ll kind of change day to day Interesting.
Vlad G 34:57
Interesting definitely has not been my experience of Least almost never. And I’m really happy you guys have it this way. I’m happy you guys have differently. I really am. And from, from what I’ve seen this one of the things that I’ve heard that scares people away from being a product manager, you know, it’s kind of like, what do you mean no one ever stops by and say thank you. Yep, that’s pretty much what it is. And that’s great that you guys have a habit this way. So I want to roll back a little. And you mentioned that there’s a, if there’s a mistake, there’s your education, and that you guys have been a big on education for the past year and change as the whole. So I wanted to ask this question. Whatever happens if you guys have deployed something, and it was a failure, and how you guys deal with it?
Peter D 35:43
Yeah, it’s happened. It’s happened. We can’t all have success stories. I wasn’t the product manager and I wasn’t able to contribute much but when one thing that comes to mind is we release application for general general availability a couple of years back. And the QA was a little bit different. It was a it was an It was a project that had changed ownership several times. This was this was about a two year project that took about five years spring to general availability release. So definitely a labor of love by the end of it. And there were just some very different practices on that team at the time. It was a very interesting team dynamic. So when it was released for general availability, the application which was mostly leveraged by internal users, was definitely just not ready for release. I can’t comment too much on the QA, but it was just done, not the right way to the point where the UI would be your responsive to a user’s action. So like there would be the UI was not talking to the Back in the backend was talking to the UI. So the application itself was not usable,
Vlad G 37:06
was that a design flaw was that the application not tested properly? What actually went wrong?
Peter D 37:14
It was definitely testing practices. And that was partially due to some of the relationships on that team. So it’s a little bit of a mix of vendor versus internal. So kind of lost in translation kind of thing going on there with with an organization that had really didn’t really, it was an inexperienced product organization. And we kind of inherited that coming from a more experienced organization. marginally more experienced, rather. And it was released kind of in triage mode for about a week to the point where it had to be rolled back to the previous application. Now this was supposed to replace and then it was pretty much two months of doing things the right way to get it back out. To release date, that’s how that that was pretty big. Personally, I have a few issues with a with the solution that I brought to market where you know, the Bucks got stopped somewhere. So in the case of like, pretty, not so great circumstances, going through raw data to kind of qualify some things or being able to be on the spot to help triage and engagement that’s going on in Japan. definitely been in there or being kind of boots on the ground for engagements in Europe in early morning hours to make sure that those operations personnel are able to do their job and learning from mistakes as quickly and as efficiently as you can to, to make sure you write down things. That’s definitely happened or I believe heavens knows that. You get a release out and you know, stuff happens And maybe it’s a missed requirement or it’s maybe a QA mess. Watching the nonce and this requirement from the pm side or the PEO side, depending on what your organization is like, and you identify the need for the hotfix. And you’ve got to, you got to manage that you got to go through the senior QA engineer to get clearance to get your hotfix deployed, you have to get grilled and
Vlad G 39:23
you said you got grilled on that? Is that like lessons learned? Why did that happen? Kind of a grill or?
Peter D 39:29
Yeah, you got to do that like SWAT meeting right afterward? Or, or it’s like in the go, No, go say, okay, hey, we got this branch ready to go. We identified the issue. It’s okay. So why didn’t you do this in the first release? It’s like, What? Oh,
Vlad G 39:45
right. And I’ll let me ask you this. Since you already mentioned that you have dollars tied to your support, therefore, you’re you can measure support calls directly. How do you guys handle support? What is what is your approach? How do you How do you do this?
Peter D 40:01
Yeah, typically we we leverage our product operations role to handle support. So what I mean, we’re lean organization and we put it in perspective, we, as an organization worldwide, we’re probably around 1000 users. So we’ve got about three to five people at any given point, sporting all that input, all that intake. Some of that some of those 1000 users still leverage a lot of legacy applications. So we would probably say about 50 to 60% of that critical volume coming through. But we typically leverage the product operations role as our support liaison, but in certain cases, kind of a little bit more strategic. We still get like tickets around like feedback or like, Hey, why don’t I have this feature that necessitates the need for the product manager to seven or in extenuating circumstances where I maybe have a bit more domain knowledge than the product operations person does at that point. So like, I may know a bit more about a service In the in the product or in the tech stack that they don’t, and I can know right away, like, oh, I’ve got to go out to this engineer who’s on our support team. We got to triage this right away, it should be like, I can kind of, kind of like, get to that point where like, it’s like a sizing exercise, like you understand just how big of an issue is by like looking at it and maybe talking to an engineer? Like, is it a small or is it a medium or it’s like this gonna take like two or three people plus DevOps to figure out, you kind of get like a feel for that, which also ties into support, we have a rotating system we refer to as l three, where a member of our onshore team will act as the kind of on call engineer, in addition to their other responsibilities, but like l three, their support burden kind of takes precedence. We rotate them through on a weekly basis for all of our teammates that are on tour.
Vlad G 41:53
And that’s pretty standard situation, at least in most of the delivery organizations. So let me ask you this. From the product standpoint, and that’s why I was asking about the support. So I would understand what feeds back into the product development. Since we’ve already talked about the MVPs. I understand you guys are developing your products after the initial release, you keep developing features after you’ve released the initial release, or the MVP. So you constantly supporting the product on the market. Once it’s been out there. What do you what is your product lifecycle how you define the product lifecycle? On average, what is the product’s lifespan? Is it months, years date decades? How do you decide then when it’s time to retire an existing product?
Peter D 42:38
Yeah, that’s a… wouldn’t say retire necessarily, but we definitely reached like a metric, like a maturation standpoint, and I think right now we’re kind of in a yearly cycle, or like a year as long cycle for this. So I’m the product that I’m kind of coming off of being the pm That has been my life for the past two and a half years. And it’s like an off that it’s now kind of reached a point of maturity, where it’s going to get kind of included into another team’s stack or in their product stack to kind of manage. And I’m going to be transitioning to a new project, the the flagship solution that we have in the platform that has been in market for about four years. And it’s probably just now reaching the point where we can say that that is, or maybe it was around the three year mark that it really started to reach maturity. Like we’re, we’re a little bit of an interesting business in that like, it really comes down to adoption. So once we reach that critical mass of adoption, that’s kind of where we draw the line of maturity to kind of move on and kind of go into maintenance mode as opposed to like feature mode. So right now, still on like a pretty big macrocycle for that, but in support. We’re actively taking feedback if we can And add on out, that’s not hitting the mark right away. Like, I think there’s that quote, like, if everyone loved your first release, you probably waited too long to get it out. So it’s kind of the way that I’ve approached like the feedback tickets in terms of God in the backlog. There are definitely some recurring themes, but we try to get like trying to have like a residual backlog review at least twice, if not four times a year, where we just kind of take a step back and be like, okay, here’s the ticket volume on this request. Is it worth the time? Or is it really client specific and it would kind of like handcuff stepwise do that. And then we’ll periodically have those conversations to kind of revisit or do we have like a back breaking bug somewhere that’s just been lurking that we’ve been ignoring like what are the ankle biters? Do we need to like spend a few Sprint’s and just tightening up everything that that will definitely that’s been a little bit of my focus as well, like, do we need guardrails to introduce based on or user errors that we’ve seen in the system? Like how do we address that? Can we solve that with a product rather than just excessive documentation? So that’s, that’s something that like, is a good support learning as well, from from my perspective?
Vlad G 45:14
Uh huh. And here comes the tricky question. There are different names for this. So we’re going to use features and capabilities. And let’s stick with features for simplicity sake. Have you ever had a feature of a product during the product development that you absolutely wanted to take to the market, but were unable to do so? And why?
Peter D 45:35
Yeah, got a few. There is one that comes immediately to mind is there was an add on. When I was elevated into the product manager role. There was an add on that was kind of in flight. And I wanted to take it to market we had done we had gotten heavy commercial signal that this was kind of like a make or break deliverable for several clients in our region. It was going to unlock that region to kind of a different user base that we hadn’t been And exposed to. And it got to the point where we were, we thought we were ready to commercialize, and we couldn’t come to agreement on price. And then the loudest voices in the room for the commercial stakeholders couldn’t align. They couldn’t agree that it was actually valuable anymore. So we’ve done the def work, we’ve got the feature flag on, we’re kind of waiting to go. And it’s kind of one of those things where it’s still there. We could still turn on tomorrow if we absolutely needed to, but we couldn’t get the signal validated. We kind of it was kind of one of those moments of where we acted in good faith with commercial. And maybe we should have done a little bit more homework to kind of offer that bit more. But we we acted on signal we did what we thought was the right decision in the moment. And right now, it’s just it’s just not the most necessary feature.
Vlad G 46:55
And this is interesting. How do you approach the validation is that some kind of data driven decision As a subject matter experts is that product managers gut feeling, how do you guys approach this?
Peter D 47:05
I would say, use the phrase data informed, because sometimes you don’t have you don’t have the data, like what do you do? This was one of those situations where we knew how the deliverable was configured, we could we could reverse engineer the legacy deliverable to to, to meet the needs of the product or the outcome that we wanted to develop. But it was in a pretty operational silo that we couldn’t get the information. So we had to rely on commercial commercial feedback to go ahead with development. It was something that we just, it was it’s just one of those situations where you, you got to make decision whether or not to deliver this feature. And the voices at the time were louder than what the data may be suggested or the lack of data suggested. And we kind of took a chance and we kind of learned the hard Wait for that. But it kind of goes back. I think we were talking before we went into the to the recording. But I’m like, What do you do when you don’t have data? You kind of have to ask why a couple times to commercial people. And sometimes, you know, commercials focus at the end of the day is to hit their number. And sometimes their feedback can be most influenced by what’s preventing them from hitting their number at the time. So you can kind of see these spikes where something is make or break, and then what by the time Deb is finished, it’s like on to the next object or onto the next target for the quarter. And that can be kind of tough to stomach and kind of let it silently go away knowing that like, dammit, I wish we could have done a little bit more to ensure that like had a little bit more valuable value to more users. And I wish we could have done a little bit things a little bit more differently in discovery to validate this before. Really Going through it. Or you kind of do the post mortem to understand like, Okay, why did this work? It was their data that we missed. Or like, how could we avoid this? No one wants to work on something that just never sees the light of day.
Vlad G 49:16
Wouldn’t you say it tremendously increases the waste. I mean, you’ve worked on this full time. And at the end of the day, after you have delivered it, the feature is there and you just never turn it on. I mean, it’s there, you just need to flick the switch. And that’s just no one letting you to flip that switch. So my question is, does it create huge amounts of waste. And maybe there are other ways you can you can work this out, maybe you can work through certain things and reduce that waste through experimentation, prototyping, market testing, got an A B testing in the market, a B testing of the features, anything really, that it would turn not only will decrease the waste and will also increase your chances of things being approved.
Peter D 50:00
Yeah, I mean, ultimately, this was a, this was a really commercially specific case where we, we went through that UI UX work stream, we got the deliverable in the way that it needed to look, we thought we addressed the user needs. And to your point, like, we’ve got, we’ve got code out there that just, we haven’t been able to turn on for a while. And you don’t want to do that from from a from a developer from like a team management point of view. You don’t want to necessarily communicate that to your teams all the time, or, or you need to like kind of walk line in terms of how you communicate that to your teams. But it really does call into question like, Okay, what could have made this a little bit more successful? Or like, how can we how can we revisit How can we like rethink getting us to market How can we have the acceptance criteria shifted, in terms of turning on that feature flag?
Vlad G 50:55
Interesting. In one of our previous episodes, we talked about experimentation and fails during the experiment. And we talk about the general notion of how experiments are usually negatively perceived by the upper management because they tend to fail and and to us, it’s normal. The experiment failure is good, because scientists tell us, there are no failed experiments, they are data rich experiments, and that we are naturally really learning from our failures. And it’s kind of understandable. If you’ve guessed all every step of the way, your experiments a success and you just moving on if it failed, you get to learn about new edge cases that may be useful later. You know, some things that you haven’t foreseen haven’t accounted for. So in general, you arrive at the end with a lot more information about your product, promote product development process. And in upper management, the experiments perceived negatively because it paints them in a negative light. Oh, you’re failing. Yeah, you’re failing. Yeah. That’s not good. And Overall, what you’re saying. And what I find genuinely interesting is that even though you have developed features that were wanted and needed by the customer, they never made it to the customer. And even though you’ve done the lessons learned postmortem, you have figured out, what could have done could have been done better. It doesn’t sound like you got any amount of negative flak from upper management saying you guys wasted the effort or wasted time and resources. And this interesting because in my previous experiment experience, and this is just the indicative story that I was developing prototype of a product that was parallel to our main core product offering. And I’ve heard literally from everyone except one single person who sponsored the r&d work that I was doing, you wasting time you should be focused on things that customers want today, instead of doing this, and when I was done, it became obvious that instead of focusing on the few customers who are giving us marginally better ROI, we’ve arrived at the product they gave us an additional 60 to 80% of revenue per year. And again, in your case, I’m happy that you didn’t get as much negative flack as I got, which is, you know, it’s a good thing. Your management is understanding. I’m just trying to understand what was the overall sentiment around the corner office if the corner office is still a thing? And how does it even work that you’ve developed something you never get released to the customer? And it’s okay.
Peter D 53:25
I think, you know, time with our macro cycles being pretty long, to to a certain extent. I think what first of all honesty goes a long way. My experience, so if you’re just upfront with the, with the right stakeholders, they can usually they’ll critique and they’ll probably take it up with the manager, or my manager or our unit leader, and kind of express their concerns, in a way but that’s happened before. Like when discovery goes awry. I’ll definitely get some flack for For the case, this is probably not what you should be doing. But this was a case where, you know, it was already in this deliverable and I think that should have been our first hint. To really question whether or not was the right decision to make. But I’m are already the second point going kind of pain in hand with Icos, like our organization has been remarkably patient. So the MVP for our flagship product compared to what it was four years ago to what it is now is is night and day different. And the organization has kind of come around to understanding that things come to those who wait like MVP is truly MVP sometimes, and we just have to iterate off of it. It maybe it’s a unique circumstance, like it sounds like it is I know that for this one that has that never saw the light of day. They’re equally frustrating things where there’s There are other products out there where people are definitely like pissed off. They’re like something that’s not out there or something just missed the mark completely. thing it’s talking to just like organizational patience and understanding that like we’re on a journey transitioning from a business that is anchored 10 years in the past and preparing it for being 10 years in the future. The You know, it was a one off circumstances does not happen. Usually, if something goes wrong, we will be the first to hear about like the the app that I was talking about yesterday or earlier. yesterday. You know, compare and contrast my one off little add on deliverable with this application and being just unacceptable for general ease. You had you know, the entire season like our call it C suite if we were an independent business, but the most senior stakeholders in the business unit we’re pretty much on the phone with our product leadership organization, just trying to understand what How happened here and why and this is unacceptable, you need to understand, like, there’s that negative failure, aversion to failure that comes out. And I think there was something that’s like still below the surface a little bit. But um, we’re in an incredibly client sensitive business where it’s like you only get one shot to make a first impression, and then the client may not want to talk to you for another 12 months. So there is like this aversion like, if it’s not ready for clients, that might just do what you have to do to make it client acceptable. And that will take it from
Vlad G 56:37
the so don’t release it until it’s ready.
Peter D 56:41
Exactly, and that’s kind of you know, that’s kind of influenced my approach as a pm like I make. I consider each bug that we kind of get through into our release, like, is this a blocker? Yes or no? Or is this edge case valid? It’s like, yeah, it’s valid, but like Or maybe it’s like a super edge case. But in this case, like, I want to avoid kind of shipping bad code. And I want you to avoid having bad code go out under your name. So let’s take the time let’s do the due diligence. And just make sure that like anything that goes into release is buttoned up top to bottom.
Vlad G 57:16
So we’re nearing the wrap up of this episode. And I want to ask a couple of standard questions that I have for each guest on my podcast. On first these obviously, what do you think about working from home working remotely? How does it affect you if it affects you, given our current situation?
Peter D 57:34
Well, you know, the afternoons have been a little bit different. Like, I’m used to like my day starting I used to do in the mornings from home with like stand ups at a beam, you know, usually split between onshore or offshore. So you’re going on India time or talking to Europe a little bit earlier in the morning, but the afternoons have been where it’s just like, I really feel like I should be in the office right now. I’m kind of going crazy working from home all the time non stop. I don’t I don’t even Like I don’t like not being able to walk down the hall or you know, my UI UX designer is kind of at the end of our bench seating, or my, my product leader is sitting next to me in the bench. So it’s just like not having those people there to kind of just bounce off random ideas and just like, gut check a decision or like, get a quick hit, or a quick read from someone in the office really sucks, or being able to talk to my lead. My lead and my QA lead are both in our office so it’s just like not not being able to go over and ask them like, hey, how’s testing going? Or like, Hey, is this like story makes sense? Does this like scope make sense or anything that’s in here that you want me to like? Like a certain thing we can do to improve this? Not having that it’s definitely coming through a little bit more. And that’s been challenging.
Vlad G 58:44
Yeah, that makes sense. I I personally had experience working from home anywhere from a single day to just a couple of days, to full time to having been being in the office full time and I get it. It’s hard to change your habits when you can have face to face. So the subject matter experts clients or key members, and slack or teams or whatever it is, if you don’t have right habits, and sometimes it’s just not. It’s just not that close or not that personal and you’re feel like you’re missing out on something. So yeah, I get it. This is challenging.
Peter D 59:19
Yeah, the biggest thing is also, like, missing out on users. I mean, just being able to walk and do like a quick gut, gut check what’s on like, internal. Okay, what do you think of this?
Vlad G 59:30
And yeah, if your users are sitting in the next room, or in the same building, at least, he’s definitely easier than trying to connect to them. If they’re thousands of miles away. in a different time zone. I can definitely relate to that.
Peter D 59:43
Yeah, I guess makes you appreciate the luxury that you have that makes you appreciate the little thing, but overall, it’s been like, kind of same old, same old.
Vlad G 59:52
Great. And the last question for today. Actually, last question for me, is, do you have any questions for me so let’s see. turn the tables and asked me anything. Let’s just not try to solve the world hunger just yet. Let’s try to limit ourselves to something that can be answered in a few minutes. Yeah.
Peter D 1:00:11
You know, in your vast experience, I guess, I mean, have you ever had to like ship, like a product that kind of blew up once it made its way into production or like in, in general really just missed the mark kind of going completely against the grain of everything and discovery? And if so, how did you kind of manage that situation?
Vlad G 1:00:35
Thank you. That’s a great question. Yes, of course, obviously, you know, every product manager has failures has flops, has things that didn’t work out. This one in particular that I wanted to talk about. It was developed as a part of the concept of Endless Aisle. So I was working on the product that was a retail point of sale system. And it was popular thing back in the day to have an endless aisle. So if you order something online, you can get it to the store. If you order something in the store, it can be shipped to your home to your home. And it just the concept of Endless Aisle is that you order anywhere you receive anywhere. And part of mentality at the company I was working for was can we get it for free? Whatever that is, can we get it for free? Why do we have to pay for it? And I was looking for solutions that would allow us to do things inexpensively or for free or somewhere on a you know, shoestring budget. We were able to find a few solutions that worked for us. I developed that into a prototype. We’ve built it as a as an MVP. So we figured that these things are working. We started doing customer demos we started showing you to the real customers, we actually had a customer who wanted to buy it and he was willing to sign the contract for a year to start paying for it. And the whole thing came down crushing. When we figure it out, it is enabled to do certain things. And developing the required integrations would take a lot longer given the new idea that companies management had. So a basically literally in the middle of a road, we’ve decided to change wheels and everything kind of crashed. And it’s it wasn’t that that wasn’t that problematic. We were able to figure out what to do with a client. We were able to kind of gave them give them different incentives or different things to kind of please Him and alleviate the situation. Of course, as I’ve said, you know, if it’s a failure Yours, I failed to develop a product, which was a good thing because we figured out through the true experimentation in the market, we figured that this is not the the activities, this is not the product that we want to be in because it doesn’t really align with what a company wants to do. And at the same time, we were able to figure out that it wasn’t the product that was wrong. It was the market niche that we’ve attacked, that niche was wrong. We’re going after kind of low to mid level market, because we thought, but that’s where the numbers are. But given the complexity of the task, given the complexity of the problem that we’re trying to solve, it would actually make sense to go off their upper segment of the market and instead of having a product for as a software product, it makes sense to have it as a service. So we’ve instead of building a product and selling it to 5000 10,000 hundred thousand customers on the cookie cutter approach, we’ve dropped that that considered a failure. But we were able to transform that into a service offering. And through the service offering, we were able to make a lot more money. And we were able to get higher caliber customers. And we were able to stand up a whole new practice, which is, again, I would say it’s kind of one of those things where you have a failure that you turn into success. But from the product perspective as a failure from prototype, going to market, it’s it’s complete failure, because we did not achieve any of the goals. But as I was saying, every failure is a is an experiment reaching data. We were actually able to collect enough information to make things right to turn things around, make things right There was not a product it was not even you know, was not a my game my part of the town anymore. But we were able to use the information that we’ve collected and get something useful out of it. So Did that answer your question?
Peter D 1:05:16
Yeah, yeah, actually, do you have time for one more? Do we have
Vlad G 1:05:19
more? Absolutely. Like I said, we’re not limited. There’s no set limit to length of each episode, at least not yet.
Peter D 1:05:26
It seems like we got a pretty big experience across industries. But what would you use for a KPI for something that is not necessarily revenue generating? How would you gauge you know, what, what defines like a healthy product or a successful product? And in your experience when it’s when it’s come cases?
Vlad G 1:05:45
Well, that’s a great question. Thank you. So let me start by saying there’s a number of frameworks that you can use. As an example, there’s one metric that matters. There’s AARRR, or pirate metric. Then there’s Google HEART. But if I had to pick one, just one KPI instead of a framework or instead of number of metrics, I’d say it’s an adoption. And it’s pretty easy to understand that if the product doesn’t, doesn’t generate revenue. The only other way you can judge, judge if the product successful is if people are using it, if customers are using it. And that’s ultimately why you’re building a product, you’re building a product, not because you’re bored, although there are people like that you’re building the product because somebody needs a solution to a problem. And if your solution works, if people like it, then people are going to use it if your solution doesn’t work, that no matter how sophisticated complicated or elaborated your product is people not going to use it because it’s a bad product. The product is not successful. And I’ve seen this happen. Many times when people will force the use some sort of a solution they will force the use a specific Data Storage platform they were used to force the use a specific CRM and they hated it so much that they ended up using their own tools they were using Excel, or they ended up using on licensed or on, not allowed or prohibited methods of exchanging the data thus, endangering business continuity and security of the business. Because the product was not was not good enough, the product was not user friendly, because the product ultimately was not successful even when users are forced to use a specific product. So as I said, there’s a number of frameworks. There’s number rose weight and number of ways to measure it. But if I had to pick a single KPI, I would say it’s adoption.
Peter D 1:07:49
That’s, that’s really helpful. I’m transitioning to a project that’s a little bit more internal focus and not necessarily revenue generating more more geared towards usability and satisfaction. So it’s just helpful to get your point of view on there.
Vlad G 1:08:04
Glad this answer was useful. And that’s a wrap. That’s our episode. Thank you very much. Peter Dwyer.
Peter D 1:08:10
Yeah. Thanks for having me. Boy, this was fun.
Vlad G 1:08:12
Thank you for being such an amazing guest on our show. Hopefully we’ll hear you many more times. And I wish you luck in your career development and as always, feel free to get back to us with any questions. Any questions for Peter or myself can be sent to the email askvlad at VGrubman dot com or visit us on the web at https://vgrubman.com/podcast/
You’ve been listening to the real world project management and I’ve been your host Vlad Grubman – Until next time!