All posts by Vlad Grubman

Real World Product Management – Episode 17

July 26, 2021 Vlad Grubman

In this episode, I am focusing on just two aspects – what is (and what isn’t) a good product manager and interviewing product managers.

Also, I highly recommend watching this episode on YouTube as well or watch it in the same YouTube glory as an embed below.

You can listen to this podcast on any of your favorite podcasting platforms:

This episode transcript:

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Vlad G 0:16
This is Vlad again, and you’re listening again to the real world product management, I apologize for missing. At that time. I know I promised to release more episodes, and I’m releasing less episodes, then, once a week, I was sick. Sorry, got better, you can still hear me speaking a little bit in those but should be sounding pretty normal right now. So we’re back to our regular scheduled programming. And today, I decided to narrow down everything, and only talk about two things. Number one, what is and what isn’t a good product manager and number two, interviewing and hiring a product manager. Why those two? Well, because I figured it would be a good idea to narrow things down and dive deeper into things. And to if you define what a good product manager is, it will allow you to interview and hire better. At least that’s my understanding of things. And that’s why I the way I’m thinking about this. So what is and what isn’t the product good product manager. We’re not going to cover everything today. Obviously, it would be insane. If we could just talk for an hour, define what the hell a good product manager is. And that’s it with that there’s nothing else to talk about. But I am going to talk about a little bit about things that i’ve i’ve seen, I believe I’m somewhere in the middle of the road between product manager who hasn’t done much. And the product manager who’s done a goal. So I’m I think I’m somewhere in the middle. So let’s talk about what what does it mean to be a good product manager? One of the things that I consistently see on in the field, let’s let’s put it this way. One of the things that I’m consistently seeing in the field is that everybody knows what a product manager does. There are executives, there are project managers they are there are delivery managers that are sales executives, salespeople, they all know what the product manager does. Surprisingly, they also think that they can absolutely do a product manager’s job. It’s just that they’re so busy being delivery managers, sales, people, janitors, whatever else. Thing is, it’s clearly then in Kruger effect. For those who are too lazy to dive into the Wikipedia, a Dunning Kruger effect is a hypothetical cognitive bias stating that people with low ability at a task overestimate their ability. In other words, the less you know about specific field or area or specific tasks, the more you think of yourself as being able to do that task. So if you don’t really know what, what’s included in chopping wood, you’d think that hey, I can do it, as you know, down physical work. Guess what, if you don’t know how to do it properly, you probably they’re probably either shut your muscles, you’re damaged, you chop your fingers off or something, something something, it takes practice to good be good at something. So there’s a lot of people out there who can think that, Hey, I know what product manager does. It’s just this really simple. They just need to come in, do this, do this, this and that’s it without they don’t need them anymore. Or, you know, I could do it. And that’s how you see sales executives or account managers or project managers or or some such people coming into the meeting with a client and they’re like, okay, so from the product perspective, how do we do this or from the product perspective? I think we should do this. And and here’s here’s, here’s the trick. When they say, as a from a product perspective, how do we do this? That’s not a bad idea. That’s not about choice of words that we worse is when they say silver from Rado, effective, this is what we’re going to do.

Vlad G 4:40
And the reason why I’m making this distinction is because when they say from a product perspective, here’s the solution is a it’s the person who’s not qualified to be a product manager be they jumping straight to the solution and see they think they know what product management is or how to do a job overall. Manager. And they like fast forwarding the whole thing. The Fast forwarding, like the whole movie, basically, right? And they come come straight to the conclusion. And then you see resource plans, you then you see timelines, you see project plans, like, hey, we’ve figured it all out, we know what we’re doing, we just need two teams. For six months, we’re good, we’re just gonna write, we’re gonna have a product manager, write all the user stories upfront, and we’re gonna execute. Well, you’re in, you’re about to enter, big, scary world of pain. But that’s not my problem. It’s their problem. So what is and what isn’t the good product manager? A good product manager is the one who is a who’s present these conversations, and be a good product manager is the one who doesn’t jump into the conclusions. And he is aware of his own, then Kruger effect, that maybe he’s the main knowledge is limited, maybe his or her industry knowledge is limited, maybe her understanding of this particular approach to building a product is limited. And instead of jumping into solutioning, or instead of jumping into how we’re going to solve this problem, they stick to what is the problem? Can we define the problem? To the best of our abilities today? Can we define the path of building this product moving forward? We didn’t know how we’re going to build it. Okay, wait, let’s let’s all agree on that the product manager should not be saying how we’re going to build it. They can say we’re going to use this methodology, they can say we’re going to use this approach. We’re going to use this. I don’t know ways of working. But they should not tell you how we’re going to build a product that’s for a solution architecture. That’s for engineers. That’s for developers, for technical technology, folks, for legal for sales, there will be a way to build it, we’re not going to talk about it at the at the beginning, we’re going to talk about what is it that we’re building? What is the end result? What are the okrs? What are the KPIs? How are we going to measure success? All That Jazz? Now, in other words, we want to define the ideal state, we don’t want to say, oh, is we going to, you know, use Java and we’re going to use react GS and we’re going to use I don’t know, I bought cheap stock on the back end, and all is gonna be peachy, I don’t know, is a product manager, I don’t know. So a good product manager sticks to defining the what and making making the was so clear that the engineers and solution architects and deck leaves, MBAs are able to make their own decisions about how they should be able to make their own decisions based on the ideal state defined by a product manager. So that’s, that’s kind of where the sexualize we as product managers should not be going into the solution. That’s somebody else’s task, we need to define what requires solutioning? Which is why we’re simply talking about Product Manager defines the what? And then engineering defies the how the business defines why the whole thing exists, right? What’s the business reason for certain things, another? Another trait, okay, so let’s understand that there’s gonna be a ton of stuff that nobody’s like strictly good product manager, or strictly bad product manager that we’re all somewhere in the spectrum. Another thing, another trade that we want to talk about is the obsession with certainty versus embracing the unknown. Let me talk about that a little bit. When you are a product manager, you are intentionally putting yourself in a very uncomfortable position of not knowing basically anything, you don’t know the how you are in a constant state of discovery, you’ve constantly thinking, what if you got slow thinking, I’m gonna, I’m gonna assume that’s gonna be my decision. But I also understand it could be wrong. I

Vlad G 9:26
want to build two different ways I want to build through different approaches. I want to do a B testing, I want to interview my potential users or existing users. And I want to understand what’s going to work best for them. I’m only based on that. I’m going to go to experience designers or graphic designers, or UX or developers, engineers, or solution architects and I got to tell them, hey, our users want to do this or our users expected this to see this or our users will only be able to use our app or our solution if we provide them with X, Y and Z features. It’s its constant state of uncertainty. It’s a constant state of I don’t know. And the best part of it is, the more you though know, the more you’re willing to experiment, the more you’re willing to think it through, the more you’re willing to ask questions from the users from the engineers from the US from the legal from the sales from support. And the more questions you ask, the more information you have to make the right decision, this is how you get this is that you collect data for those data driven decisions, right. However, the rest of the organization does not like that. The rest of the organization wants to be certain of things. They want to know when they will be able to release, especially first release or major release or something like that. They want to know, when are you able to deliver whatever it is you promised to deliver? They want to know how many people you need. They want to know the budget, they want to know the timelines, they want to know the scope of the release, are you okay, you’re going to release on? I don’t know, January 30. But what’s in the release? Are you able to deliver these features? Are you able to deliver these features in full? Or what’s the scope? How much time do you need? How many? How many people do you need, which results in how much money do you need, and on and on and on you go and it results in really funny. It comes out to a an absolutely weirdest things are weirdest constructs you could ever see. One of the things and I talked about this last time. One of the things is the PRD, which is an abomination of the product management world. I keep saying it. Someone just recently asked the question on Quora. So how do you use the PRD? I use this template? And I’m not sure if that’s the right template? How do you 40? What do you use? How do you use purities? What other teams are doing? And my view means some quite as an answer was other teams don’t use purities period, you don’t use purities purities are via elimination for one specific reason, as a compromise a bad compromise that that it’s a compromise between the need for certainty and need to for clarity, from the regular you’re off off the shelf of the shelf. For off the shelf enterprise, and new ways of working, that are well, they’re not so new anymore. But for some enterprise, they are for some of the prizes, they are the product management approach, the product mindset approach, this uncertainty and discovery by delivery. What happens is, you’re right Brd is like the same way you used to write functional specification or business requirements document, or, and in many cases, br DS both the I think the question that I mentioned was, how do you you define functional and non functional requirements in yo PRD? And it’s like, No, you just don’t define New Zealand very PRD. And that’s, that’s, that solves your problem. You just don’t realize that but not having to write PRD solves your problem. And then the person got confused, somebody else has started digging deeper. And they went exactly the same way that I would have one, which is there’s a reason why you write user stories. And there’s a reason why user stories have a specific format. The user stories are the bread and butter of software development. And the reason for that is each user story, it’s it’s it has this like minimal consumable minimum deliverable. It still delivers business value, if you if you if you go in, and then Google

Vlad G 14:04
what the what the criteria for a good user story is, it delivers business value, but it delivers them in tiny little chunks. And it’s kind of that age old question. How do you how do you eat an elephant you eat it in small chunks, you can just swallow an elephant in the hole. And PRD is effectively trying to define that elephant. Not not just just what that looks like, not just the size and you know the weight and the volume of the elephant but also how he walks or how he eats and his his his behavioral patterns. And after you define all of that, and you try to kind of like make that elephant that the real elephant that we tried to find has the diarrhea and that’s it and you’ve done in Europe er D is no more it doesn’t make any sense anymore because now it’s completely different elephant. So what do you do is You break down your product into the structure, whichever structure you use, I don’t I don’t, I’m okay with using any structure. In our product mindset we use product is divided into capabilities that are defined into features. And then their features are defined by epics and user stories, you don’t have to do that you can do epics, features, user stories, you can do themes you can, whatever works for you prop point here is you keep chunking, smaller and smaller and smaller, smaller pieces until you get to the point of the user story, which is the minimal business value delivered in a sprint within a sprint. And the goal here is to have that delivery. That if you deliver in these small chunks, that one of the reasons why initially, people want one of the reasons why people initially loved the idea of a sprint, and aside from many other things, was that if you need to change the direction, if something happens in the market, the product, you discover something, you only lost this small, tiny chunk of work, you haven’t lost the whole project, which is the predefined project by the predefined documentation, which is what the purity ultimately is, the only lose this much of work. You’re saving yourself from you know, drama, a huge drama, huge losses, you’re minimizing the losses, and you continuously deliver and develop and deliver product, you continuously giving something to the business, every sprint business gets some business value. So if you think about it this way, you’ll need to define everything upfront you what you need to define is is the product strategy, the product vision that the product roadmap, product roadmap is not a sequence of user stories, or epics and features, its capabilities and maybe breakdown of capabilities into something smaller, and then you can you may or may not call it a feature. The trick here, as you may think about does a product, then there is a capability underneath the product. So let’s say I don’t know, a phone, a smartphone, one of the capabilities is actually making phone calls right now that could be abilities making video, which is what I’m doing right now. So then you have the video. And then you can say, Oh, I want to record video in different formats. Is that a feature? I don’t know. I don’t know. But within the capability, I have this idea, idea space, whatever theme that I want to record videos in different format. Maybe it’s a gr maybe a 16 by nine, maybe it’s four by three, maybe it’s 30 FPS versus 60 FPS, whatever the hell that is. But when I’m thinking in terms of the roadmap in terms of product strategy, that’s the level of staying at. And that’s what I need to have. Once I get to that point, I can start working with individual themes, I can start working with individual delivery teams solution architecture, I can talk to Solution Architect saying, Hey, this is my roadmap, I intend to have this different formats of recording the video, the friend form factors 16 by nine four by 360, FPS 30 fps? Do you think it can be done within the single feature? Do you think it’s still different things? From technology perspective, from a user from experience perspective? Is that same thing? Or is that a different thing? How do we approach that. And then, based on conversations where you wax based in the conversation with technology, as a product manager, I make a decision

Vlad G 18:58
to say Oh, is going to there’s going to be a feature, selecting frames per second. And then there’s going to be another feature, selecting screen ratio 16 by nine or by three, whenever. then based on that, I can structure my more detailed roadmap into the actual capabilities features, and then break it down by epics and user stories. That’s kind of how the process should work. However, that pushed for certainty. And you’ll see this in, in a lot of enterprises. They they buy they I mean, PMO, upper management, whatever it will tell you. So here’s what we want to do. This is what the product should be doing in the future. We want to see all the user stories written up front. They don’t have to have all the details but we want to see all the user stories estimated. So we would know what the budget should be. Write me all the user stories and estimate them in story points. And I’ll ball we’ll see how long it will take for you guys to build it. And that’s what it’s going to. That’s what our project plan is going to me. And if that’s not a two barrel shot into both feet, I don’t know what is because you should be writing user stories all the way at the end after discussion where about product roadmap product, vision, product strategy, coming from the business coming from the product management office or innovation office or whatnot. Then you have product managers defining the roadmap or a feature of feature roadmaps or product roadmaps. Then they start working with the teams on the second that into smaller into smaller chunks down to the point where they get to the point of either again, their features, or epics, and then individual teams with their feature teams or which however your teams are position, they take over and they start writing epics and user stories while in my world is capability feature epic user story. So capabilities and features are developed by product managers with social Arctic’s dev dev leads or teams or whatever. And then from the epics down to the user stories and down to tasks and execution. It’s more teams that would hate agile is enabling teams to be self sufficient. That’s where you hand it over, not handed over, it’s probably not a good choice awards, they had over means I gave it to you, you own it, they own it. But I’m still here, I’m still participating in discussions of some participating, participating grooming sessions. But the team is responsible for delivering it, I don’t get a say in how they’re going to do it. All I’m saying is, I need this one, when you’re done. This is what needs to be happening. This is how the product should behave. How you guys get there. As long as all teams are on the same page, we’re using same programming language, same frameworks, same solution architecture, we’re good. We’re good. I don’t need to tell you. And that’s why writing purity is probably a sign of a bad product manager. And then not resisting having impurities is, well, maybe not bad product manager, but more of a inexperienced product manager or not. The Product Manager is not mature enough, unfortunately. But yeah, don’t don’t resist, resist writing parodies, as it’s bad, is best for business. But I think I already I already talked about that. Now, we’ve talked about what’s good and not good product manager. We’ve mentioned the Dunning Kruger effect, we talked about what versus how, and we talked about obsession with certainty versus embracing the Unknown. Unknown is a friend or a product manager unknown is good, because it allows you to experiment and get the let me let me let me see if I can say this, right. It allows the unknown allows you to choose the right solution, versus choosing the best solution. The trick here is, you know, you can you can spend all your budget,

Vlad G 23:41
because spend all your time and money and waste all the resources on building the best solution out there. By the time you get to be done, your products going to be probably going to be obsolete. But picking the right solution, it may not be the best, but it’s the right solution. For the moment, it solves the problem. It allows the user to do what they want to do. Ultimately, it keeps the user using your product. And that’s what matters. That’s why you want to pick the right solution, not the best solution. And the unknown leads you to the right to begin the right solution, not the best solution. So there’s that. Alright, next big topic. We wanted to talk about was interviewing and hiring Product Manager. And this is this is the part that I promised to leverage right it and I kept reading the Reddit in the last few weeks, especially when I was sick what else but also what I do right. You should sit there and read and there’s a lot of there’s a lot of pain going around. Interviewing in Malta Pull rounds. I agree, I tend to agree with the fact that too many rounds is bad. I don’t agree with the video, just one or two rounds is good enough? No, my personal opinion. And it’s just my personal opinion, and I don’t think we’re doing this, what the company that I’m currently working for, I need to check. But what, what I think is, you need to have about four interviews total. One should be HR, they should do a good job pre screening the candidates. And they should do a good job, figuring out the fit the company fit the culture fit, whatever else is their job. If they can’t do that job, fire your job. Or in more realistic terms, don’t let them do the whole interview. Let them do 15 minutes per screening, give them a couple of questions, pretend they’re doing it, and then do the actual interview yourself or with the peers or with other departments. But you do need, you do need the cultural fit, interview, but somebody needs to do it from the the people who are doing it for the person who’s doing it. He or she needs to know what what that means what that cultural fit is, if if one of the departments doesn’t do that job, then somebody else has to do it. But it has to be done. Now, another two interviews should be the the meat knowledge, the knowledge, the main authority. So if you’re interviewing for a product manager position, it should either be two hours with two different product managers or two different interviews with different product managers. A reason for that is each product manager has their own has their own quirks. I have my quirks. I know, a bunch of other people that I’ve been on the interview with who are interviewing the person have their own quirks. Sometimes they’re compatible, sometimes they’re not. It’s not fair to the candidate, that we did not resolve our quirks internally. And the reason why we didn’t have that result down was because with a no, because we didn’t have time, because I don’t want to give up my quirks, because they allow me to make good informed decision about the candidate. So what this means is, I don’t need to be on the interview with that other person. It’s not fair to the candidate. And it’s fair for the candidate to see at least two different people to have at least two different opinions. The more the merrier. But again, I don’t think there should be more than four interviews total. So one generic cultural fit to the main role specific, and one with upper management, or upper management, culture fit or expectations or somebody with a lot more experience a lot higher position, see how that person would fit the team. And I think that’s it, that’s that’s the most,

Vlad G 28:30
you need to go through four is the top again, you may be able to combine some of these, you may be able to do away with three. But at least from my opinion, these are necessary steps. If you can’t make a decision after these four interviews, you’re just wasting everybody’s time you’re not hiring, you just got to know proving to somebody in HR or in the management team, that you’re doing everything you can to find the right fit the exact person that specifically knows the ins and outs, well, you’re not going to find them. By the time you get to find them. It’s going to be two years and your competition is going to be way ahead of you. So four steps, one cultural fit, one, upper management, sort of kind of thing, make them feel better. And then the two different people from your peers, specifically from product management. here’s here’s an example. So I think I brought this up earlier. But just in case I’m going to repeat that again real quickly. The way I interview people, I tried to understand their experience and then move on to case analysis. I know other people hate case analysis and they try to avoid it. Okay. So the drill into the experience. I, personally, I’m on the fence with that. I don’t think just drilling into the experience is fair to the interviewer. The reason for that is I may not know the specifics of that particular role. So I mean, not ask. I mean, that’d be asking questions that help the candidate open up and tell. Tell me more about them and more about their experience. And on top of that, they may well, we will think just just to be clear, rolfing candidates are honest people who are trying to get into the role because they understand what it is they want to work with us. And they are good, honest, people who are looking to fill a specific need the company has. Without said, I’ve seen my share of people who pretend to have done something they have not. Again, I go into each interview with good faith in the candidate. And I literally again, they think I’ve, I’ve made that example before, I’ve literally seen a project manager resume where the word project management was replaced in the word Ctrl, r Ctrl, F Ctrl. R into the words of product management. So all the responsibilities are proof of project management, all the words on time and by Ponyo writing things on time on bhajan the scope, but he’s a product manager. And my grievance here is that I don’t want to resort to idiotic questions like how many, you know how many x fifth in the in the y when Z’s is alpha. I don’t want to talk about you know how many baseball’s fit into the bus? I don’t want to talk about how it’s annoying. It’s annoying. It doesn’t tell me anything about the candidate. It tells me that either candidate study these questions, and they can pretend really well that they are answering these questions on the spot. Or they didn’t. And they also just like me, they think it’s idiotic, these questions are useless, and they didn’t prepare for them. It doesn’t tell me anything else about doesn’t tell me anything at all about their abilities as Product Manager. All it tells me is either they’ve studied this idiocy or they haven’t. That’s it. That’s all. That’s all it tells me. Why would I waste time on that? So instead, I give because there’s use cases, hey, I want to build a product that does this, how you approach it? What do you think you will need? What do you think your MVP would look like? How do you approach building an MVP? How would you use your previous experience? And I think

Vlad G 33:16
that’s a legitimate question. I’m not asking them to write up. Actually, that’s something that happened to me, I had a way way back. I had an interview with a company with a startup they just received their round a or OMB something like that. They create the open the office, huge office, beautiful view on Hudson River, I think was awesome River. So it’s amazing, like millions of dollars in rent one of the prime downtown locations in the financial district in Manhattan. Absolutely gorgeous office, absolutely gorgeous view a ton of money. These guys are asking me Well, here’s here’s our product. This is what we think why don’t you take this home? If you have any questions, ask us. Today’s I don’t remember the first probably Thursday. So let’s buy Monday. Let’s have let’s have a PowerPoint that you’re going to come in about to the office and you’re going to present it and we want to see you present a roadmap of this product. How do you see it? And it got me thinking, Hey, guys, you have a ton of money. It’s not it doesn’t look like you’re, you know, starting in mom’s garage. You’re asking candidates to create a roadmap of your product for real, which means you want them to ask you questions that you want them to call you discuss things effectively you want them to work for week on strategy on road mapping of your product, present the results for free? The answer is no. And there’s there’s a ton of ton of red flags right there. If you want me to do consulting gig per week, that’s fine. Let’s write up a contract. I’ll spend the week I don’t have a problem with that. I’ll take time off of my other work or whatever. And I’ll consult with you. And he, you take these results, and then you can either hire me to continue as you would with a consulting company, guess what? have come. Other companies do this with the lawyer. Other companies do this with McKesson, other companies do this with whoever else be the VCG. It’s normal thing to do you hire someone for a week, two week three week consulting, engagement, the write up the strategy, they ask all the right questions, they do everything you just asked me to do. They produce a tangible result, which is actually a PowerPoint deck. You pay them for their time, or you pay them for the result. And then you if you want to use that result, you hire them. And they help you with the execution. If you don’t want to use their their result, you hire somebody else. That’s the way it works in a normal, real world. How much do you think a week of my time would cost? Whatever it is, I’m sure you guys have the money, given the office that they give you this cheers square footage, and the view, you can definitely afford someone like me for a week. So I’m going to break a budget. Why do you Why Why are you Why are you doing this by trying to get this freebie from from people? I’m sure I’m not that wasn’t the one the one. I’m sure there’s like a ton of other people who were doing same or not doing doing or not doing this for free. So bad way to interview people, by the way to interview product managers.

Vlad G 37:11
for interviews, give them an abstract case, my personal opinion, give them a abstract and abstract case, I say this five, probably at least five times every interview. This is a hypothetical case, you I don’t expect you to build anything really real I want I expect you to build realistic, which means I get it if I’m asking you to build, I don’t know a chat application or messenger or something like that. I don’t expect you to actually come up with I don’t know, user interface or something. But at the very least, I expect you to tell me what your view would look like, Well, you know, it’s 2020 or 2021, or 2025, whichever the case dictates. And a lot of people are moving away from typing. A lot of people are doing voice messages. But not everybody can listen to voice messages. So I would my MVP would have typing, voice messaging and transcription service. There’s a ton of capabilities on ashore on AWS on Google Cloud, where you can transcribe the voice. That’s it. That’s my MVP. perfect answer. Even if you if you didn’t include the transcription, I feel good with that. As long as you justify, as long as you tell me this is how I would approach the MVP. This is what I know, this is what the questions I would ask. This is how I get there. And this is what the impact should be. This is what how I’m going to solve the problem that you gave me in the case. And again, simplifying them, not not giving out the real case here. Just tried to get the point across that. Why am I giving cases? Even though I do know, not everybody’s on board with cases during the interview because they take a lot of time. And yes, sometimes people get drowned in the case. Yes, sometimes people get scared. I had a kind of candidate who got scared by the case. And they failed the interview because of that. My rationale is if you’re going into a product management role, and in the consulting company in technically speaking, we are a consulting company. If you’re going to into the product manager position at the consulting company, and you can’t deal with the case on the interview. You’re a bad fit. Sorry. That’s that’s that’s the way I think and feel free to disagree. The happy to hear your thoughts view through comments. I don’t know, Reddit comments you can find me whatever. Tell me why you disagree. Because for me, k use cases specifically business cases or or generically calling them cases on the interview is your freakin bread and butter. He kids get away from that. And actually, as a matter of fact, if I remember correctly, there’s, there was a ton of ton of times when I’d be with the client and client would approach me in the informal fashion in an informal fashion, either over teams or over slack or, you know, in the room, hey,

Unknown Speaker 40:46
I think

Vlad G 40:48
this, what do you think about, you know, this, this and this, what do you think about, you know, what the product would be here? Believe me, it’s 100%. The same case is just wrapped in a slightly less details. And it’s a little more convoluted, and it assumes you know, everything about this particular client and their environment. But it’s the same case I’m giving you on the interview, or it’s the same type of a case, I’m giving you the assessment. And that’s, that’s exactly the reason why I keep pushing for it. I know there are people who disagree with me, I, I’m okay with that. But I’d love to know your reasons, please let me know. I absolutely hate just going through the resume and leave it at that. Which is another reason why I’m saying this should be two different product managers with two different approaches to the interview. A then there is the candidate, believe it or not, people who interview for product management positions are really smart people. I absolutely love talking to them, even to those who fail. I still love talking to them. They’re smart, they don’t fail, because they’re stupid, they fail because whatever, they fail, they weren’t ready. They couldn’t solve the problem. They forgot something. They took specific approach and decided that, you know, sticking to that approach was more important than being flexible, whatever it is. They fail, because whatever. There was somebody better. I don’t know. It doesn’t mean they’re stupid, doesn’t mean they’re bad candidates. It just there was there is reasons why they didn’t pass the battery vote the interviews. But they’re absolutely smart people. And I absolutely love talking to them. love hearing those stories, those things. And one of the things smart people do, guess what? They’re smart people, right? One of the things that smart people do, they prepare the stories. And I think I said that at least couple of times on this podcast, sparks evil prepare their stories about each and every job they held each and every product they built each and every engagement they were in. So whenever you ask them, they come up with this story. I know because I do the same thing. I know because I’ve seen people do the same things. I’ve seen my bosses do that. I’ve seen my colleagues do that. My peers, I’ve seen people on the interviews on the assessments. That’s the whole point of being prepared for the interview, you have to have a story behind anything you say on the interview has to have a story behind it. I then recently I did a pre assessment, which is kind of like a mock interview. person comes in you pretend you have this like pretend interview. It’s not really an interview. Because you don’t you don’t really ask tough questions, you ask questions to assess if the person is ready for the actual interview. Because again, you can do a mock interview and then let the person go into the interview. And that’s called the same questions. Right? That’s that would be cheating. So you kind of try to assess how ready are they? And when we were talking and I was asking questions about around the resume around the jobs and around the engagements. The person sounded very off the cuff, the person sounded like, Oh, yeah, here’s what happened or in they were trying to come up with a store in a spot. And I realized that after conducting hundreds of interviews and houses of assessments, I can actually tell I can actually tell when the person prepared is prepared. And they have a good story or person is not prepared and they don’t have a good story and they try to come up with a good story on the spot. Find wise, it takes them longer to get to the point. And story wise, it takes them longer to get to the point. And story’s not that interesting. Because again, your story has to be engaging. If you tried to tell a story, during the interview, specifically you’re in the interview has to be engaging, it has to be engaging story. And it felt like, you know, I wouldn’t I wouldn’t go into the interview, I wouldn’t go into the assessment, I wouldn’t go with this type of level of preparation. Just because you did something. Doesn’t mean you know how to tell other people, what you did and how you that isn’t, and what was it, I think is one of the biggest ones. And that’s I think that’s why a lot of companies are thinking cases are not fair to candidates and the subsidy cases with eight rounds of interviews with a bunch of other people. Don’t Don’t do that. That’s, I mean,

Vlad G 45:48
I, if you’re hiring an executive, I can’t imagine eight rounds of interviews, you probably want to give them time to talk to all the other executives or you know, all the big big shots at your company. But a product manager is not that is not that type of a role where you need a interviews was everybody including the janitor? Why? What are you trying to accomplish? What are you trying to extract? If it’s free work, if you’re, if you’re trying to ask questions about your, your own company’s roadmap and your own company’s stuff, and have them do free work for you. While while we’re like I said, pretty much if you’re if you’re if you’re going to the product management, especially if you have experience if you’re not fresh graduates that are trying to become, you know, a product managers, whatever, they can pick a spot, and they can call you out. So don’t do that. I mean, I know there are some desperate people who would go for it. And they get taken advantage of I, I think you have to have some kind of BS meter. And don’t fall for that. But I totally, I’ve been through some desperate times. So I completely understand why people would fall for that, or why people would still go for it. I don’t blame anybody in this. In this case, I don’t blame candidates, I would blame the company. But who does that but like I said, I understand they’re desperate people, there are some companies. This is kind of a good, good place to wrap up, why companies are doing that they come they think the answer. And I, I want to give the credit to the credit is due. I read this as one of the comments and read it. And I felt like it’s it’s too buried down there. And I wanted to bring it up. companies don’t need visionaries, they need cogs in the machine. And product managers are no different. It’s really rare when the company is looking for a visionary and I don’t mean vision or like Steve Jobs, right? We’re talking about person who can have a vision about the product. Because that’s what the executives are doing, right? That’s what sales execs together with a CEO CEO are, that’s that’s what they do, they really don’t buy, okay. Problem with us is everything is is getting commoditized and that’s probably a conversation for another time. But product management is no different is getting commoditized that’s why you have all these MBAs or people who, oh, I just graduated from school or from college, I want to go get an MBA, and then I’m going to go into the product management, they will and they won’t, because companies are indiscriminately hiring everybody who is you know, who gets to be who gets to have MBA and want to be a product manager and they did an internship, build this, you know, software, whatever. And maybe that’s good, maybe it’s, it’s good that we’re gonna have we’ll have same level of this same levels of seniority, you have fresh graduates going and writing code and then you have solution architects who build really truly complex systems, you rarely see fresh college graduate getting into solution architecture, but getting into coding is not that big of a deal for a lot of college. So maybe the product management will have similar

Vlad G 49:51
will have similar growth and similar distinction. That would be beginning or junior product management where Have you end up doing basic things, small things, small product, small features, writing user stories, getting acquainted with what the product management really is. And then you’ll have system architects or business architects, or people who understand the full scope of responsibilities and do all those things that are on on the diagram from legal support to go to market to sales through support to everything else. So maybe that’s the way it is. But as of right now, Product Management is no longer a vision or a job, it’s a cookie cutter approach to, hey, we need someone who’s going to be doing the roadmap, and who’s going to be doing this who’s going to be doing maybe user stories, maybe, you know, breaking down features into epics, or epics and features depending on your framework of choice and your voisey new things. And that’s, that’s why you get these humongous interviews, because if you don’t need a visionary, which is really easy to spot, that that type of a person is really easy to spot, if you need just need a cog in machine, you’re really trying to get that cog to fit the existing structure, you’re really looking for somebody who’s going to solve your problems, you’re looking for someone who is going to fit into the structure. And that’s the difference. That’s why you’re going to have all these growing pains. And there’s a lot of companies that are going digital or after during COVID or after COVID they change their ways of working the moving fully digital, they need that product mindset they need that the front approach, they’ve never done it before they were always putting on the back burner. Now they’re trying to get into it. And they just need someone who fit the structure of fit they fit the bill fit the bill, the space with the card needs to be so that machine keeps keeps going forward. And maybe that’s the best that’s a different topic for this topic for another day. That’s a different discussion altogether. But that’s why you get that interviewing my massive massive hiring massive interviews of product managers but your interview them like you would any other lower level, lower lower lower level worker, again, not because I want to talk somebody down or disrespects how long just because they understand worth talking about cogs in the machine, just another brick in the wall sort of thing. So that that was it. We talked about what’s a good product manager wasn’t versus was a bad product manager talked about the everybody’s in the experts and Dunning Kruger effect talked about what versus how obsession versus with certainty versus embracing the unknown. Then we talked about the interviews, multiple raw rounds of interviews, dive a little dive a little deeper into cases why I prefer cases versus not being ready for the interview, doing free work for companies as a part of the assessment or as a part of the initial interview. And we ended up on the prowl, little bit sad note. But it’s a you know, real world product management. So the real deal is, companies don’t need visionaries. They need cogs in the machine. With that. Thank you very much. Appreciate it. Give let me know what you think comments, emails, anything works. You’ve been listening to the real world product management and I’ll be your host Vlad Grubman. Until next time.

Transcribed by https://otter.ai

Real World Product Management – Episode 16

June 10, 2021 Vlad Grubman

Picking up where I left off a year ago. New episode with just me talking about

what happened to the podcast
how to become a product manager
what to expect in product management interviews
tools vs process
PRDs are bad for business

You can now WATCH ME SAY THINGS! In the glorious HD. With sarcastic subtitles!

You can subscribe to podcast on your favorite platforms

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:08
This is real world product management.

Vlad G 0:17
Hello everyone, my name is Vlad, you probably watching this or listening to this, recording both the video and the audio separately. So it could be a podcast and a YouTube video. I am a host and the author of the real world product management podcast. I started it in March 2020. Right in the middle of a pandemic, sort of, technically, it’s a sort of cope with pandemic and the whole thing. And I the podcast was pretty had a pretty good run, we did 15 episodes, I almost always except for the first and last actually no scratch that acceptable one first episode, where I was doing things solo, I believe every single episode after that, had someone one or two people, as guests, this is me tried to restart the podcast, the trying to sort of restore and continue the work I was doing and other people were participating in almost exactly a year after I started after I stopped doing it. Last episode went out June 17. I’m recording this on June 7, exactly here after so we’ll see how that goes. As, like I said, I’m not just restarting it as if nothing happened. I’ve learned a lot I’ve seen a lot of things, know that a lot of feedback sub doing a lot of things differently, going to try a lot of new things, previous 15 episodes, let’s let’s go the season one I did mostly in the same Canvas, same approach, same, you know, hey, this is the guest, they had something to tell us. We’ll listen to the story that the guests is telling us and then we ask questions, kind of sort of like an interview, but not exactly. I’m going to try to do in different things from now, one, maybe make it shorter. I remember, some of the episodes ran for almost an hour and a half, I’m looking to record more episodes, but they potentially going to be shorter. So we’re looking at 20 3045 minute episodes. But maybe, you know, once a week, twice a week, sometimes we’ll see how it goes. No promises, but I’m gonna try to do different things. I’m going to try doing live videos, try leveraging community a lot more through the last year, the company I was I’m still working for by the way. We’ve built an amazing community of people have amazing community of product managers, a lot of them have tremendous amount of experience huge lists of things they were able to achieve and they were able to accomplish. So hopefully, I’ll be able to drag some of them kicking and screaming onto this podcast. I know some of them were interested, as Firstly, I was not available to continue the podcast. But hopefully now we’ll, we’ll be able to work on that. But additionally, the community on Reddit, which is what prompted me to start the podcast in the beginning, the community on Reddit began to be a lot more active. I’ve seen a lot of interesting people. I’ve seen a lot of interesting thoughts. One of the one of the cornerstones of this podcast was the guests should not agree with the host or host is not always agrees with the guests. And that’s what makes the conversation interesting. That’s what I’m seeing on in the community. So I’m hoping to drag in more people from Reddit product management subreddit, see if we can give them a platform to express their opinions. I’m actually if I’ll have time today, I’ll try to talk about one of those topics that were discussed that piqued my interest. Again, like I said, No promises. We’ll we’ll see how this how this is gonna go. And we’re still you know, in and we’re still in COVID pipe pandemic, I’m not sure if it’s called been called off or not, but people still cautious on moving away from New York to a whole different state. So being there physically may not be an option. So we’ll try to leverage online collaboration tools that

Vlad G 4:17
They’ve gotten… we don’t really remember but when I started working on the podcast again, I figured that Hey, remember how last year we were struggling with, you know, doing the recording or bringing several people in and getting with quality recording so we can post it as a podcast. I remember going through the WebEx going through teams Zen caster, zoom, we’ve tried a lot of things. And it’s surprisingly it’s gotten so easy now, just the year after a lot of online calibration became almost like a non issue. It used to be a lot harder for us. I remember that’s the backstory, why what happened and why we’re restarting the podcast, podcast, as you remember, cold real world product management, the goal still the same, bring in a real world perspective on things that people who are close to product management and maybe just starting in product management. They think that it’s some kind of glorified position where you just tell other people what to do when they just hop, hop to it and do it. We try to bring in the real world perspective on things. So having said that, one of the things that I always wanted to talk about is, and I know it sounds corny, but it is one of the things that I wanted to talk about, and I felt pretty strong about it, are the top asked questions. And we’ve done a number of online conferences, the company I work for. And there’s a there’s always a number of questions that come in. And they’re, they’re they basically divided into two parts. One is tactical, so specifically about the topic has been discussed or a question about something somebody said during the presentation. And then there is a whole set of strategic, kind of like strategic questions, and they all start with one single question. How do I become a product manager if I am a blah, blah, blah, and that blah, blah, blah, can be literally anybody. If I’m a software developer, if I’m a business consultant, if I’m a BA, if I’m a UX designer, if I’m experienced designer, if I’m a janitor, how do I become a product manager? I had a slightly different opinion year last year, and we talked about it on some of the some of the episodes, that opinion has changed, guess what people do change their opinion. I know, that’s not really, you know, cool thing to do. Everybody now has their opinion, and they’re sticking to it. I don’t, that’s kind of one of the things I bring to the table is I will make mistakes, I can make mistakes, I do make mistakes, and I try to fix them. And I try to if if my opinion was a mistake, or if I had to adjust my opinion based on facts that absolutely will do that. One of the things that we did discuss back in the early episodes was how do you become a product manager, most of our product managers back then came from Ba, this is no longer the case. We are hiring off market we are hiring from a lot of other sources. We do hire from experience design, UX design, business consultants, one trend that we’ve noticed, not in us in some other geographies is that people are coming from being in within the general management, sort of line of business manager, oh, bank, you know, CFO, Director of Operations, things of that nature. Apparently, one thing is it pays better I again, it’s not us. So situation is slightly different in other countries. And another thing is that experience that they had in those roles helps them become product managers around it for software products. And it helps them get things moving, get things through, because they have the business acumen or they have that understanding of how things work in the business or the in their in the enterprise. And it helps them get better at this job faster. If you compare someone who comes from ba background, they know very well how to write requirements how to how software works and how stuff has been built. But they have very little if any understanding of how business is conducted, what is

Vlad G 8:50
you know, me doing business? How does that whole thing work. And that’s that’s kind of the part where they struggle the most these people who are coming from other non it backgrounds, but they have a lot of business acumen, they’ll have a lot of business knowledge, they tend to be successful because they understand that overall big picture. And they get very well positioned in terms of Hey, I know how the business works, just tell me what I need to do to get from the you know, one step to another to another in order to make things happen at the business level. So people do get there successful if they come from non product or non it background. They come with a business background. So how do you become a product manager? You learn you, I recommend reading and taking classes I’ve conducted about hundreds of about 100 interviews and assessments in the last year. So that year that I was not producing episodes, I was very heavily interviewing people. I was very heavily participating in a product management assessments. So the difference between interview and assessment is when the company hires someone from outside from the market, it’s an interview. If we want to promote someone from within, and our company does that a lot, we run an assessment. So it’s kind of like an interview, but it’s internal interview is a different format, it’s a lot harder, believe it or not, it’s a lot easier to be interviewed and get an offer, if you’re on the market than it is to be to pass assessment and get promoted internally. So I’ve heard of cases when people leave, would leave the company for a couple of years, get a bit more experience, and then come in, sort of from the market and set up from within and get into the position, they were unsuccessful in getting internally dirty little secrets. So a little bit about the interviewing and assessments, I’m going to kind of bundle them together because ultimately, the goal of either interview or a set or an assessment is so understand your skill sets and their send your How does your mind work? Right? How do you think, and I use suitable for this position. And so that’s what hire for a specific position, sometimes we’ll hire, because when they’ll will, will get positions in the specific

Vlad G 11:23
areas specific knowledge, the main, very rarely, we stumble upon. Someone who’s a generalist, somebody who can be put into almost any role aside from very specialized give you an example. I’m not like a super perfect generalist. But with with my experience most my set of skills, I can put in general, finance, healthcare, retail Telecom, very general software, enterprise software, is a set of domains that I can fit because I’m a generalist, but if you want to me, if you want me to work on a very specific domain, for example, clinical trials within Life Sciences, I’m not sure guy because I don’t have sufficient depth of knowledge in that area. Or, for example, if you want me to work on something that’s very consumer oriented, that requires a lot of graphic design background or industrial design background, even though I may manage, but I’m not going to be a good fit, because I lack those specifically those design, industrial design or graphic design skills, that would be preferable than a candidate for the role like that. So like I said, it’s really rare that you get generalists off the market or internally, but they do exist. And we try to maintain kind of a healthy balance. So and the trick is interviewing a specialist, if there is a specialist is easier than interviewing a generalist, because with a specialist, you know, specific, their specific domain knowledge, couple of domains. And you can kind of either drill deeper into that area, or you can kick them out of that area really easy and kind of take them out of their comfort zone and make them think in terms of pure product management, pure product pathologies, without relying on existing experience with a generalist is harder because they’ve seen so many things. They’ve seen so many different things based on different domains, it’s hard to get them out of their comfort zone. I’ll talk a little bit more about that later. But I would generally break down the interviewing assessment, again, doesn’t really matter into two pieces. I probably want to talk more about the interviews, because generally public don’t get assessment general public gets interviews, when they tried to get into new work. And I have been on both sides of the table. I have been the interviewee and I an interviewer a lot. I have the I have gone to interview for for other roles. Because I want to understand what else is happening on the market and how people interview for this role. Obviously, early in the days during the pandemic, there were all kinds of things were going on. I haven’t interviewed for about a year, but I don’t think a lot of things change in terms of the interview mythologies, the acceptance rates have fallen, but the methodologies are most likely the same. I don’t think they’ve changed in the year. So there are two types of interviews a product manager will go through when they are applying for product manager role. And I want to make sure we’re talking about product managers, not product owners. There’s an interesting split in the product owner roles. This product owner as an agile role, which is a team member of the agile team. It could be senior lead ba developer or somebody who understands whole concept of thing or capability of feature whatever he was working on. And they represent the team in as they are a teams representative, they work with other teams and with the product management office, then there’s another meaning to the term product owner. And that is a very senior person kind of like a business owner and the sense, but they said, because they own the product, everybody tends to call them product owner. And they are senior, usually senior executive at the program or even higher level where they get tend to make business decisions about the product. And they could be senior executive director level, they also get to be called the product owner. We used to ask this question we still do sometimes. But we try and tried to phase it out. Because especially in the interviews in the assessments is different. But in the interviews, it’s really easy to fall into this trap of what is this specific companies calling a product owner or the calling product owner, a person who assists with the developers and gets the rights, their user stories? Or are they calling a product owner, a executive director of operations, who makes calls about a Hey, this product is going to do this in three years, boom, go. So that’s the trap. Like I was saying, there are two levels of the interview. One is the peer interviews where you get interviewed by Delivery Manager, project manager, Program Manager, and obviously, your fellow product managers who are tend to be on your level, maybe you know, a little higher, a little lower, but generally they’re your peers. And then there’s an executive interview, that’s where your senior product owner or, or your executive director of operations come in, and either review, and sometimes these occurred the same on the same day at same time, sometimes they’re different. Hope for your sake, they’re different. Because it would suck to have both on the same time. And what what are the questions, right, what are we expecting to see from fellow product managers? And what are we expecting to see from executives? What type of answers, but sometimes they ask the same question. I love it when it happens to me, because it allows me to leverage this understanding of the difference. But obviously, you don’t have to love it. You can hate it if you if that’s what you’re if that’s your thing. But sometimes we do ask questions like so how do you resolve the conflict? in whichever form that question is asked? How do you how do you approach building a roadmap? Again, there’s a number of ways to ask this question. What type of questions are coming from fellow product managers? And what kind of answers we expect to hear and ultimately, what we’re looking for in in the interviews? Well, the questions that are coming from product managers, I usually try to understand how much the you know, in terms of product mindset, in terms of product management technologies, in terms of building creating, and using product management artifacts. A lot of times we’re seeing a resume that looks like the person is next Bill Gates or net next Steve Jobs, the resume just screams

Vlad G 18:22
I’m a genius. And these usually make me very uneasy, because I know something’s up Believe it or not a perfect resume is not a recipe for success. It’s more of a recipe for a little bit more scrutiny, I guess. So what happens is I I probably should not talk about what other people are doing I should probably only talk about what things that I do not you know, my secrets are my secrets, their secrets, their secrets. So we when we interviewing usually as a one person was leading the interview and then others kind of supporting them. So if I’m the I’m the interview lead I do what I think is right other supporting me. If I’m not the lead, I look for clues from the either lead and support them and their questions. That’s usually how it works. So if I’m really if I’m asking the questions, I want the candidate to elaborate on some of the things that they’ve listed in the resume. Best thing here. Best thing I can ask is, what was your role or or more specifically you did on this, you know, working on this product. there sometimes people would say I did this either this either this either this, and there’s too many eyes. Although the three E’s the time to brag is when you absolutely must leverage your knowledge, your expertise, you should brag neither of us you tend to brag However, if you always say I I suspect that you have done absolutely nothing. by yourself. I Everything you do whatever you say I, you just wanted to say we and then, you know, consciously or subconsciously you just think, Oh, no, I can’t say we have to say AI and I’m gonna say AI. And you’re saying AI all the time. Ideally, you should say, hey, my team did this. And my part in it was this good example would probably be like he was working on this product. I sat down with a C CIO or CEO within the ideation, we came up with this idea. I did research, I took my developers, we came up with a PLC, the proof of concept developers build technical prototype, or UX designers, or UX folks, built this, you know, visual prototype, we looked at it, well liked it. I took it upon myself to create kind of like a high level roadmap, we dive, dive into it with the teams, we broke it down into blah, blah, blah, Bs or appeals or user stories and the rest of history. That’s the right thing to tell your story. But if you’re going to say I, that’s something I, I did hear it. So it’s not, it’s not unusual to hear I come up came up with this idea. The CEO told me to go and look into this. And I came up with this idea. I wrote requirements, I wrote all the user stories. I created wireframes. I did that I Nope, nope. No, you did? No, you didn’t? Nope. No, I’m sorry, I don’t believe just I just don’t. So another another thing that I’ve noticed a lot of companies started doing I do it myself. I know others are, is

Vlad G 21:36
giving you an example, a sample product? I’m sorry, that’s a simple product doesn’t here’s your sample product. If you were to build a Google Glass from the scratch, right, what would you do? How would you build it? How would you go differently? Or if you were to build the system that should track online usage of something something? How would you approach it? How would you do it? So those types of questions? Technically, they’re scenarios, right? You in good consciousness? And if somebody somebody who’s interviewing you expects you to answer these questions correctly, 100% of the time, they’re out of their mind, you should not be working for the company like that. Just my personal opinion. But you should not be hitting 100%, all of that you should be getting close. But don’t expect to be I saw I had a case, one fellow Product Manager interview, who was interviewing me was basically trying to find out if I can configure Google Ads platform for a specific client. I don’t know why. I don’t know, what was the purpose of it? I don’t know, how much did he learn about me, but it didn’t go well. Ultimately, a good interview should allow you as a product manager to demonstrate that you can operate within the product mindset, you should be able to do discovery, you should be able to do you know get from ideation through discovery, through you know, product vision, product strategy, product roadmap, define the MVP, define some kind of sort of preliminary plan, get to the point where you, were you able to demonstrate you don’t you’re not just Yeah, I know what the product lifecycle is. But you can navigate the product lifecycle, you know exactly what happens at each step of the way. Sometimes you may miss things, because you’re under pressure. Everybody understands that interviews are pressurized environments, but nobody in their right mind should be expecting you to hit the nail on the head 100% with them, ultimately, a good product manager should be able to leverage their existing experience. Where if you’re brand new product manager, their knowledge? And if you don’t know the answer to the question, that’s normal. I’d be very, very cautious about the person and we knew all the answers all the questions I’m asking, here’s a little dirty little secret. I don’t know the answers to those questions. And if you know them, then you should be interviewing me or you should own the company I work for if you know all the answers to all those questions, because the trick to it is not knowing the answers, but showing how you get to the answer. To me, it’s pretty obvious if I’m asking you, so you, you want to build a Google Glass, right? So I’m gonna use that example because I already mentioned it. So you want to build a Google Glass. So you know, who’s your Who’s your target audience? Who are you going to sell it to? I don’t know. I don’t know. But if you tell me Oh, that’s easy. You know, all the hipsters or you know, all the vlogger vloggers or bloggers or Instagram or Tik talkers. Well, maybe, I don’t know. Prove it. Show me how you get how you got there. Show me that these. These are the people who We’ll be able to pay for it, then we’ll have a conversation. But using this example, instead of telling me Oh, all the vloggers bloggers, THE youtuber stick Dockers, I want you to say things like, well, we’re going to do, I don’t know, pressure test a few hypotheses, like, let’s see if bloggers will like it. Let’s see, travel bloggers, food bloggers, whatever. Let’s see if military will like it. Let’s see if I don’t know if somebody else will like it. Doctors and clinicians, maybe there’s other niches, but I want you to demonstrate the process, I want to see the process, I don’t want to get a get a get an answer right away. Your answer means nothing. If it’s not supported by something, there’s a lot of talk about data driven decisions, that’s fine, you can you can do that. That’s basically, pressure testing hypotheses would be your data collected that you will use to inform the decisions. Sometimes you go by a hunch, sometimes you don’t have enough data in the enterprise world with enterprise software products specially are in early stages of enterprise software products, you just don’t get to have that data, you don’t know, you can make an educated guess you can estimate.

Vlad G 26:14
That’s what happened to me early in my career, or the company I work for I was working on the ipcress product, I was launching it first time ever bring it to the market, we had absolutely no idea how many companies there would be interested. We know each of our clients, each of the companies to a degree suffered from same problems that prompted my company, our company to create this product. But we didn’t realize how hard it would be to sell it. And it took us a few months to actually zeroing in on proper positioning on how do I talk about this product to my client? Just that part a long, few months? And it’s it’s really hard. So I if if if I’m asking questions like that, and you have an answer right away. So to summarize, what are the questions from product managers, what we’re looking for? What kind of answers we’re expecting, we’re expecting to see how you think we’re expecting to see how you would approach this problem? How would you try to solve it? What is the methodology? What are the tools? One of the ways of thinking? How do you how do you do this? Basically, I don’t really care what the answer is, as long as you walk me through getting to that answer to getting it. Here’s my process. Here’s how I approach this. I think I’m going to get x as a result. Okay. Let’s assume that that’s your resolve and move on to the interview. Right? But if you just tell me, oh, it’s going to be whatever I had, I had a person, we’re talking about developing mobile application for pretty wide audience, pretty large set of people, and the person just straight up told us Nope, we’re not doing Android, iPhone is the way Android is really insignificant share of the market. So we’re not going to waste our time building anything for Android in the first couple of years of this product, was just gonna go ahead and build iOS. And that was, for me, it was pretty indicative of the way the person thinks it’s the person has all the answers. They don’t have a methodology to get to those answers. They just know the answer by heart, and they’re giving it to you. So no, don’t do that. Second part of the interviewing and assessment, the executive interviews, we don’t do executive interviews as part of the assessment, at least, not that I’m aware of at not not at the level of product management, Senior Product Managers. Maybe higher, I don’t know, nobody told me. But executives are exactly the opposite. So which is why I said they really, really suck to have an executive and the product manager interviewer at the same time, you would be in a really bad position unless you listen to me and I tell you what to do. So executives want clarity. They exist in the world where things constantly break the things and I hear and I ask I get I get a chance to ask what keeps executives up at night. And it’s uncertainty. They’re there they leave and breathe by uncertainty they their whole Well, technically speaking, their their whole existence is to make money for the company they work for. But what that means in the real world is they are battling uncertainty. There is an uncertainty there’s a risk and they try to address it, the more certainty they have, the better they feel, the better their projections come through reality. The more uncertainty they have, the worse the off their projections, they’ll meet the reality. things go bad. So with that said, when the when the executive asks you questions, they don’t really want the process, at least not to the same extent as the product managers, when the executive asks you a question, so tell me about the roadmap. My answer to that is usually Well, I confer with the team I work with the team, I communicate across multiple teams, get their feedback, collected all together, put it all together, make make make a presentation for the whichever the forum whichever The meeting is to discuss it with the executives, and present my findings for present the risks present, how they would affect the outcomes, maybe a proposed couple of what if scenarios, to make sure everybody understands why certain decisions are better, why certain decisions are worse. What should you do

Vlad G 31:10
if things happen, so to kind of sort of paint the bigger picture for the executive, to help them make the informed decision you expect them to make. And I believe that’s the biggest difference, the executive does not want to know your process. When you’re thinking about executives always think about your customers, they are effectively the same in the way that customer does not want to buy a drill, they want to buy holes in the wall. Same as with executives, the executives don’t want your process they don’t want you to sell them product mindset. They don’t want you to sell them, you know, JIRA, JIRA versus other tools, or conference versus Aha, or whatever. They don’t care. give them what they want. They want information, they want clarity, they want to understand what are you doing today, and what’s gonna happen to the product in six months, 12 months, two years, three years, and so on. So that’s probably the biggest difference. And that’s probably the biggest disconnect, when people come in into the product management interview, and they start talking to HR, and they start talking to product management. And then they started talking to executives, they tend to tell the similar or the same, a similar story. And they tend to fail. And the reason why they fail is because you’re the product, and you’re presenting it to different audiences, you presenting your story to the users, and then you present the story to the buyers. And it’s always in product management. When you position your product, there are users and their buyers, there are people who pay for it, and there are people who use it. If you approach it from this perspective, you will immediately again, if you’re a product manager, you will immediately understand the difference, and you will immediately change a pitch. Hopefully that helps. All right. One of the last things I want to discuss today, we see in interviews in assessments, and in the real work in in the work that we’re doing for clients or in internal products. We see people either over leveraging or under using certain artifacts systems solutions. At one of the things I do want to talk about is sometimes people get obsessed with tools. I remember our company spending literally a month deliberating which system to use, should we use JIRA? Should we use Microsoft product? I forgot what’s called used to be TFS. Should we use Aha. And guess what? They ended up not using anything, they end up using Excel, but they spent the month talking about it. Why? Because they are obsessed with the tools instead of being obsessed with the process. I used to say this I don’t think it holds true anymore because of reward from home work remote and all that jazz. But it holds true in the way that you can use online tools to that extent, but I used to used to say that my job as a product manager can be done with pen and paper. I don’t need tools. I need the whiteboard. Obviously, it’s an exaggeration. But it’s a good exaggeration, because I don’t really need to use JIRA, I don’t really need to use. I don’t know, aha, I can draw. It would be very convoluted exercise and probably not really productive. But I can draw or write everything down on a piece of paper. I don’t really need to use any of these sophisticated tools. As long as my process is good.

Vlad G 34:51
As long as my process helps me move things through. If my process is broken, no matter which tools I’m going to use is not going to help it nothing get done. If my process is broken, I can use the most sophisticated the best tools in the market, Mind Maps, mind reading applications, whatever AI systems that will predict with predictive analytics, nothing will help if your process is broken. So one of the things that we’re seeing in the real world is people being obsessed with the tools, instead of our process, I would actually argue that a good product manager should be good process enabler. If you come into the existing product, to work in the existing product. Or if you’re just starting and building a new product, one of the most important things to do is set up the right process. Time and time again, I am seeing and my colleagues are seeing that if the process is wrong, no matter how good you are, no matter how talented you are developers, no matter how talented, they are designers, no matter how great everybody is, bad process will ruin everything. If you’re if you’re writing user stories, if you’re building technology before you even look at the product roadmap, or if you don’t have a roadmap, if you don’t have a process to review the roadmap because it’s a living, breathing document, those developers will get upset, they get frustrated, they will leave you in like three to six months tops. And you’ll end up with bad developers bad product roadmap, bad process, back to square one. One of the things that we’re seeing, and it’s a matter of fact, back to the interviewing, to kind of circle back and tie it up when we interview or when we do the assessments. One of the things we try to get for indirectly, because of this direct questions prompts the obvious direct response. So we’re trying to see if the candidate is actually thinking about it, is give us the plan of work. Let’s say sometimes it’s a plan how you plan to release the MVP, sometimes it’s a question of what is your personal plan for the next, you know, one to three months? But it’s always a question about what is your plan to do XY and Z for the product you’re building? And part of the expected answer is, I want to set up a process a product development process, there’s whatever the your process and whichever methodology you subscribe to, it doesn’t really matter, I’m not going to judge you or not. Nobody’s going to judge you unless they’re, they’re religious about a particular unless they’re religious about the particular framework or a particular approach. And then you know, anybody who’s not using x processes, you know, not our not our candidate, but you have to, you have to, you have to say the magic words, basically,

Vlad G 37:46
you have to say that, hey, I want to make sure my process works, I want to make sure that, you know, we’re doing whatever, if we’re doing agile, right? Then we have all the proper ceremonies for the product office to review the roadmap, break down my product of the capabilities and the features until epics, what however, your your your hierarchy works. And you do this and you review care, you review regularly, your business goals, your okrs, your whatever, again, depending on what your terminology is, what your concept is, what your methodology is, but give me something that will make me feel better about you having a process to build the product, actually, actually, which reminds me at one of the candidates ever interviewed that said something like, Hey, we’re going to spend for first 60 days, building your roadmap, then we’re going to fix it in time, right? All the documentation, right, all the product requirement documents, and we’re going to start building product. And I generally, like my candidates, they’re very smart people. It’s just they just stress and I asked the candidate. So what’s gonna happen if developers discover something that doesn’t really fit your roadmap or isn’t isn’t in the product. PRD is in the product requirement documents. And again, they responded, well, this shouldn’t have happened because we would figure out everything as we ride the pierogis. So this should be no, no surprises are very, very minimal risk. Because everything should be thought about up in events that didn’t go well. Which brings me as a final final note, brings me to the topic that I picked on Reddit, because I promised we’re gonna leverage Reddit and this whole podcast started off of me reading Reddit to somebody came up, hopefully you know who you are. And you leave me a comment or something. Somebody came up with the topic on Reddit about how it was a good template for a product requirement documents. I believe the my biggest problem with the right app was about 2010 to 20% of the document would be me talking about what the product is and about 80% 70 to 80% of the document is How to Build how we’re going to build the product all the architecture, design, wireframes technology, technological specifications, functional specifications, very, very detailed documentation from which then you can construct user stories. My my concern was hate history. So you just write this one document. And people just can just take them and write user stories off of that one document. The the author said something to extend it. Yes, this is, you know, a pretty self sufficient documentation, they doesn’t need anything aside from obviously design mock ups and UI UX component. So you do understand this is insane, right? You’re going to spend six months writing it, then it’s going to be obsolete probably for four months in person disagree with me, they had a little different vision, which is why I’m bringing this up. Because I do want to give voice to people who disagree with me or I disagree with them. And one of the main concerns that it’s actually happened to me more than one specifically was prepare their pre recorded pre written parodies, before the beginning of the work on the product, is that they’re obsolete The moment you start writing them. And here’s why. Because you have set a certain vision in your head of how things should work. Allegedly, you did some research you’ve done you did some user experience testing, I’m giving you the all the benefit of a doubt, right, you did, did do some discovery. The problem is, in the real world, building product is always going to be a continuous discovery, continuous evaluation, continuous reassessment of where we are what we doing, how are we solving? How are we solving an individual issue?

Vlad G 41:48
If I if I say, these parodies are written in June 2020, when I stop doing this podcast, and I am here today, in June 2021. It’s not the case, I’m giving you a hypothetical case, if I had a PRD, written in 2020. And I start development in I don’t know, October, September, October, November, something like that. And I am here today, one year after Not a single word of those parodies will still hold true. How do I know that because it happened to me more than once. There’s no way you can write anything. If you’re doing product development, if you’re not doing waterfall. If you’re actually doing genuine product development and following one of the product development methodologies, there’s no way in hell, you can do this and survive, either a product is that you’re most likely impurities in it. But either a product is there, or your teams are dead, or your product management team is there. But most likely your purities are dead. Because most of what happens usually happens in the real world, people tend not to read documents that are longer than a couple of pages. Someone actually we were working on the clients assignments. And we gave them a playbook product management playbook that was only 80 slides. It’s not in the book, it’s just slides. And each person had maybe 20 slides to look at. They didn’t even have to read the whole thing. They could have just like flip through. But their part was 15 to 20. slides, not like 20 2020. But there’s overlap. And but most each person used to read 20 slides and they told us straight through there to our faces. Nobody’s going to read the PowerPoint deck and 80 slides and we’re like, no, they will, because that’s your job. And if you’re writing and documentation to developers, if you’re a product manager who’s telling developers how to write code, if they’re good developers, they laugh in your face, and they leave and work in another product. If there are bad developers, they’re going to listen to you and you’re going to end up with a mess of a product if you’re going to get the product at all. And again, I’m not trying to offend anyone, I’m just speaking strictly from experience. I would never ever, ever imagine I would have to write a functional specification that tells the developers how to build something, I will tell them what I want built, I will never tell them how to do it. Because I’m smarter than developers in that particular niche where I understand what needs to be done. What is the problem? What is the solution, what the customers want? What the business owner wants, what business wants, right? And notice how I say what I always say what they want, what needs to be built, because I’m responsible as a product manager, I’m responsible for what? Now, when I hand it over to developers, they’re way smarter than me, or solution architects and developers and techniques. They’re way smarter than me in terms of how to achieve that. They know way more than I do. I have a very basic understanding of things because I need to, but if you asked me to write code, I wouldn’t end I want and I can’t. And that’s because they are the smart people in the room who are smarter than me. Who will tell me, hey, Vlad, we think we could approach it from this direction. But if if something goes off, we have an alternative way. And I said, Yeah, build a proof of concept, build something that will help you determine what’s the right way to approach it. And that’s how things work. I decide what needs to be built in conjunction with business in conjunction to with UX, then UX, and developers and solution, architects will go back to the drawing board and tell me how they’re going to build it. All I need to do is I need to look at their solute proposed solution, and see if it does the what if it solves the problem I came to them with? If I came to them, if I came into development with a problem, hey, I need I don’t know, I need to chat. And they come back to me and giving me I don’t know if flight simulator will probably not looking at same thing. But if I came to them, and told them, hey, I need a flight simulator. And they gave me a flight simulator that has one plane today, but they will be able to scale it next couple of years to add other planes. Awesome. That’s what I was looking for. So with that said a little bit about the

Vlad G 46:19
these are using, and the approach and obsession with tools. And some of the artifacts don’t do parodies. There are the abomination of product management world. Sometimes you have to for documentation purposes, sometimes you have to for what it’s legal and compliance purposes of seeing those things they do happen, but don’t expect to build anything. And don’t expect to to build the actual product off of a single peer review. I think it’s unreasonable, I think is bad for business. And it doesn’t make any it’s not gonna make anybody happy. He just waste of time, effort, money. And they mentioned waste of time. Yeah. So that is it for today. Hopefully, I’ll be able to continue doing this. Hopefully, there’s gonna be a video I’ll be able to edit this. I’ve never done this before. It’s going to be my first time. So at least there’s that there’s podcasts, there’s video. Let me know if there are any questions if there are any ideas. If there are any people you want me to bring in five of people, obviously you can’t bring a bill gates to talk about his marital problems. But we might be able to bring in people who work in a specific industry specific areas. We have a lot of talented people in the company. There’s a lot of talented people on Reddit, off Reddit on LinkedIn, so we’ll be able to talk to them and we’ll be able to bring them in and interview them same way we did before. So hopefully that’s it. Thank you all for watching, listening and to it.

You’ve been listening to the real world product management and I’ll be your host Vlad Grubman. Until next time

Real World Product Management – Episode 15

June 9, 2021 Vlad Grubman

This is a backdated post. Apparently, I have recorded and released this episode on podcast platforms but never got around to posting it on the website. The episode was released on June 17, 2020.

Yulia Sitnikova, a project manager, a BA, and a product manager, tells her story about this variety of roles, how she got to transition from one role to another and what prompted these transitions.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Hello everybody. This is a 15th episode of the real world product management and my guest today is Yulia Sitnikova, who is has been a project manager at VA and play the product manager role on some of the projects and engagements that she worked on. And I think she has a slightly different story than what we usually have on this on this show. So I’m going to go ahead and let you introduce herself and then we’ll get straight to the story straight to the point. Okay. Yulia. Welcome and go ahead please introduce yourself.

Yulia S 0:52
Hello guys. So, I had a lot of projects, on business site and on site of from before The IT companies so most of the projects I belong to the e commerce so and I just want to share my experience with you guys about the maybe most sufficient project in my career up to date and probably this experience can be useful not only for US market because this project was launched for the Russian market, but probably for the other countries. So a few words about the project the marketplace launched five years ago on the Russian market the like in new business model because up to date is you know, guys marketplace is a kind of the omni channel for any ecommerce business. But five years ago, some Russian businessmen thought that it is more attractive Have a separate business separate business model. And our company as well as to our competitors started this business as a additional business based on the euro for the Russian market model. So marketplace is well known now as an online online business. But five years ago, we decided that it can be a kind of the combination of the online and offline businesses. So first of all, I would like to I would like to say that our partner was a state owned company. I can say that it is the main logistic logistic company up to date or Russian post and Russian post decided that they need maybe some expert dice outside to log on to the market. Place as new business for Russian post. I was a project manager for the Auto Group Russian known as the company with 100 years history of distance sales. And we decided that it will be interesting also for our company to share with experts with Russian post and to get additional customer base and delivery points special terrorists from the Russian post. So it was a kind of beneficial project for both of us and we decided that we can combine the delivery opportunities of the Russian post and the assortment and distance sales expertise from the auto Corporation. Also a sell already mentioned. The second key point was the idea. The idea was about to support not only online marketplace, but also create an additional way to sell marketplace products by the offline channel. What does it mean? offline? In our case, was our printed catalogs printed we printed about 10 millions catalogs, which includes all the top sellers from the online marketplace. And we’re distributed between the postal offices and by the postal boxes for Auto Group, Russia customer base. So in this case, we thought that we can support our project our sales, because as marketplace like online channel was not so popular, it was new for the Russian market. We were sure that for example, says bye bye The printed catalogs can be more successful. So the target from our CEOs was to be on zero level of for contribution. And we decided that the idea to distribute marketplace project into two channel can help can help us also, I can say that the third point, which can be highlighted in case of this project was delivery. Probably, you know, Russia territory has far north or five East regions, and most of couriers cannot

reach at least regions and derivative deliver parcels to the end users. But in case of Russian post, we could use this Deliver options because Russian post up to date has about the full year 40 2000s of postal offices on the territory of Russia and they can deliver. I can say parcels or mailings to any point of Russia Federation. And in this case, it was like our additional benefit to propose end users for our marketplace to deliver parcels and products to any point of all the operation. Also, I can say that we have some thoughts, some new thoughts about the assortment. It is important because in that time, in most of marketplaces, we saw that The most top selling products from China produced by by Chinese suppliers and producers. And in case of Russian post we decided that for example, if Russian post can deliver and get goods from any point of Russian For example, we can add to the catalog and to the marketplace online channel. Some interesting Russian goods or Russian I mean produced by by the Russian producer. So for example, it can be honey or some berries or different Altai teas and so on. And we decided that it can be important to highlight these information also in media because state owned company should also support some actions, some businesses On the media and it was relevant to the idea of the PR department for the Russian post to highlight that, for example, Russian post marketplace can propose to the ante users not only Chinese products but first of all products from the Russian producer. So in order to support the Russian market and Russian in sales for the Russian products, also, we decided that we need maybe super product and super product not not for it’s possible interest from the end users. But from the margin gross margin because for both of companies, we decided to keep as already told contribution on the zero level or have after the project lounge. Slight minus in the contribution and we needed to create or to produce or to find some products with high margin. And we created with Russian post team new product called personalized letter. What does it mean? So personalized letter E to the postcard includes text any poem or something from the dead morose. So our project should be launched in October and you know, do the first quarter of the year and since November all end users Bought mostly the gifts and presents to their kids and their families. So, in this case, we decided that we can print the beautiful, nice postcards and write the poems from the dead morose Santa Claus in the USA and add to these postcards personalized welcome message for example examples for example, you buy the I don’t know toy to your kid and you would like to it in the order the letter from the debt morose. In this case you need to put in your order. Form a name for your daughter for example. And sure each and after that our system after receiving this order can propose to this particular girl based on her age some parents, so, actually we had the poems for the adults and poems for the kids. And it was interesting also for our IT team and IT system because we needed some algorithm to calculate it, which poem should be printed on this postcard? Also, I can say that it added more personality to the product, about the margin. We actually had in the time, different producers for calligraphy and we had a good contract so we could get the good prices for the postal postal carts printing and si SM right. Our gross margin totally was about 97%. So, actually, it was a huge margin. And I cannot even leased any products with the same margin in the market. Any way, we pronounced that, for example, we can sail, a one hundreds or thousands of the postal carts from the dead morose. But in fact, we realized in January after three months after the sales that we sold about 300 thousands of postcards. So for our printing team, it was a kind of a challenge because we needed during the high season, when every retailer prints, promotions, promotional catalogs and other stuff we needed To put additional order for the postal cards, but anyway, we did it and we got our contribution mostly things for these for the for the sales for these postal cards, because actually, we had not so high gross margin on usual products even on products for Russian producers but our contribution after the first quarter for our marketplace business was about the millions on to buy the Postcards from the dead morose. Actually also we decided before the project that our CEOs could take the decision to continue with the project or to stop after the first three months of sales. When they received not not even zero but plus about positive contributions, they decided that projects

can be continued. And probably we need to do some adjustments with assortment because it is not so good to spend a lot of budget to the marketplace online to printing 10 millions catalog catalogs and receive 80% of the orders on the football possible cards. So it means that only 20% of the our end users I interested in the real products like and electronic or textile or something. So for marketplace business. Of course it is dangerous situation and we needed to rethink what we Need to save in the first quarter in the further catalogs about the catalogs. We also have some decisions because after our first three months of sales are ended, we decided that probably we need to launch new catalog with seasonal goods every quarter for example, like in fashion, winter catalog spring, summer and autumn and we decided that in each catalog we need some seasonal bestsellers on the cover for the catalog and on the most popular pages. Also, after the our allowance and short sales, we decided to test any new goods only on my channel marketplace because case, we did not have any risks for goods demand because for example, if you will print 5 million catalogs and put on the page, any good event, for example, you will receive, I don’t know 5000 orders for the one single Excel product. So you need to be sure that your warehouse has this product already, or you need to hurry up, deliver this product from the supplier to your warehouse and then to the end user. So, it is difficult to calculate in case of the wide assortment. That’s why we decided to reduce the catalog by the pages for example, first catalog has 20 The four pages, and the next catalog probably had 18 or 12. I don’t remember actually, but less pages. And in this case, we could calculate demand in best way. Because we understood that for example, from this catalog we can get, I don’t know 30 thousands of orders and we can calculate this year for each good in the catalog, which can be in case of sales. So for forecasting for analytics, it was pretty good. In case of online channel, we decided that we can test everything because you can put on web any products and you can test the demand on any product. If eventually For example, you do not have this product on the web in the warehouse. So, you can just tell the demand and for example, send some thank you letter or welcome letter to the end user and propose in other products or some agent, no promotion and so on. Also, I can say that what was interesting after the first three months of sales, it did.

Actually, we received only 5%, for sales for online channel. And we understood that probably our business our marketplace business model is not about the sales by the online, but mostly sales by the offline Of course, up to date, We can say that it is not so good situation if you try to develop marketplace business because now marketplaces is mostly online channels and they use probably printed catalogs only is as equally put to the parcels. But in our case, we understood that probably the answer on the question why our offline catalog gets so many orders in the person with an online was in the customer base. Five years ago, we did not spend enough time for non finales the customer base because we thought that the main point should be quantity of printed catalogs The sellers in catalogs and wealth distribution using Russian post deliver points. But actually we did not understand that our assortment cannot be interested for example for some regions, we realized after the first sales that 25% of the orders overall from the from the employers are Russian post and actually we thought that okay, if the main people on the main points on the trade points in our case would like to buy our products to use our products and then just to sell this product from the catalog. It will be kind of free promotion. Bye In our case, it was a kind of fraud because after three months we got 40% of the returns. And after investigation on Russian foresight, we understood that the main reason it is the administrator freedom because Russian post has some has the old structure

or some post has a lot of Iraqi points in regions in Republic’s on on the Russian territory and each postal point has own plan sales plan and this sales planning consists of the Same as for the products selling in the postal offices and services, postal services, in case of the products, they cannot, I can say forecasting the sales, because it was first experience also for the pasta offices to save the real products by the catalogs, because of course, first of all they say or different services. And I can say that in this case, probably they did a mistake and probably their plans are wrong. So high and in order to achieve this pranced to reach these plans. Employers bought the products and after the sales period and it they return them. So for us, it was a kind of the not good situation from the contribution perspective because I can see that half of money we aren’t after the last month of our sales we lost on this returns because of course, all returns should be paid by the retailer or company sending the parcels. And in this case, it wasn’t that it was a Russian post, but it was Auto Group Russian. So we lost huge millions of rebels on this with John’s on tariffs for regions, the parcels but, of course, a Russian post supported us in this problem and they I discussed with their teams in different regions about the rules, so to avoid the same situation in the future, and I can say that it worked, but not for all regions. For example, for the nasco cousin, postal offices, the station always was the same. But anyway, for example, for Far East region or for Siberia and rural visitation was was better. Also, I can say that Russian post after our first sales started to think about the technical solution because when we launched this project, they use only one merchant. So anything marketplace should based on the business model when different merchants, producers or resellers can join into the marketplace and sell their products in pay fees to the Russian post, in the case of the Auto Group pressure for the first year of our partnership with a group Russia was the one partner for the Russian post. Yes, we tested for catalogs with us to sales during different seasons. And based on these numbers, a Russian post could estimate so is it project can be interested for example for another merchants? Do we need to put the additional budget to the technical solution because first technical solution Pretty easy for one merchant. It is, I don’t know it kind of the simple workflow a WordPress website. And the one we we needed to do to integrate this website with Auto Group brush up system to get the orders and a roll call with the returns and enter users feedback after the one year selling the products with Russian post, or the group rush understood that the next year will be not so profitable for the Russian post for the article pressure because Russian post announced that they would like to add foreign EU merchants It was good companies producers mostly We use for example, good electronic products. And we decided to participate in the catalogs during one more year. But after this year, we understood that okay, it was an interesting project. But anyway, quota group ration has, I don’t know, five, four, maybe six or seven times low contribution in this case and Auto Group passion is not the producer or seller. So actually it is not interesting for us to sell products while the marketplace and we finished work with the Russian post in one year, so totally, we spent on this project two years after that, I know that Russian forced you extended the quantity of merchants and say no, they also use first of all the offline channel catalogs at the main channel for the marketplace because as I can see the technical solution their marketplace online is not it’s not so good. They added some regions some localization, but due to the assortment and absence of promotion or something, I can say that they continues. They continue use offline channel catalogs as main channel for the marketplace. I can say that’s all maybe pretty overall, but if you have questions,

Vlad G 28:55
yes, absolutely. So thank you for this thorough story in thorough explanation. So I made some notes as as we were talking and I’d like to, I’d like to start asking some questions. And probably one of the questions that I kind of yearning I was yearning to ask all this all this time is the approach that these two companies this, you know, Alliance collaboration, however you want to call it between the catalog seller and the Russian post. The approach to this program this this effectively a two year program that you guys have worked on. It was a very waterfall methodology that you guys took. So you figure out what you wanted to do. You launched full catalog, you launch this for the full market of Russia for all the post offices, and then you just sit tight and waited to see what happens is that Is this a correct Is this a correct assumption?

Yulia S 30:03
I can say it depends because for example for the logic rapacious budgeting system, maybe you know Germans in German companies, they prefer to forecast any sales for three years from from one from one to three years. So, all their project planning is about the planing minimum of one year and maximum for three years. And we are responsible always for the contribution be in your contribution and in your in your net sales. That means that most of the projects, for example, for Otto Group ration is based are based on this forecasting system and in my case, when I needed to lounge the project’s product in three months. It was an unbelievable situation because we did we even didn’t have the tools for financial planning for such short terms. And in this case, I divided for CRM, my methodology for working with this project into two parts for example, for the budgeting, for the assortment planning or for the warehouse and logistic planning, I used waterfalls. So, I gave them the forecasted figures based on my thoughts and after the show sales based on the on the sales for the first quarter, but for example for the IT department, and for the For example, CRM department, I can say that we worked in Agile methodology, because we changed a lot of the process processes, business processes on the CRM side based on the feedback from from the end users on it site. For example, if you have the three months to deliver to integrate your your, your earpiece system with the you application, so, you need to be you need to be jail because, of course, we did not understand what product what technical solution will be. So the one which my team got from the arrest and foremost was the plan, plan in projects. So Based on this plan we needed to discuss For example, we should meet, I don’t know, one time in two weeks and discuss our steps. So, we are moving we were moving step by step with the IT team from the rational side because we took the information from them. So what was delivered and then run to the our IT department and do integrations on this particular part. So I can say that for some departments in this project, it was an agile and for some processes, it was a kind of waterfall waterfall. Mm

Vlad G 33:46
hmm. Okay, okay, that makes sense. And yes, I was kind of expecting, I was kind of expecting to mix but the larger picture was still moving at you know, in a waterfall fashion. So I was kind of thinking, um, I’m just gonna say it and then I’ll let you disagree and see how you will respond. I was kind of thinking that if this is this is a very old fashioned project management project management methodology that is typical for large companies. It’s there’s nothing wrong with that. I mean, they’ve been using it for a while. And we know that we still work with that. We try to work around that when we introduce a product mentality or product mindset. So what I was thinking of was, let’s imagine, right, let’s imagine a slightly different scenario where you had an ability to move faster, you know, in a product, building fashion, but deliver less value, but also at the same time, you’d be able to get your answers faster. So for example, instead of a fourth quarter, November when you actually look launched this. Let’s say for example, again, it’s an imaginary scenario. You were able to launch in August. And you would get the answers that hey, catalogs are better by September October, and you would know that not in November, not when you just started then and you know, all things are set set up differently, but just started getting you started getting this, you know, for side, you start knowing what you knew late in the game earlier. How would you change the Christmas season, which is when you guys made most of your revenue? How you change the system now knowing what you know. And and and being able to deliver chunks of functionality or chunks of this offering faster? That makes sense. There’s a question makes us Yeah,

Yulia S 35:52
Yeah, Yeah, I agree with you. Absolutely. Because, of course five years ago and in German competence Company two was a kind of reusing all purchases for the new project a new product. So, I can say that first of all, we needed to do the customer customer development because actually Russian post as well as Auto Group Russia have own customer base. And for example, we we could send some catalogs to different target groups from the database from the clients base for Auto Group process, for example, based on the farm locations on the Russian territory. And for example, for the our best clients, our I mean, Auto Group Russia. So also we could send different catalogs for example to test Do we need actually to print 10 millions catalogs because it is the Huge amount for pre spending spent for printing, or probably, we can do different formats. For example for one target audience, we can send, I don’t know, email for another target audience we can send the catalog with I don’t know why hundred pages for another target group for 24. So, we could attest on a lower volumes, even printing the catalogs and there’s the assortment in these catalogs. For the online channel, I can say that we could test different clients based I mean online on the client base because we focused on the offline channel. First of all, because it was easier for us, of course about in case for example, if you would like to develop the market Place as online channel assists, you probably need to test the different online audience. And we already knew that Russian post sales a lot of services buy online for their customers in that time AliExpress already was the first and the main, I don’t know ecommerce company in Russia and Russian post you about the quantity of parcels for each end user what they order it and so on. So, deep analysis of the Russian post customer base online customer base, also could help. Maybe do not spend time on printing catalog on buying the products, which which we needed to keep on the warehouse to sell For the catalog sales, but to use the online channel, maybe in most of cases using the current Russian post clients database, so I can say that we need it actually cause deaf and in this case probably we might spend less budget and earn more contribution, maybe Event of the Year for example, if you understand that your catalogue have each person have the postal card sales. So do you need actually 24 pages I can see now I can say we could send on the I don’t know. six pages, catalogs and good The same contribution from the postal carts gross margin and spent much more with less money on the printing and delivery they get a loss because of course, we also paid the delivery the catalogs to the postal offices on the Russian territory. So,

Vlad G 40:21
so I’m sorry to interrupt. So let me help me understand a you would pay different amount of money for for a smaller catalog. So like, it depends on how many how many pages they are, so it weighs differently. Yep. And then you pay less. Okay. Yeah, makes sense. Interesting. You said customer development and not customer research. As an know your customer versus customer development. I was kind of hoping you would go into somewhat different direction. Although I I agree with what you said. You’re not wrong. I was thinking. Maybe there is something you wanted to do in terms of educating Your customers about the online services. Because most of most of your response, ro ease is situated around, you just you know, we could have sold better through offline catalogs, which is true. But, you know, the point of the project was to build an online marketplace. And I was kind of hoping you would go into the direction of Oh, hey, we could have spent this time making our customers like online interactions. Do you think is a possibility? Do you think something could have been done? Back in the day, you know, those five years ago when you started this project? Would you be able to do something to increase the percentage of your profit or percentage of your revenue generated through the online sales or was it not completely not the right time and not the right place? And never are going to happen.

Yulia S 42:01
So I can see that it is the kind of my thoughts, but I can say that five years ago, it was not the right time for the online marketplace. Because on that time, we had two competitors. First competitor award the real marketplace for the fmcg goods launched by the Stanford students, because they would like to move this idea, this business model to the Russian market. And as they know, they closed their business because actually, the main reason was mentioned by the CEO of that company that the idea of marketplace is good, but that it was not right or right time and probably Probably our idea to print the catalogs saved our money in this case, because I can say that, yes, we could spend more budget for the promotion and advertising, targeting and so on online, but I’m not sure that our end users were ready to buy the products using this business model. Because for example, the next competitor was more successful than we and the closest competitor, it was it is a company called pvh. So it is a kind of club with the for the goods with the

great discounts, which I available, I available for the limited time so this idea was good for that for for the market on the time, and that’s why they are alive up to date. But for example in key so the competitors with the similar business, they closed his business and on the reason that it was not the right time, I can say that probably, of course, we could use the online channel in more ways for example, I know that Russian post rejected any budgets on the promotion. And they thought that for example, if you put the link on the porch, the market marketplace on the main porch that.com, the true website where where people find their parcels or direct their parcels, so it will be enough for the bush tomorrow. Because everybody knows Russian posts, but in our case, maybe because of the new business maybe because of the not so good promotion from the Russian post. Actually, we did not get enough customers for online channel, I can see that. From the AV perspective, I mean, average order value, online channel was more efficient because, for example, users could buy two or more goods in one parcel. So if, for example, we will wait three years, I don’t know four years and started to do targeting started to do promotions for the online channel. I can say that, for example today, this marketplace can be Can can be the business with good contribution. But of course, it is a long way it is a long way to the hour days.

Vlad G 46:15
I see So you’re saying that was not the right time not the right place. But if if you were to start the same thing today you would be successful.

Yulia S 46:23
Yes. With yeah with opportunities. So which Russian post has it is definitely because even today I can say that nobody even Yandex in other ecommerce platforms do not have such delivery network as sir I suppose test. So it wrote Yeah,

Vlad G 46:50
because they’re they’re the largest. Yeah, I mean, even even here United States we have companies large companies like Amazon Walmart partners. up with the United States Postal office, which is basically the government, the government Postal Service. They, they they have them as part of the logistics chain. And there there’s a lot of partnerships one way or another that utilize USPS capabilities. I wanted to ask you a couple of more questions about this, this whole thing with with understanding that this is, you know, five years ago and and things were the way they were. So, you said there was no promotion on about this, this whole service. So how did your customers find you? How did your customers know that they can receive this catalog? Or did they just announced, you know, did the catalog and analysis just arrived in their mailbox? How did that Bark Bark? In other words, how did you made your service known to Your customers?

Yulia S 48:01
Yeah, actually we have we had a few ways. So first of all, in case of the printed catalogs, we used customer base, Russian post and Auto Group, Russia customer base. We just sent the catalogs to their mailbox, physical mailbox. And so we knew that these customers really appreciate printed catalogs, not online channel. That’s why we got for example, orders from these customers and the response rate, I mean, conversion conversion was pretty good about the 4%. In case of the catalogs distributed to the postal offices, it was another case because we are not The orders we got from the postal offices because each postal office so, the team from postal office should send the paper order forms from the customers to the head office and to the our CRM department. So, we knew who that So, what are the H? What are the H for this customers their personal data and after data analysis, we understood that we got about the 90% of new customers so, we did not have any cross with our I mean, Auto Group Russia. Customers who who always buy the products by the catalogs And it was a kind of surprise because we did not understand what to do with this customer base because they order it only in postal offices like informat MLM business so somebody from the postal offices, proposed them some offers from the catalogs and they bought it so we did not understand So for example, if we will skip the postal offices and send them the catalogs, do we want to continue with buying our products or it was a kind of the one time one time sale. So for online business, actually essay, total radium. We put the link on In which the market marketplace on the main porch their website, so this site can be used by any person who or who orders the products from the alley AliExpress or I don’t know, another e commerce companies and not only commerce or who always buys the products, not products but postal services. So we thought that it is enough to put this link and any banner on the main poster website and after that, this traffic traffic can be used by the push to market but I can say that it did not work because probably First of all, People who check I don’t know, dragging their parcels on the main page, the website, first of all use this site for the main purpose to direct their parcels but not to why any additional products. So I can say that they needed additional promotion. Maybe you Chanel mailings or something.

Vlad G 52:26
Okay. Okay. Thank you. That makes sense. Again, given the context of that time and given the context of that market, it makes sense. And x actually explains the fraud that you mentioned the 25% of orders that generated the return. You said about 40% of the of the ghosts were returned. Yes, that makes sense. And I was wondering, was there anything unusual Was there any other challenges On, maybe again, the service offering side of things, right? I’m trying not to say the product because technically that was not a product that was a program divided into multiple projects. And I understand that. Was there any Were there any challenges, especially around the logistics in and I’ll explain why. In the software world where most of us live in. But as software product managers, we tend to forget that there’s a huge world out there the real world where certain things don’t really abide by software rules, right? The deliveries that instantaneous deliveries not electronic, there’s a physical item that needs to make it from point A to point B to point C, and over you need to be able to track it or at least you need to be able to make that physical delivery. Personally, I’ve seen this and one of the systems I was a part of building, not building Myself, obviously, but was part of planning and building was a set top boxes tracking system for one of the cell phone, TV and other service providers here in United States. Again, I’m not sure if I can name names. Yeah, so I’ll just I’ll just be generic. Yes. And one of the one of the funny things was that how much details you need to be able to capture for a physical item that is traveling from, you know, through several kind of several goalposts or several checkpoints like this item is at the warehouse item left the warehouse and he’s in the distribution center item left distribution center and he’s on truck. traveling to the customer item has been installed with the customer item has been activated and now is in the working condition. And now you have all the telemetry coming in. So the was interesting. It’s still saucer world, but it was kind of was in touch with physical world. So I was wondering, you mentioned Far East regions, and again, since I’m from for me, so with union that kind of have an idea what you’re talking about, but for for the audience, I wonder if there were any issues. And I remember we mentioned that when we were prepping for this episode, that there were some logistical issues with delivering to forest regions. Now, if you can, maybe briefly talk through that. Yeah. What was the challenge? And how did that challenge manifested itself? In your offering? What What did it change in your service?

Yulia S 55:43
Yeah, of course, we faced with the challenge with the fara foreigners region. For example, probably you know, but you’re in three or maybe five months. You The Far North region is absolutely frozen. So, I mean that if, for example, somebody from this region, or durrett, the product from your catalog or buy online does not matter, you can deliver this product as this product only in several months. So actually, we had on our logist in our logistic Center, the calendar, so it was the, I didn’t know freezing and freezing killing there for the far north or regions. Because we should understand for example, if somebody from the Sweden’s or the with the, I don’t know cookies or tea or something. So you need to understand when you need to deliver this product. So for us Sample our warehouse employees storage these orders during probably one or two months before this region will be unfreezing and only after that deliver the parcels to these regions. Also you need to understand that for example, if even you have this calendar, so the climate can be different and for example spring in one year has a differences with this print in another one year. And for example we faced with the situation of when we delivered some parcels to the farmer knows and understood that we cannot get any payments for these parcels because they have the cost on delivery. Payment during probably three months and only after investigation and chicken with Russian post we understood that these particular region has I can say the longest winter season and it need more time for freezing and only after that by helicopter helicopter parcels could be sent to the to the people who order it by the catalog and by the marketplace. So, was it was a kind of surprise because you cannot forecast demand on this products you cannot forecast the payment and returns on this parcels because for example you can we got the returns from the far from the far east and far knows, even in six months and for example, when a calculated business model, first of all of course, I took the medium of figures for the Russia for example, you can receive the last return or in three months since date of sending the parcel, but for example, in case of the farmers, you could receive the return in six months, so, and it will bouquet So, it means that, in some cases we needed to update business processes also for CRM, because probably you know in CRM, for example, if you sent the parcel with the payment date, in one month, you did not get any payments. So, this particular customer consumer can be added to the blacklist and we understood that we need to rethink to review our business practices for some regions because for some regions we needed to white released all the end users because they had a huge parcels and orders on a huge amounts but they cannot just receive their parcels due to the climate within and they cannot wait but we needed to change our CRM system, our earpiece system or for this type of clients. So it was a pretty interesting call. So in case of the postcards because I can say that we lost our sales because We just started to send the postal card scenes in November. But in case of the foreigners, I can say that we needed to start to send the Postcards from the admirals from the September to be sure that keys from the far north can receive the letters from the dead Morris in the right time.

Vlad G 1:01:27
Okay, yeah, thank you. That’s that’s something that is something I was. I was wondering. Now you could share. Okay. Probably last question that I have about it about your story. And it’s more around the planning and financial forecasts, I remember you said you were forced to create a forecast for three years and then you divided that into a smaller chunks to so it would be manageable and you can work with that. Let’s not spend too much time on that. But if you can Just briefly talk me through, what did you do in order to avoid having to forecast the whole three year span? What could you possibly have done back then to improve your situation?

Yulia S 1:02:14
Yeah. So first of all, we decided we CEOs that cannot forecast this project for three years, because, first of all, to the kind of new business new business model, and we got a lot of new points, which should be to the business model. And I can say that we had a lot of dependencies on merchants or new merchants on Russian pause because if you work with the state owned company, it can be kind of surprise some changes in the rules or business process. So, it is the kind of risk for any commercial company and we decided to with the supervisors to calculate only forecast for the one year for the one year First of all, so, actually, I used the business model which calculate some of the main the main role. So, in the our budgeting like or net sales demand, gross margin on products contribution level percent of the returns, because we understood that is the kind of risk for the contribution to have more returns. In comparison with plant also we use like epi a V because we understood that for example, increasing The AV I mean average order value by even 200 rubles can rise our contribution. So we need to sing to think what we can do in get along which assortment we need to propose some suggest to the end user to authorize our a VM. Also, we decided if we want to bring to the catalogs one time per quarter, so we can divide our annual budget and business model calculation into four parts. And each part has different different First of all, net sales. Because for example, the lowest lowest net sales Aurora in summer and of course, the highest In winter, so it means that, for example, even for each quarter, I had different KPIs for AV for the quantity of four doors for this year for the for example, of course, I understood that for the new year time, my share of the personalized letters will be higher or highest during the year. And for example, in summer, we do not have any holidays when people can send to each other personal letter, so I need to focus on the products. So in this case, my ARV can be different in case of four quarter and for example, second quarter. So all this KPI was pretty different and have dependency on the Season, a winter, summer, spring and autumn and totally after calculation each quarter and KPI for each quarter I needed to combine all these parts to the in your plan in your budget. And our CEO can could estimate is it will be profitable project for our company in years year or not. Do we need I don’t know two to eight budget to it some more products or something. I don’t know do it some watches for the serum department or call center for example if you would like to attract more customers. So it was kind of mathematic based on the in Europe Plan. So, when you’re already calculated for parts, you can easily to understand when you can lose the money I mean contribution and for example, which Porter can cover all your losers during the for example, in this case we agreed with our supervisors for example, for the summer sales, but can be even in minus So, it means that contribution will be not positive, but it is okay for us because if we will see the in yield, then we understand that you are in four quarter, we can cover all the minuses for the summer and even an additional contribution.

Vlad G 1:07:52
I see. Thank you. That was interesting that was interesting. And that was very useful in understanding how how to approach this and how people will probably approach this as they as they work on these issues when they are forced to operate in Prague program project environment without the product mindset, I’m not advocating product mindset for you know, as a key to solving all the problems clearly it would have been beneficial in your case to an extent not not everywhere not even in all the not in the in all the areas of operations but in some definitely would have been helpful. Thank you for the story and thank you for answering all my questions.

Yulia S 1:08:41
Okay. Now, thank you a lot for the chance because it is really interesting to describe your projects, which which which was five years ago, but to the kind of the good Time to me to understand maybe my mistakes or what was good what was not. Because only in this case you have time to think what was done really and which project was launched by me and by my team. Thank you for this chance.

Vlad G 1:09:18
Oh, sure, absolutely. Thank you. You’ve been listening to the real world product management and I’ll be your host Vlad Grubman. Until next time!

Real World Product Management – Episode 14

June 17, 2020 Vlad Grubman

In this episode, I am talking to Alberto Soldana – a seasoned business analyst who shares his real-life experience working on several products, the challenges he faced, and the future of BA and Product Manager roles.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

This is real world product management.

Hello, everyone. This is another episode of the real world product management apologize for a delay. Last week, as you know, there were some events in New York City that kind of sort of distracted me from recording the episode but we’re back to our regular schedule. And I’d like to introduce Alberto Saldana, one of the VA slash product managers at FM. I same company I work for so I’ll bear the Why don’t you go ahead and introduce yourself.

Alberto S 0:44
And Hello, everyone. My name is Alberto Saldana, I’m from Mexico and I have quite good experience as a business analyst almost 14 years and one of my most exciting experience as a product manager Your last few projects that actually fail. So thank you, Vlad, for inviting me and to be able to share experience with all your audience. I think we’re going to have a very good time talking very good information to share. I have heard some of your previous podcasts and they look awesome. So thank you very much again for inviting me. And I hope your audience are enjoying

Vlad G 1:25
this talk. Oh, yeah, definitely. Thank you. Thank you very much. By the way, since you’ve mentioned some of your previous projects or engagements failing, you mind telling me a bit more about what is it that you did? And obviously, what was so challenging and what’s, what triggered the failure in in those engagements?

Alberto S 1:49
Well, that was a very cool story.

Unknown Speaker 1:52
It was it was 2008. Do you remember there was a recession all around the world? And unfortunately, I got fired. From my, from my, from my, from my job, right? And during that time it happened about two months without job, right? It’s like an employee I offer myself as a freelance Ba, okay. And someone called me and say hey, you know we require a freelance ba for six months project. This is not a permanent job. This is just just six months and that’s it we will not have a contract, we will not have any relationship job relationship. So we just need you to do the work and that’s it. Okay, I don’t have anything, anything else, or more funny to do. So I accepted it. The project was about creating an app. And this is very related to what we are leaving nowadays because there was three remaining three main instances there was editorial company who actually They do books, they write books, and they do illustrations for books. So they were full of UX designers and intellectuals and people who actually knows very good grammar and great books, right that that’s actually their primary activity in a daily basis. The other instance was bi company. They were more into data, right? They extract data from databases, they collect this data and analyze it, clean it up, and so on. So everything related to business intelligence. And we had the stakeholders, right the stakeholders, what they wanted to do about this app was to combine the experience from the tutorial company with a bi in order to create an app for schools. So this app was intended to extract data from the app and analyze how is the student’s behavior based on race based on economics based on where they leave where they come from, what is the family structure if the parents were married, if there were not married, if they were divorced, if they, if the students live with the grandparents, and so on, right, so all this data was supposed to be analyzed by the analytics team, and then provide some statistics, some dashboards and information that could help the school to to improve their programs, right. So based on the assumption that not all the students learn the same way or should be treated the same way. So that was a very good approach to improve. Students in the K 12. Great, right, which is actually preschool to middle school, I suppose. So it was a very interesting project. So they hired me as a BA right to connect all the dots to be involved with editorial company, the stakeholders, the analytics team and create a product. Okay. So after some years, I realized that I was doing kind of a product manager role because my job was to analyze all the insights and try to connect them and create a product that satisfy the stakeholders. Right? So one of the challenges they face was like, there was

Unknown Speaker 5:27
a different perspective. So what was the strategy and the vision of the product while the stakeholders wanted an application that actually provides dashboards and information for schools. In the other part, we had editorial who was more interesting on create a very good product very easy, friendly, who provides a lot of features for the school for teachers, for directors, for principals for for the, for the parents of the students, and for the students itself right to have that task and homework, right? So this is very interesting because it was around 11 years ago, right? So 11 years ago, we didn’t have that necessity that we have right now, while the students are at home, and they use conference applications to, to do their classes and so on. So it was challenging because we didn’t have any information about how the school how the schools plan, their programs, and how the students actually attended to classes and so on. Anyway, we, we were able to collect different information about this, how this application was supposed to be, right I mean, division, I mean, what was intended to be, however, the strategy was not quite the correct one because we were focusing more on the analytics, on the dashboards and so on, which was not very interesting for schools in why is that because the schools, they have their programs, they have their legacy plan. Right, like you have to attend to school, you have to sit in a chair and you have to listen to your teacher. Right. And they were not very willing to use tablets or laptops in order to be able to input data. And this data will be analyzed later, right? Because they say, how you’re going to teach me how to do my work, right? If you’re not a teacher, if you’re not, if you don’t have any experience on teaching students from four years old to 12 years old. So that’s why one of the key points that that the product failed because the strategy was not good plan, we should focus more on the product that was very exciting for the schools, and at the same time, it provides data that can be analyzed later on. So the first thing I did was to extract features to collect the information from the stakeholders from the school and from the tutorial company and create a design approach right? I create a mock Create the features in the product backlog and so on. The problem was that we first develop the dashboards and the BI analytics and how to strike data and so on. So, when we first went to the demo to show Okay, so this is all the capabilities that the application will have this could say, I’m not interested, right, this is not what I need. So, this is one of the biggest learnings I had, because you need to understand what is the market needs, what are the customer needs in what is intended at the end of the day. So that was he was very, very interesting and challenging, and I learned a lot from from that experience, and I try always to focus on the needs which is the the primary thing we need to focus when we are developing a product. And I want your audience to also understand that not you Because they think they have a strategy, that strategy will work. Because sometimes the strategy is not aligned with the market needs, or the business goals. So that’s the first thing I want people to focus in. That’s why it’s very interesting to talk about these points. And thank you again for bringing up this question. Sure,

Vlad G 9:22
sure. My pleasure. And it’s really, really cool to hear the, you know, a good story about the failed product. Unfortunately, I don’t hear a lot of those around me hear more about Oh, hey, this is you know, this is how much success we had with this, or this, how much success we had that. But the real the real thing is, is the failures I keep repeating this. I quote someone I can’t remember who that was that scientists love failed experiments because they don’t even call them failed experiments. They call them experiments, recent data, because they provide a lot more Information about what went wrong and how to make your right and then the experiments that went right from from the start. So thank you. That’s this was early educating. And I think if I’m not mistaken, this is one of the few examples that I’ve heard of where the vision and strategy gone wrong in again, in a lot of failed cases, you hear about things like, Oh, hey, we had a good idea, but it was executed wrong. And it’s, it’s kind of a cliche, reason for failed products. Like we had a good idea but it didn’t work. But I actually had a similar similar experience, where the vision was wrong and and we stopped at the prototype stage. We stopped developing product at the prototype stage. So this is this is really cool. And I like how you how you talked about the transition. From freelance VA to a full blown Product Manager, when we were prepping for this for this episode, you mentioned something about ba, ba role being disrupted. And I wanted to sort of piggyback on your story about transitioning from VA to pm, we have a lot of people doing this. And in our company, we have a lot of people doing this and other places. But you mentioned something around VA roles being disrupted and I’d like you to sort of unpack and talk more about what specifically is being disrupted and how from your years of experience, how do you see this happening?

Alberto S 11:42
Yeah, and this is this is a good question that is coming up nowadays. more often. Whether there is understanding that the BA should become a specialist or should become generalist there are there are ba certain ba or different roles it can be engineers. So, it can be project managers itself who can be more experts on what they do right. So, they certify on days right on certified business analysis or they can be pmis or they can they can certify in Java core aid and so on right. And that is totally fine. It is nothing wrong with becoming especially something however, the trending in the market is that the products require more like a generic type of role in what does that mean? That means that you cannot just enclose in a bubble like Okay, so this is my role and I will not do anything else. Because that way you’re not bringing value to the business. So you should people should be, is it not like I should, but people could be more open to perform different tasks and using different skills. But what happens when you have a To perform different tasks, and you don’t have the skills, that is 100% failure, right? That is one of the alerts, we need to take into account. Say that, And based on the experience we have in our projects and the companies I have worked on, you always need to prove the skills in one of the skills that the BA it’s more feasible to learn. It’s a product management role. Why? Because they share different skills, right, like communication, engagement, on understanding on the business understanding on the technology, how the development works, what is the end to end life lifecycle of a product, so there are certain skills that are sure between product managers and business analysts. So that’s why a lot of business analysts are more interesting in learning about product management. And that is a very Good, very good approach. Because that way they’re adding value to their careers to their resumes, and they’re adding value to the, to the business and to the development team. So from my perspective, the people who who is learning the product management role, should be aware that it’s not an easy an easy job, because I have heard a lot of technical people say no, you know, Product Manager is very desease just giving orders and do documentation and talk to people. So that’s very easy. Anyone can do that. And yet everyone can do that. But Can anyone can do it? Well, can anyone can do it in the proper way? Can anyone can do it, adding value to the business or to the project? That is the main question. So that’s why I’m understanding the need of learning and having Of course, of course the Rizal learning learning curve, right? So you will not born being a product manager, just just because just because like that, right? You have to get trainings, get more experience, of course, have your own failures and learn from them. It’s important in order to become a great product manager

Vlad G 15:19
saying that he’s saying you need, you need training, you need experience. What I noticed, and it’s both, it’s with a lot of online resources, I see a lot of people who want to become product managers without experience, and I understand that we need as a general industry needs more product managers, then we can we can have so there’s going to be some you know, so there’s got to be a way for someone to become a product manager without prior experience, otherwise, it becomes a chicken and egg problem, right. But at the same time, I have a very strong opinion that Any person who is trying to become a good product manager and good being the key word here, not just a product manager just you know, doing things, but product manager that gets things done, should come into the field was having some experience of getting things done before. It could be a project manager. It could be a developer, it could be a VA, but like, for example, I came to be a product manager after a career in a tea. So I was a developer, I was a project manager, and I had two businesses before I became a product manager. So not only I had exposure to getting things done, I also had exposure at running the business, which is what product manager essentially is doing on the high level. Not Not everybody’s exposed to all these elite aspects of it. But there’s definitely more than just you know, talking to stakeholders and condensing down your product is great. It’s also taking the product to the market and selling it. That’s that’s kind of what is what is being overlooked. And I think again, this is something we talked about before, when we were prepping, there’s the classic agile model that companies are operating on trying to operate on sorry, is does not have a role of a BA. So in a way, the role was made obsolete. Some by some point in time when the whole agile model is introduced, yet, almost every company I’ve seen almost every project I’ve seen, almost every engagement I’ve seen, has agile teams, and has ba as a part of the team. So how do you see that what’s your take on that?

Unknown Speaker 17:47
Yeah, you’re bringing up a very good point on making the BA role obsolete. And I see that as true not that not all the people understand When Dr becoming a specialist on the BA role, because and nowadays, Venus analysts are required to perform also Scrum Master role, sometimes project manager roles and testing roles. And it is not purely like it’s getting obsolete, but it’s being the role leads being evolving. I call it like that. So they’re always been evolving. It’s like if you have a Java developer who is very experienced in Java two, or geometry or Java four, it’s it’s not like they’re becoming obsolete. But now the training needs to learn Java eight, right? So they need to learn it. For example, the UX designers, so if they are still working on html5, and now the training is using Angular, and they don’t learn Angular, they’re getting obsolete, right? So they need to evolve the role so they need to learn new skills, learn new technologies, and be able to To perform the work as the business requires. So the same happens for the business owners right so they they need to learn project management skills, Scrum Master skills, of course kind of decision making skills, more likely as a project manager how to plan a project understand the full lifecycle of the product. And that’s one of the takeaways for people who thinks that the business analyst role is the same like 10 years ago, it is not it has evolved. And people need to understand that in order to be more productive in order to be more effective on the daily job basis, because at the end of the day, as much as you add as much value out to the business then as much capable and more knowledgeable and more efficient you become so Again just as a summary, the business analyst role is not getting obsolete, it just it just been evolving and people needs to evolve to interesting

Vlad G 20:10
and I agree with you in a way that the roles are evolving including ba role. Same way. Same way, the product manager role is evolving I keep seeing responsibilities of a product manager where non have existed before or seen product management being divided into separate areas. And just the same way product managers becoming either specialists or generalists. I like to think of myself as a more of a generalist, because I have experienced in multiple industries and I understand that I lack you know, depth and precision in specific areas, but I can bring you know, things across industry, the experience and understanding of things across the industries. And it’s not just me, it’s everyone in this in the current world same way DBAs are doing the same way others are doing this. So you you become a subject matter experts in how to cross the industries rather than a specific industry.

That’s right. So, let me kind of transition to next question. Then we talked about the A’s, we talked about product managers. And this is one of the cliche questions I tend to ask. So how do you define the role of a product manager you know, we have in different economies, we have Product Manager, we have product owners, we have proxy product owners, we have days. Sometimes companies build hierarchy around that sometimes they just throw everybody in the same soap. So what from your experience from from, you know, all the roles you played on? Probably Where do you see those roles? How do you see those roles aligning

Alberto S 22:05
for Okay, so first of all, we need to understand the structure of the project. There are different structures in every project. And there is not there’s not just the right or wrong structure is just how the structure fits more into the nature of the project. So I see the product manager, discovering what the user and what the market needs, and also helps them prioritizing and helping to build the product. And of course, one of the deliverables or one of the documentation that the product management does is product roadmap, right. This document can also be done by business analysts or project managers. However, the nature of the document or the deliverable should be made by the product manager who is involved with the business stakeholders, marketing, economics, and so on. And also the product managers I visualize it as the main the main role who gets the things done, it can be also combined with the product owner, but the product owner is more like a product backlog owner right he is more interesting in features and what is being developing next, what is coming the next iteration, what is going to be the demo or the sprint review, approve whatever was develop and so on. So, product owner can also be seen as SME as a subject matter expert, who is more in touch with the business with the processes with the day to day business activities. And not always sometimes they do but not always. It’s not like a must be technical knowledgeable. Right. So sometimes and I have seen a lot of product owners would they don’t have a clue about technical implementations about this. development lifecycle about how the giant team works, right? So they were just okay. So you are the expert on this domain on these processes on the business, you will be the product owner. So you need to be the owner of the product backlog and make sure that whatever was intended to be built is actually implemented at the end of the spring of the duration. You mentioned also pro proxy product owner. And it’s it’s kind of similar as a product owner. However, these guys technical, he knows some more technical stuff. He is more involved with the technical team, he can provide their suggestions, their approaches, sometimes they come from a technical background as a developer so they can also provide some solution design approach and that kind of stuff that product owner sometimes and I’m not saying like a general thing or like a must be The product owner cannot provide some technical details about implementation rather than done the proxy product owner. And then the business analysts come into play to connect all these dots, that that’s the kind of a strategic role from my point of view, because he’s not having a direct reports as well as the product manager. But the business analyst has more like

responsibility to make people communicate each other without being communicating itself right. So they go to the business and product owner or product manager, collect data, start developing specifications, understand the features provide sometimes insights about what is a better approach to implement a certain feature, and then going to the technical team to explain how it’s possible. be implemented, in my experience during the grooming sessions. And when when we review the specs, the technical team ask, okay, but why we need this feature. And sometimes the business analysts, they’re not aware of why they need because they didn’t ask why the business need this feature. They say, okay, so we need these drop down and show these data in, click a button, and they don’t understand what is the business value. So that’s why the business analyst is the connector. It’s how the transition transition goes from business needs into technical implementation. And that’s one of the skills that sometimes product managers or product owners, they don’t have that skill. So, now talking about the structure again, I visualize the product the product manager, as a driver on the business needs. Create the business goals and understand and apply this strategy. I see the product owner just next to the product manager, owning those features, owning that strategy and making sure it is implemented. And one line below them. It’s the business analyst who interacts with all the stakeholders, talking about businesses, stakeholders, technical stakeholders, management stakeholders, in order to create a common understanding on what needs to be done. That’s kind of the structure that has been working more for me. Of course, there are different structures and different projects, but that’s on my experience how more successful project looks and structure looks like.

Vlad G 27:49
Interesting, thank you. Thank you for the insight interesting that you’ve mentioned that it’s his job to be this you know, this bridge between the business and its class. Team because in the classic agile, that’s exactly the job description of the product owner. I’m not I’m not arguing I’m just know kind of pointing out that in the classic agile form VA roles doesn’t exist. And that’s what po is supposed to be doing. So, your product owner is basically a BA with some additional responsibilities in and I just said this is this is a practical example but this is kind of this is why I like to see different different perspectives. They’re they’re different points of views and different experiences. So you can say, Oh, my point of view is right, or I can say, by points of views, rifle have kind of, we all have proof that you know, our ways of working are working. They’re functional. So in the, in your example, in your structure, I email I would imagine I’d be very hard pressed to explain why would I need a product owner in that scheme of things, I understand what the product manager does. He owns vision strategy and high level, high level functionality of the product capabilities and features. I understand why I would need a low level lower level ownership, whether it’s appeal or Ba, who would honor feature epic features, stories, level functionality, and would interface with the business on a tactical level resolving tactical things. But there’s this, in my view, there’s no third person who don’t need the third person there in the hierarchy is a high level feature or palatal capabilities, ownership and low level capabilities ownership and that’s it. There’s no middle level there, but I understand that you’re coming from a practical perspective. So what I’m saying is not necessarily true, again, which is why I like having different People have different experience on the show provides you different things. So let me just unpack what you said a little bit. Question question that I have in my mind. And again, I’m assuming this, this is what you’ve tried this is what worked for you. How does product owner work with Product Manager in in that in that structure? What what are the How do they divide the divide the ownership between themselves? Yeah.

Alberto S 30:33
Let me tell you a true story. And these will provide more, it will be more visible to understand the structure I just mentioned.

Vlad G 30:41
Oh, absolutely.

Alberto S 30:43
And I’m not going to say the customer name, not to pointing fingers. But this is this is what what happens in with that customer. So we’re building a large set of products at the same time. Okay, one is for 4g scene one is for creating more engagement with the customer. Another one is to create analytics that provides better information for the for the customers and customers who are navigating in their application in the website. There is another product to enhance the actual look and feel of the of the website. It’s a completely rebranding. And there is another another product just for normal processes, right like, this is online shopping. So there are some new enhancements on the product related to return some reforms, order order management, shipping and so on. Right. So we have a large number of product owners, okay, and they are from business perspective the experts on each business business processes Me. And what happened is like they don’t have background on agile methodology. They don’t have background on technical implementation, they just know how to do their work. Okay. And what happens is, as you mentioned that the product owner is the one who is in charge of communicating things between project teams, development team engineering, architects, stakeholders, product managers and so on. That is true. However, in practice is it’s ideal, but it’s not what is happening in the real life. So what happens is one of the challenges where we were facing is that we explain the process of development we require these in order to create specifications, we need to set an expectation. We need to collect UX designs in order to make the development team develop what is required and so on. And in the product owner was no no why you need to exercise just Just building functionality and later on, we can take care of the designs. And that is not how the project works. And at the end of the day after realizing that the product owners are very experts on what they do in terms of processes, they hire a vendor to teach them how to do agile projects. Okay, so we were were working with this client for about more than three years, and they have just hiring someone to teach them how to do agile projects. And, and that’s fine, right? Once you realize that you have a problem that you have a gap in your team, then you need to fix it as up Right, right. Right, right. So ideally, all everyone is involved in the john methodology, agile thinking bank mindset and so on. But this is not happening in the real life, right? So you need to adjust. So what happens if you don’t have a business analyst in place in that kind of projects? Then the project is a completely flat failure, right because the product owner doesn’t know how to implement how the implementation works. And they’re more focused focusing on other things that the technical team doesn’t care. Okay, so that is how the structure I recommend fits into real life projects. If you are doing for example, a project from the very beginning, it’s a initiation and you as a product Product Manager, very experienced project managers, you know that you will require product owners who actually understand the methodology who can help the technical team to understand what needs to be done and also know about the business processes and understand the business needs and strategy then it’s totally fine. That is a perfect case. However, it is not happening on every case. So, from my perspective in what what you have mentioned coming up, taking up what you have mentioned, it is true that the product owner should candles are a different set of activities that might make you think that the business analyst is not required. However, not all the projects work the same. And you need to adjust and bring the more expert people to fulfill those gaps. And if it is a business analyst, let it be, you need to have business analysts on your project, you cannot just say we don’t need it because the metallurgy says we don’t need it. Right? So do you need to adjust your structure in order to be more productive and more efficient?

Vlad G 35:28
Thank you, that makes sense. And thank you for bringing this as a real life example. That’s, that’s always that’s always interesting to hear. So I’m building up on that. Just wondering. So, bH and product owners are kind of in the mix of it, you know, in the middle of it all, you know, fighting on the front lines or or working on the fault lies and making things happen. So where does that Product Manager in this hierarchy or in this structure come in, what? What is it that that they do and how they interact with stakeholders with maybe development teams? Or maybe they don’t direct the development teams in your, in your schema? How do they what do they do? What do they provide what kind of value they add to this whole process?

Unknown Speaker 36:26
Okay, so the product manager has two type of activities, I mean daily activities. One is working with the project team, and second type of activities working with external stakeholders. Talking about the inbound activities with the project team, it’s setting a vision, of course at the beginning setting, what is the purpose of the project of the product? Being able to understand what are the market needs, the business needs and be able to create a strategy okay. Create the product planning. That’s why we have the product roadmap as an excellent tool to visualize what is the plan, right is to to make sure that everyone understands what’s the mission strategy and the plan and the business goals. Of course, also, make sure that the implementation goes as expected and the end product is what was required right is what what the people expected and provide some perspective on the release plan right on how the product is going to be launched. In the other hand, we have external activities, which are, differentiate the competition, understand the market make some benchmarking and marketing research. Of course, of course understand what are, for example, the analytics that the product will require in order to extract data provides more information in how how the business goes. And this analytics can be for example, rotation of customers, customers, customers engagement by customers. And this is something that this was one feedback that we received from my last talk is like to differentiate between between customers from the business perspective and customers from the market, right like end users, people who actually use your product. So I’m talking about the people who use your product. So how to engage them, how to retain them how to, you can log in, for example, if you are going to purchase something in online shopping, for example, Amazon and jewel maker, you create a login and you start navigating, but you don’t purchase anything and you just log out and never go back to the, to the website again. Then it’s called. Client turn, right? It’s people who is not willing to purchase something from your website. And those kind of KPIs So that kind of information, the product manager needs to understand in order to create proper strategy right or change the strategy. If something is not going finding the business. If the profitability profile of profitability is not the one that the business expected, then something is wrong with the strategy. And in those kind of things, it’s what the product manager is able to understand rather than other rolling the project, right? So the product owner, it’s its execution, the business analyst is communication and in breaching their teams don’t seem understanding but whatever happens outside outside the product needs to be understanding, understand, understand it by the product manager. So that is the difference in here is where the product managers add value to the business and add value to the project team.

Vlad G 39:54
Makes sense and reminds me of a thing that I’ve heard. It was Before I joined the company, but there were stories about that as one of my previous jobs where key was developing a small business solution and they understood everybody understood small business differently. So the product manager didn’t explain what small business means. So the VA of that product team took it upon themselves to figure out what the small businesses and it was, it was a point of sale solution. They decided that small business means they only have 20 products to sell really not more. And it was really funny when they started sell try they tried to sell this or try to demo this to a real small business and real small business operates you know, at least 100 different SKUs if you think about it, if you even if you’re selling I don’t know 20 phone covers or or 20 dresses, you still have different sizes, different colors, different different models of the dress and so you will end up with more than 100 SKUs. And the system wouldn’t accept more than that. And it was it was really, you know, an eye opener. But, you know, that’s kind of kind of a strategy. So kind of a strategy. I don’t know if it’s a strategy flop or execution flop. But, you know, it was definitely a misalignment between the product management and, and the agile team that was working on it. So it’s interesting. So you’ve mentioned again, during the, during the conversation we had you mentioned that product managers also provide thought leadership, and I think it’s, it’s aligns with defining vision and strategy of the product. If you can elaborate more on that they’ll be great. What do you what do you mean what do you understand as a sort of leadership of product managers?

Unknown Speaker 41:55
Yeah, right, like so. If we understand that the product manager is Someone who actually does not have direct reports, right? It’s kind of our role who is navigating across all teams and giving orders and you need to do this and you need to do that, then we cannot call it a manager, right because doesn’t have a direct authority over people. So, they provide information for executives about the resources, what they need to the engineers, what they need to do designers how the product should look like to customers, how to use the product in how to envision it. However, all these actions required kind of leadership or top leadership right to in order to inspire others and have empathy with others to make them follow you and that is the big difference between normal leader I have I have explained these in previous conferences about the difference between a man a year in the leader And everybody understands that right like their manager, give orders and makes the team work the way you expected and the other side that the leader votes on other shoes and say, follow me and I will be the first in the front line and so on. But the difference between lead normally there and thoughtfully there is that actually the top leader has followers and I’m not talking about influencers, because influencers are different, right the followers are people who doesn’t have direct reporting structure on you, but anyway, they follow you, they trusting you and you inspire them to do the things right. So, coming into that concept, right. The product managers constantly communicate across organizational functions, okay. And that is that is one important If we talk about responsibilities of the product manager, which is being a central central half of communication, right, so if technical team has a question should go to the product manager, if executives or business team has or business development, especially if they have a concern or something like that they should go to the product manager. So it’s like consolidating all the dots in being able to attack different or to tackle different perspective about the product. Okay. And also these are the top leaders create a strategy right that Okay, so this is the game plan. And when we say the game plan, we always really make a relation with the sports right? So you can have a different different often leaders on your sport team. However, there should be one who actually provides the direction. What is the strategy to beat the other team right to beat the competition or to beat the market needs. So here’s, here’s where the thought leader comes into play in order to provide more insights in order to be the main the main person who people wants to follow and that is not easy, right? So, product managers need to develop a lot of skill sets. In order to become a top leaders, you can have a product managers buy the role, but we need to understand that is not a role or a position it’s actually action, right how the product managers action to different challenges and how the people react to those challenges to follow the leader. So that’s why I used to make trainings or or to teach other people in my organization in how to engage with others how to be more kind, and have empathy with others in order to make them be a better leaders. Not Normally, there’s more leaders that can provide business value at the same time they care about others. Okay,

Vlad G 46:07
that that is interesting. And you maybe think so when you were talking about product managers not having direct reports and spending across different organizational parts. He kind of reminded me of early days, when I was just learning to become a project manager when the matrix organization so same, they use the same words to describe it, like project manager doesn’t have direct reports, but it has influence over things being done and all that. But project manager definitely does not create a game plan. He would support it, he would make sure it gets done. He would make sure that communication happens but definitely not creating one. So I, I’m not gonna say I’m not gonna say they’re similar. They’re not There’s just certain overlap, it definitely is definitely there, but not there. And I agree with you, I guess it’s kind of one of the things that I like about product management is that you don’t have direct reports and you get to manage without the authority because its sheer power of the influence in a good way. sheer power, the influence of your thought leadership and your product and your the benefits that that product brings for the organization or for the customers or for the stakeholders that empowers you to move, move things forward and get them done. That was really cool. Thank you so much. This is this is really interesting, this really interesting approach to product management of I haven’t heard that in a while. So thank you for that. So as we as we approach we’re getting slightly closer to the end of the episode. I’d like to ask Start asking my cliche my canned questions that I usually ask them the episodes. How did this new norm, new ways of working from home you know, not being able to communicate with people face to face. And in my experience face to face communications was most huge was paramount to getting things done as a product manager. So how does this new normal or new pandemic and post pandemic world is affecting your ways of working your responsibilities, your ways of getting things done?

Alberto S 48:38
I can give you the real life example for myself and then I will talk more about what I hear from people around me. First of all, in my experience, it is not as good as being in the office. I’m not saying that like I love the office and now commute times but, but when you are face to face to someone, you can see their expressions you can see their faces you can see their body language right if you say something to someone you can immediately see if they agree or disagree with you. And then you can change the perspective of the talk or maybe change the words and something like that right. So it is more you can use a different set of skills like charisma and engagement with people and collaborate with each other in a more efficient way in terms of how you work with them. Okay? The other side working remotely, having 100% communication and collaboration through tools like Microsoft Teams or skivvy or, or whatever tool like zoom. Ease Okay, however, when you’re in front of a camera, You don’t always use your expressions, right? And you can be able to use and you can be all happy. But that’s not what you mean to be, right? This you’re hiding your emotions. That way you don’t have to waste communication like I say something and I and I have immediate response from you. Right. So that is one of the challenges I see we are facing right now and in by what people are saying about this is that the communication is being a powerful tool to get things done. However, you don’t you don’t get the feedback from the other person. And, and of course, the number of meetings in the North America of calls is increasing the metric dramatically. Sometimes, you don’t have time to do your work because you have a bunch of meetings in the morning and once you finalize them, it’s lunchtime. You have to grab some sandwich or something that doesn’t then go back to work to finalize what you’re doing. That’s different when you are in an office and you just have five minutes talk and everybody understands what you’re saying. And let’s back to work, okay? Or do you just turn your chair and say, Hey, I need you, they need this from you. Okay? And you will, they will send you one hour, two hours. And now you need to schedule a meeting and the meeting cannot be immediately because someone else is in another call and you need to get maybe by tomorrow, or two days after tomorrow. And that is not a good approach because he’s getting things delayed. right because of people availability. That’s, that’s my experience. What I hear from others is that it’s being more productive working from home because you can mix your time and create our work life balance. Because you are with your family. You can have lunch together. You can help kids with the school homework at the same time you’re attending your college conference calls and do your, your daily work. And, for example, if you normally finalize your work at 6pm in the afternoon and you just close your laptop and go home, and now you keep your laptop open and probably you have some free time you just go back to work and finalize whatever you were doing. So you become sometimes more productive in case you organize your time. Well, and one of the recommendations that we receive in one of the trainings is to if you need time to do your work, your personal work, you need to block your your calendar like two or three hours and avoid having meetings in these in this time slot. And because people say oh, do we need our origin meeting? Yeah, we I have another origin work to do. So please get in some open slot I have on my calendar. So this is kind of a recommendation in order to be more efficient on time management. That is something that is a new skill that people need to understand. And it’s to learn, which is a time management in order to be more productive and effective working from home.

Vlad G 53:09
Interesting. Thank you for your your perspective, I I do agree that there’s a lot of there’s a lot more meetings, because you can’t really schedule a five minute conversation that you may otherwise have with someone in the corridor or someone in the cafeteria. But now you have to schedule a 30 minute meeting. Pretty formal, I’ll invite a bunch of people that otherwise wouldn’t need to be there. And things in timelines are extending things or not are getting done. So you need more time to get them done. And, yeah, it gets a little out of hands at some point, I agree with that. On the positive side, it’s IE, easier to balance, work life. But not always. I see a lot of people working way more away over time. I see people working over the weekends, because they didn’t get enough of work time during the week because they were in all these meetings that otherwise would not have happened. So, I’ve seen I’ve seen a lot of trainings, I’ve seen a lot of articles. I’ve seen a lot of materials online, about how to maintain your work life balance. Guess what, they don’t work because because one of the one of the reasons why all this happens is because you’re not working in the vacuum. And most of these articles are written by people who have maybe a couple of years experience in the workforce, they are pretty low on the hierarchy of the organization and the higher you get more the more specificities you have. And more people you rely on to get things done because you delegate more things, or you rely on the inputs from other departments to get things done. And that’s where all those meetings come in. And that’s where all these, I don’t want to call them delays. But these time extensions occur. So, okay, I need, you know, two people from two different or two different parts of organization to be on my meeting to make a decision, but they are stuck in other meetings. So instead of having conversation that follows up by email that overall should take 15 minutes. Now I have to schedule a 30 minute meeting a week from now because that’s the only time on their schedules when they’re both available. So that’s the reality. That’s where we are right now.

Unknown Speaker 55:45
I totally agree with you. And that is something that again, there are a lot of trainings and online courses, as you mentioned, but it is done by people who actually are trainers, right? They don’t do they see we do we do so they promote their recommendations and that is totally fine. However, in the real life, we need to have a custom made time management. Because not everybody works the same. I see a lot of, for example, developers who say no, I work perfectly in the overnight. Because that time I am alone, I have time to think I don’t have any meeting schedule. I can do my work perfectly. Right. However, there is people who say no, I need time to sleep, right? Like I cannot do that in the overnight or in the weekends. And that is that is why I say like, we need a custom made time management, if not all recommendations work for everybody. But actually, we we need to create our own time management, we cannot do like a step by step type of thing. Instead of that we should accommodate our time scale. There are people who, for example, doesn’t have kids, right? So they have the whole day for themselves. You know To make their work and so on, there are people who have skates in, which is in my case, and I and I need to split my time with my kids with my job meetings, wife and some other stuff. So it’s pretty cool to know how people it’s doing the things around and understand that there are different needs for different kinds of people in situations. Right. And good. That’s a good talking point. That’s probably we can spend another hour talking about time management. But

Vlad G 57:35
let’s just say I sense a market need for another solution for that maybe AI driven time management tool that takes into account all your responsibilities, family, social, work responsibilities all together and aligns that with other people. So it’s kind of like a crowdsourcing. I don’t know you ever seen ways it’s an app that Crowd sources, traffic and road conditions. So you know you as you drive, the app collects the information about you driving, you know, what speed are you going at which direction you taking. You can report road closures, you know, the speed cameras, police on the road or accidents on the road or even the road surface imperfections like there’s a hole, or there’s a damage to the road. And based on all that data, the app redirects other drivers to a safer or better or more efficient route. So I feel like we need something similar that would guide us throughout our days based on crowdsourcing information, how people spend their time and how this whole thing works. Yeah, and

Unknown Speaker 58:51
there’s a funny experience with some of my colleagues. I remember around six years ago we work in a project with the Very tough manager, who has kind of micromanager, who likes all the things on point, everything is kettle, no delays and so on. So he wants all the people to be at nine o’clock in the office. And there was one colleague who always arrives at 910. And there was a concern from the management perspective, you say, oh, you’re running 10 minutes out of time. So you’re missing the daily standup or whatever, meetings get a little day off. And he can makes a very good point because he said, Okay, so in order to arrive at nine, I leave to be my home at eight. So it is one hour commute time. But if I leave my home 10 minutes before nine, I arrive at 910. So in terms of productivity in terms of efficiency, it is better for me to leave the home at 10 minutes before nine because I spent only 20 minutes driving stead of spending one hour driving. So he made a very good point. And I think management and have kind of levels of authority should understand that. That is one of the people concerns more in terms of going to the office like they can skill, their time management in a better way or the way they think it’s better for the cover up a good work life balance. So, based on that experience, I was kind of I had the same concern for good people who always arrive late but now I understand that are different situations that make people behave in a different way.

Vlad G 1:00:42
Yeah, that’s literally exactly hundred percent of my story. I in at least two jobs. I had Express Boss, I was taking Express boss that runs on schedule on a very specific schedule and one boss would take me probably about an hour and a Change to get to work and I would arrive half an hour early, and just space out and nobody’s pacing and nobody’s paying me for that. Or I would arrive seven minutes later than the start time and it takes me 40 minutes to get there. And you know, I would start work and manager insisted exactly the same same thing, measurement system that I would not be late by seven minutes. And I said, like, literally, what’s the rationale there. And another one was the same thing with what I was driving about hour and a half. It was hour and a half versus 55 minutes difference and same same thing. The arrival, the arrival time was probably like 10 1015 minute difference. So I hear you. Alright, um, we’re moving on. And this is the last question that actually last question for me. But this is your chance to ask me a question. And my question is, do you have any questions for me as we always asked, On this on the show, are there any questions for the host? Or hosts? If we have more than one, this is your chance to kind of turn the tables and ask me a question or two. Again, a reminder, please, let’s not boil the ocean here. Let’s focus on things that we discuss in product management and something that I can answer within the realms of this of this episode.

Alberto S 1:02:24
Okay, so it’s now my time.

Unknown Speaker 1:02:31
So the first question I would ask you is because based on my experience doing assessments at a time and interviews to external people, I always ask how people solve problems in communication in different understandings across the stakeholders. Let’s put a very basic example. So you have a product manager who has a different vision and his strategy than the product owner. And we already talked about differentiation of the roles and responsibilities But according to do two different books in I will provide an example. For example, the Bible says that the last call is from the product owner. And it is not always right. Because we have a stakeholders executives above the product owner who says I’m paying for the product. So my decision has to have more power, right or more or more weight than the product owner. So in your experience, what do you think should be the best approach to handle this kind of difference perspective or opinions across the stakeholders? Who has the last call?

Vlad G 1:03:35
As a great question? I, if I’m not mistaken, I just answered a couple of days ago in a real life situation. And my answer is always, this has to be a product manager because even though we do say that product manager represents business to the rest of the delivery, or the product manager is way more impartial than anybody Else, because if you think about it, you have requests or or feature requests or epic requests or or capability requests coming from all over the place, you have requests from sales, because they feel that’s what market wants. They have requested, then the requests from stakeholders come in from management, upper management, C level executives wants to see, you know, dashboards really cool, but that’s not what the market wants, but they want it and they’re paying for it. So their voices must be heard. Then the requests are. another level of request is from the technical team from from the implementation team from the delivery team. They saying, Hey, we need to refactor this, or we need to change the functionality here because it’s not scalable, or we may see issues, you know, six months to a year down the road. And it’s the job of the product manager to kind of juggle all these requests and ties them with all the involved parties and understand what the impact is, are those reversible or irreversible, irreversible decisions? Let’s say we prioritize the request from the market first. And we started implementing it. But we quickly realized that we should have listened to the technical team because, you know, there are certain technical limitations and had we listened to the technical team and had we implemented their requests First, we would have been able to do two more things in a better way. So we can reverse the decision reprioritize and do things. On the other hand, if there’s there’s an example of irreversible decision for example, we decided not to listen to the sales team, and not implement the features that they were demanding. And guess what 50% of our audience 50% of our users are no dropping our application or dropping our product and switching to a competitor’s because ours that Doesn’t have critical feature that is now absolutely a must have and in the market good example in recent events is encryption and security in zoom. Once you know word got out that there there’s a security concerns. You have bands from government organizations, a lot of educational institution jobs zoom for other solutions. Get because somebody didn’t listen, probably somebody in the product development didn’t listen to sales folks or or market analysis that said, Hey, this is really important. So Product Manager is the one who’s tasked to this and they are they are impartial, impartial to all these other organizations, and it’s really down to them to make that decision. I guess that’s that would be my answer.

Unknown Speaker 1:06:49
And I, I agree with you. And one of the reasons that I also mentioned during this this stock I have with you is that sometimes product owners are more Focus on business processes rather than the overall picture right let lay a full landscape of what is going on in the market what is going on in the trending of technologies and they have a better word to say what needs to be changed or what is the strategy that we need to follow. So I totally agree with you. And my last question and again based on the real life cases and this is not kind of a step by step answer I would like to hear another perspective from from others is like I have a friend who runs a retail store so it’s more kind of a fashion boutique for for women and he has a very good product so he’s now he had to hire an architect to create a boutique looking very good to hire people who actually is very charismatic to engage people who is going to the store and have a very good dressing rooms and have perks like coffee and Why and for people who is just arriving and have a very good purchasing experience in the store. However, nowadays he is struggling because people he’s not going to the store based on the current situation on the COVID-19. So I propose him to together help him to create a Facebook marketplace, okay. And I asked him to hire a community manager because I am not a community manager and I cannot help with that. I am not a designer or marketing specialist, but he needs to hire someone. And he needs to create a very good photographs of their products and to create a very good branding and marketing marketing strategy in order to reach out to people who is purchasing online. He’s not very technical guy and he’s concerning about the options he has because he will not be A full website, right? He can use whatever tools there are in the market. So I see this as a strategy change, right change some priorities. So he has already an architect and now a lot of people working on their on their physical store, but he needs to stop that in mobile to more online experience for customers. So as a product manager, how would you influence this guy? And what are your recommendations to him in order to change his mindset to change the priority and move towards a better strategy for his business?

Vlad G 1:09:40
I guess this is more of a psychology question rather than specifically or purely product management, but it’s it’s a way how you sell the idea to a person and yes, I would go for number of examples. Hey, look, all the museums are doing virtual tours. Hey, target is closing 200 plus stores all over the country. Because, you know, shoppers are not going to the store. Everybody everybody’s switching over to the mobile or online experience because of the situation. It’s not the question whether you like it or not. And I think that’s one of the main things that needs to be communicated. It’s not the question whether people like this, or they don’t like this, whether they agree with this or not, it’s the way the market moves. And that’s what knew they need to account for it. If they want to preserve the, you know, face to face the real life experience versus virtual, that’s fine. They can they can keep it they can do it as if nobody’s taking that away. But the focus should be on the online experience, because that’s where everybody is. It’s sort of like back in the day eBay was taking over and if you’re we’re not selling on ebay you are selling the same way like right now Amazon is selling everything and if you’re not selling on Amazon you’re not selling that Facebook marketplace maybe I’m not that thrilled with it and I know a lot of people look at it they kind of tap you know try try try using it but they don’t and they fall back on on the regular e commerce or retail shopping experience but ultimately it’s kind of innovate or die situation if you’re not online you don’t exist and and it’s pretty simple message that you know with enough examples and enough convincing will will be accepted by everyone I have some really good real life examples where the person who was resisting I anything online, any kind of online interaction you just insisting on face to face interaction with your customer. Once the pandemic hit, switched almost overnight to a full online experience, they closed their office, they don’t see clients in person anymore. And guess what their business is booming, because nobody really needed that in that line of work. And they just happily working from, you know, wherever the remote offices, they’re happily serving their customers from whatever the luxury location they’re in. And nothing really changed except they don’t have to pay for expensive office. And I think these examples and you can really put an ROI on that if you want. Like, hey, if you know if you reduce the hours then you can save on a retail. So you can expand your online presence. You can expand your online portfolio, you can expand your online presence. Yeah, I mean, you can even do the virtual tour inside your store and create an interactive experience. there so there are multiple ways of doing this. I think it’s a it’s a solvable problem with enough convincing enough communication.

Alberto S 1:13:08
Yeah, yeah I totally agree. And what I what I told to my friend was use the architect to create renders of the store, create a bag like you said that the museums are doing to create a virtual tour. And use use a the existing personnel on the store to use for example coupons for discounts to their relatives or friends and create a marketing campaigns using them. And of course, instead of firing them just create a more engagement with them and people are online. And of course it will get commission in for the number of sales they do online. And in and also they can create, for example, good photos of the products and make like a beard oil. built all experience for the customers who can actually see the sizes and the clothes that they’re purchasing on. So, yeah, so I agree with you that is more like a psychological to convince people that is one of the skill sets that the product managers should have when they’re talking to executives and technical teams. So they used to have communication, empathy and putting on other shoes to understand what their their thoughts and convince them like to influence others on what is the right path that needs to be followed.

Vlad G 1:14:34
Yep. Yep, I agree. It’s it’s all it’s all in there. It’s only communication. All right. Thank you very much. These were released in questions. I appreciate that. Thank you. This has been a really great episode. So Alberto, thank you very much for being here.

Alberto S 1:14:49
Thank you for inviting me and for all your your audience. Hope you enjoyed this this podcast this episode. And looking forward for here. hearing from other people to share their insights experiences in if you want to share my email. Let me just tell it to everyone it’s Alberto A, B, E, RT o underscore, s n. s as in Sandra, and us in number one are@hotmail.com. So if you have questions or instances you need to do want to share, feel free to reach me out and we’ll be glad to to answer it or to hear from you. Thank you very much blood for this amazing experience from the podcast was my first time on a podcast so hope people enjoy it.

Vlad G 1:15:42
Yes, I hope you did. If you yes thank you for sharing your email. That’s great. You can connect with Alberta directly. Or you can as always connect through our website wiig Robbins comm slash podcast or ask a lot as big Robin does calm feel free to ask any questions or kind of directly with Alberto or connect through us your way doesn’t matter as long as we get your feedback in. And thank you very much. This has been great episode. Thank you, Roberto. Thanks everyone.

You’ve been listening to the real world product management and I’ll be your host Vlad Grubman. Until next time!

Real World Product Management – Episode 13

May 27, 2020 Vlad Grubman

This is another solo episode where I talk about some updates, ask for feedback (yes, please – send me your feedback!) and walk through Productization Canvas – a little worksheet that should help with productizing an existing solution. Also, some updates on the whole podcast thing and questions for the audience.

As promised – I am making my Productization Canvas available for download. Please enter your name and email so I could send you the link. I promise to only use your email to contact you about this podcast, never to sell it to anyone or send spam.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Real World product management Episode 13. Yes, the dreaded 13. Anyway, because this is morial day, it’s a holiday in United States. For those of you who are not from United States, we don’t know. So a lot of people are not available. I decided not to pursue any specific people to interview and decided to do the whole thing by myself. So, if that’s that’s your thing, if you like me to go on about things I’m doing and things working on please say if you’re more interested in other people talking on my podcast, feel free to tune out and skip to the next episode. Hopefully it will be available by the time you’re To get to this one, I have a little reminder, a little PowerPoint presentation in front of me. I’m not going to share that, but I will share some of the materials that I’m going to be talking about. It’s, Well, I’ll tell you later what it is. So what are we going to talk about today? What is the topic for today there actually, if you I’d like to cover since this is a special solo episode, I’m going to talk about a little bit about what I’m up to things I’m working on. I’m going to talk about promoting product management to clients about product ization as in how to take things you have already developed to the market. And because I’m working on about six or seven streams currently doing Same thing with different services, taking them to the market, I figured this would be interesting to discuss and most of them This is the kind of thing this is the kind of the approach. I’d love to hear some of the feedback to. So if you’re you agree, let me know if you disagree, definitely let me know. I might be missing something I might be talking about things that you know, I have no idea what I’m talking about. So by all means, let me know this is this is a Costco. This is a, an episode where I am specifically asking for feedback specifically asking to let me know about things I’m doing and your opinion, your audience opinion on things that I’m working on. So let’s move on. Part number one. What am I up to? One of the things that I’m tasked right now the company I work for is promoting product Management competency or product mindset to the customers? What does that really mean? A lot of customers that we work with are I technically I am in a consulting business. So a lot of customers that we work with are not really product mindset oriented. They’re very projects program, budget, way of thinking they operate pride of with projects and programs. Think Okay, we have a several year to three years program that consists of multiple projects. You have a chunk of money allocated for those that three those three years based on how you know how you plan the first year, get your budget, you get your budget for the second year, third year, and at the end of the program at the end of three years. You see how well you did obviously not a good idea, as We in product management think? Well, some of our guests, some of my guests from the breeze episodes will tell you that’s the only way. If you remember the episode with a project manager from NASA, she is straight on point that you can’t really apply product management and I think she makes sense. But in the more sulfur in the social world in the more socially oriented projects, not necessarily fully self oriented, obviously, there are other things, but you definitely need to take into account

Vlad G 4:33
you know, wherever you can do better. By using product management approach or product mindset, then you should one of the one of the problems. One of the challenges is that no one wants to change and the current situation the world with the COVID-19 on them because everybody must change. I was browsing several online resources just today. And I noticed that a lot of people are not really happy with how their company handled the pandemic response. So, you know, people are working from home. That’s not, that’s not where they want to be, or people working from home, but that’s not where their bosses want them to be. So it’s really hard to kind of put a same measure or the same yardstick against all of that, and understand, what does the change really mean? There are software companies where bosses and I’m using BOCES collectively because this could be project managers team leads, could be CTO or CEO. They want they want boots on the ground, they want boots in the office. On the other hand, we all know we can agree that 99.9% of software development can be done from literally anywhere. I think if I’m not mistaken that some of the major Fang level companies have announced that They will be working from home forever, unless you really want to be in the office or unless you want to come to the office, then by all means, use the space but you if you don’t have to be there, don’t be there. And that’s really interesting, really interesting concept, really interesting approach where basically everybody understands that there is no requirement for developers testers to work from home door from office, they can work from home has been done. We’ve been working from home for almost two months now it’s you know, the case has been proven so far. So it would be interesting to see how things will turn out. Another another problem there is that companies because of the whole economy, issues that the company has lost visibility to their revenue. So what this means for those who are smarter than me already know what that means. I just learned about it. Last week, I’m not that smart. What that means is, companies cannot predict whether they’re going to make the money they were planning to make. In other words, we plan to say I’m a software company, I ship software, people subscribe or buy or whatever. And I project I’m gonna make, I don’t know, $10 million in q2, q3, q4, at this point, we don’t know, and they don’t know. And nobody knows whether we can make that $10 million or not. So I can’t really say, Hey, you know, I’m gonna have that $10 million in q3 q4, so I’ll have the money to pay developers I have the money to pay for the office, I’ll have the money to pay for literally everything. So as companies don’t have that visibility to the revenue as they have no idea whether they will make those money or not. They can’t really say, Oh, hey, we’re going to keep this size of the workforce or we’re going to keep these projects running. And that’s where product management kind of comes in. And you say, hey, Product Manager, product mindset allows you greater flexibility. You don’t have to project for the next year, two years, three years. What you can do is you can start the cycle of value delivery, lean agile, however you want to call it, and build value incrementally, in short bursts, rather than plant a hat for a year, two years, three years of a program or, you know, six months 12 months project and, and and not really having flexibility to to do anything. Truly, there’s no other there’s no other way. And in that says, I think my belief is that product mindset product management mindset allows you that greater flexibility, agility and Lean approach, if you know you don’t make the revenue target if you know you don’t have the money to support a large development team, you can support a smaller team. But at the same time, you can still continuously deliver chunks of value just not the value, just not the amount of value you were thinking about initially, but it still allows you greater flexibility, keeps you afloat keeps you in the business keeps you in in a shipping

results, shipping shipping software mode, or creating value mode. So that’s really interesting challenge to talk to people who were operating projects programs mindset and try to convince them that those days are over, at least for the next several years. Those days are over. And, you know, the there’s a better approach. There’s more lean more agile in the true agile, not agile, agile sense awards. And it’s really interesting to see how they react. So we’ll see how that turns out. The next topic that I wanted to talk about is product ization. I’m going to drop, I’m going to drop the canvas that I’m going to be discussing in to show notes. It’s going to be available on my website. It’s a v1. It’s not, it’s not an alpha or beta. I’ve drove with, I think about six themes by now. six different services that I am productizing in our company, and I think it works. I’m not sure if there’s a better way of doing this. If there is a better way to approach this. If there is a more efficient way. Maybe I’m missing something. So by all means, take a look. Have a listen to the next segment and let me know whether that makes sense or not. If you want to email it to me Your feedback if you want to email me, as asked a lot at V grubman.com. Obviously, there’s a forum in the website, obviously, you can contact me through Facebook, by the way, question, I don’t have a Twitter and all the podcasts that I’ve know that I know of, do have a Twitter. So question to the audience. Does it make sense at this day and age to start the Twitter? I’m going to be probably I’m going to have like three and a half followers and other 15 will be bots. So I don’t know. Should I should I not drop me a note? Let me know. I’m really curious if you guys are following Twitter for things like this or you know, the regular things that I where I spread the podcasts are sufficient. So back to product ization canvas. Product ization, is when you have a solution. you’ve developed the solution and you realize that you can reuse it, you can package it, sell it as the Right. So you need to come up with a way to package it to create that packaging around the offering around the service in my particular cases to service. So you need to create that packaging around the service and kind of tell it to yourself, tell, explain to yourself, what is it that makes it a product? Is it a product, this is make sense to develop it as a product, maybe, you know, as good as a service and you don’t need to prioritize it. And you just need to, you know, service is good enough as a service. So I took a product vision canvas that all of you know, if you don’t, well, what are you doing here, go check it out. Product vision Canvas, and I’ve modified it to go specifically to be able to talk about existing services. that have been exposed to clients that we’ve already talked to clients about. We’ve already kind of been there done that. But they’re not back to this product. It’s a bit Nish use use case. But it’s still pretty widely used because I find myself repeating the same, same old dodge saying, Hey, I don’t want to be solution looking for a problem. In many cases, we have a solution to a problem as a single problem without realization that there are other people there are other companies there are other potential customers have the same problem. And we can repackage this solution as a product and sell it to them offer it to them as a product because they have pretty similar use case. So it’s easily that that solution is easily productized. If that’s the word So I approached this product ization canvas. It was a bit of a trial and error initially, it turns out it did work. It turns out it helped us create this unified approach us and the teams I’m working with. As a consulting Product Manager. I don’t know what that means. Honestly, I’m not a part of the team.

I’m just kind of helping teams themselves realize whether their services a product, and if other any people inside the team who can take on at least junior level, product management responsibilities and run to them. We’re not there yet. We’re don’t have any outcomes. If you guys are interested, let me know. I’ll definitely try to report whatever, you know, whatever is not under NDA, I’ll be able to speak to that. Obviously, you know if it’s trade secret, sorry, tough luck. I can’t tell you but as a jeweler, nary approach to product ization. I believe this works, because again, I’ve tested it, but I haven’t tested it, you know, hundred times. That’s why I’m talking about this. That’s why I’m presenting is this with more details than I had initially thought to present it. And then hey, let me know, if you think that will work. If you tried it in your organization. And you say you succeeded, definitely let me know. I’d love to have you as a guest on a podcast to tell me about your success. If you tried and failed, I want you here even more to tell me where would I go wrong? And where can I fix this? Right? Maybe the whole thing is wrong, I don’t know. But again, I’ve tried it. We’ve done it. I think it works. So again, basis for this was product vision catalysts or kind of like half of the business. Lean does canvas. So a lot of things are very familiar. Hopefully, again, the purpose was not to create anything new but to adopt the existing approach. So you don’t have to learn new tricks. You can use existing tricks to help you with this over this journey offered station. So imagine, save several columns, or sell sections of a document here preparing the first section in your document would be target audience. And again, I’ll drop I’ll drop a visual in show notes on my website. Yes, it’s a way for you guys to drive traffic from A to B, the traffic to my website. Yeah, I need those, you know, a couple of people visiting. So first item on the list is target audience. We split the targeted audience into two sections, two distinct columns because we’re talking about Closer to the enterprise services and software, we divide them into customers and users. But it’s really, who will buy and who will use in. In our enterprise world, it really makes sense to divide it as such, because people who buy your software are not the people who are using the software. Realistically, if your CTO is buying you JIRA, he probably probably spent five minutes on it any given year. Why? Because he looks at the reports that project managers or product managers or team leads or department leads or program level leads generate for him, he doesn’t have time to dig into JIRA reports unless it’s a pretty small company. In that case, he’s probably hands on on more than just JIRA. So, who will buy and who will use the reason for that distinction is also to understand How to talk to these people and how to convince people who will buy it, that they also get something out of it every step of the way, we are answering the question, what’s in it for them? What’s in it for the client? Why should a client pay you money for your product? So people who buy who spend money on your product must get some kind of return on that or return on that investment, some kind of a benefit, they have to buy something tangible, that, you know, later on, they can report a Hey, this is what I paid for, and this is what I got. And people who will use it should definitely realize some benefit, because otherwise, why would they? Why would anybody buy it? Right? So if you’re buying JIRA, and that’s a good example. If you’re buying JIRA developers are using it testers are using the project managers are using product teams using it, but they’re not the ones who are paying for it. So that’s that’s the distinction. It will help us later, figure out a couple of others. things. So break down cool by who will use the thought goes that people who buy our budget owners,

people who use our actual engineers, developers, whatever, specifically to the services that I was working with, but obviously, whatever your product is maybe different. And again, this is more applicable to large scale operations, large scale companies, enterprise products and services might be different in the b2c world or might be different in a smaller scale world. I don’t know. Again, let me know. Write me an email. tag me on Facebook. Tell me what you guys think. Second section in this scandalous second big section, is needs. way we approach needs is really simple. We copy paste and that’s it. The exercise I went through with all the teams will copy paste our target audience. And for each entity, I don’t want to say the person or I don’t want to say the role because sometimes entity is HR, right your product serves HR, then head of HR will buy it and regular HR personnel will use it. So what are the issues that what are the needs of each person, head of HR, head of HR once better productivity. People in HR want not to login to 15 different systems and just use one to see everything they need to see as a really simplistic example. So we literally copy pasted the list of entities we’ve identified as a target audience, and we try to list what are their needs, and what are those knees which those knees are covered bar which of those knees are covered by a product. As an example, if we’re talking about HR and if we’re talking about regular HR personnel not wanting to log into multiple systems to understand what specific workers eligible for one of these hours, what is roles responsibilities with us experience, that would be the need, right save time on not logging into 15 different systems and potentially my product or services of the future will alleviate that will provide a single point of reference or single point of entry or, you know, single sign on or dashboard with multiple screens for multiple systems, whatever that is. And that’s how you would approach the needs section. You identify needs that this product is covering, you can’t really list all the needs this person has or this role has. CTO has many needs. head of HR has many needs. One of them is probably vacation. But that’s not what your product does. So you identify the needs that can be closed by a product. And you briefly talk about how that needs, ie how that needs, how a specific need is closed by your future product. Once you’ve done that exercise, once you’ve done with target audience and knees, there come another, the third section of this, that is different, and that’s a full transparency. That’s where we’ve got a lot of, I can’t say, problems or challenges, but it was challenging. That’s where our teams, the teams that I was working with, that’s where they got a little stalled. And we we’ve approached this and again, these are not the people who are very familiar with product mindset, so I had to give them a bit of a break by Round. But these are the people who’ve been building these services for a pretty long time. They know this service, they know what they’re working on. They just don’t know what how to approach this from the product perspective, they have don’t have the experience. And they need to catch up. And again, it’s it’s a pretty niche use case. But I’d still be very interested to hear what you people may think. So I approached it from three different quarters, three different angles, what I was selling, why should they pick us over somebody else? And what makes us unique? So what are we selling? it especially when we talk about service when we talked about service? And we talked about platforms, and we talked about solutions? Everybody was

first time around when I asked question, what are we selling? Oh, well, we’re selling the platform that allows us to do the service that will bring in the consultants and that’s why I stopped them. I said, Listen, you already named three things. You’re selling, you’re selling a platform, you’re selling a service, and then you’re selling consultants, consultants time. So what is it that you’re selling? You’re selling platform? Can we sell a platform separately? Can we sell a consultant separately without the platform? Would there be a value there? Can we sell just the, you know, the playbook, this is what you have to do. And you know, you’d do it. And and that sparked a lot of interesting conversations as to what actually makes the product, the product and each particular case, I believe, it’s very, very challenging to go through this specific share stage with people who know the solution but have no product background. That was that was the experience that had to go through. It’s very interesting. It’s very showing how people think about their product, how people think about their solution, how they think about making these into a product, because they still gonna own that and not this. It’s not it’s not a given that you as a product manager will step And then product manage the product. It could be that you’re there just to kind of get some traction, and then you step back and let them drive the show. So it’s really important to have these conversations about what is it that they’re selling? What is this service solution platform? Can they be sold separately? Can they be sold together? Should they be sold separately? Should they be sold together? Is there a specific value we can provide in each case? And if so, what that value is, and once you’ve identified those pieces, you can start asking questions, why should they? Why should Why should they buy from us? Why should clients buy this product from us and not from competition? And what are the unique advantages of this particular product? That’s where they usually shine that’s a team’s a shine, because they’ve developed it the probably, this this solution was built because all the other solutions were not solving a problem or if no, we’re not solving a problem sufficiently. So the off the shelf product didn’t work, and they had to build something else. And that’s, that’s, that’s kind of the beauty of this thing, right? That’s where you can say hey, so this is what makes us unique. So that should be, you know, front and center. How you how you talk about this. This also how, how this piece also helps to define a couple of other things down the road. And it really, really benefits from defining the needs first, obviously, because you know what the needs are, you know how your product closes those needs. So, you thought about, hey, these are the needs this is our product. This is why we’re so unique because this need is never covered by competition. This knees never covered period people still use I don’t know. pen and paper. So ultimately, you get you end up with the actual definition of a product, what the product is, what is it that we’re selling? Is it a service? Is it a playbook? Is it a consultancy? Is it a platform as a combination? For example, you don’t want to sell a platform separately, you don’t want to sell consultant separately. You only want to sell our platform and consultants times together. Because guess what, we just sell them the platform and just, you know, send them PDF with the instructions, nobody’s going to read it, then I’m never going to learn how to use the product of the platform properly. And it’s going to be a failure and you get a reputation that you plot problem that your platform doesn’t work. It works. It’s just they never they never figured out how to use it. So it’s it’s a good conversation to have to define the that actual product offering. Once you’ve passed that and begin with people who are less experienced With product mindset, it is challenged. Once you pass that, the next challenge is probably even harder. And here’s why the next section is called revenue sources. And it’s completely different from it’s completely different from their items on product vision, canvas or business lean canvas. And the reason why revenue sources are there, the reason why I introduced it, because these people

never really think how they’re going to sell it. And that’s what makes it a product, right? It’s a packaged solution package offering that can be sold multiple times. And these people who built these solutions, these platforms, these services, they focus on build, they never focus on sale. And it’s really, it’s really dangerous place to be in because you get to keep building for the sake of building and hope that you know If you build it, they will come guess what they don’t. And that’s the problem. And you as a product manager, as a newly bred Product Manager, you are the one responsible for sales, and you need to attract people, and you need to tell them how great your product is. And guess what? They will like it, there will be market for it because you already sold this solution. So there’s definitely a market for it. But when they come to you and ask you, so how much does it cost? How am I gonna pay for it? If I buy one? Is it gonna cost me the same as if I bought hundred? And you can say, Oh, you know, I’ll be back. I’ll have to think about it. Oh, let me ask someone else. You have to be ready to ask those questions. Reminds me one of the episodes back in the day when I ran my own business. I wasn’t sure how to name the price. was my first business. I didn’t know I was really I didn’t know what I was doing. And I was really, really struggled to say I was going to cost x or it’s going to cost why and here’s why. Because I never felt justified. So, this exercise revenue sources, how will they buy it? How much they will pay it? Who will be paying for it? And you know, what, what is the? What is the licensing model? What is the sales model for your product? It’s really, really hard for people who are only building but never sell it. And sometimes, they know sometimes they have an idea, sometimes they don’t and that’s where as a more senior as a more senior product managers and more senior person, I have to guide them. I have to ask them questions like, does your product scale easily can I sell one copy of whatever it is you’re selling as easy as 100? What does it cost me as a you know, product company to sell one versus selling 100 if you’re selling, again, let’s use JIRA. Right? Everybody knows you’re, if I’m selling 10 licenses, if I’m selling 1000 licenses, there’s specific costs behind it for the resources, consume them server storage, whatever else. So you can calculate you can project what your cost is going to be. So you can pretty much settle on the price. If you’re selling consultants time, it’s even harder because it’s easier to calculate but it’s harder to project because, you know, one one service Ron and and how the service runs may be completely different because depend depending again, what what are you servicing, right? If it’s one on one service, that’s great if it’s one too many service, like, what’s a good example? If as a consultant, he was If it’s a learning, if it’s a seminar, right, if it’s a class, if you’re doing one on one, obviously, it’s one thing if you’re doing one of one to 10 people, it scales if you don’t want to 100 it doesn’t. So you need to find the balance. And you need to teach that part that finding the balance to each newly bred product team. How would they tell when that threshold occurs? How would they tell when that thing happens? How would they tell when that particular jump occurs when they start offering individual licenses or subscriptions or, you know, one time payments where they start offering volume licenses, or enterprise license where you pay one fixed fee per year for example, when you know, use as much as you can. So that was that that’s a bit of a challenge to go through with With the team that never sold the product, but it’s also an interesting conversation, because you may get an update on what the real value is. And you may they may say, Yeah, well, we think our platform was really great, but it’s really, you know, everybody can build this platform in like couple of days. So it’s not really that hard to rebuild it. So we can’t really tell them that they’re paying for the platform, or we can just tell them, Hey, you know, pick up this

open source code and you can have your platform but we’re gonna teach you how to use it, or some such thing. So it’s really interesting conversation with the brand new product team as to how they think they’re gonna sell it. And the beauty of it is all these for all these four sections of the document. target audience needs, product differentiation and revenue sources will change the more you talk to the customer Then once you start going out and offering this to your potential customers, you’re using classic customers as an upsell, you will start seeing the questions the misunderstanding or not understanding or they not getting it and you need to tell them differently or they not buying you then you need to change your model. you need you need maybe you want to go from proceed license to per use license over time license, or maybe just you know, come up with the enterprise based on something enterprise license based on something so it’s all very flexible, and it’s never a final version. It’s always a work in progress. So you’re going to have to make sure you maintain this document, keep it updated. So once you fill out all four, this is where you start coming up with the product statement. sure there’s a better word for that. I like to call a product statement because it’s your catchphrase. It’s your 22nd elevator pitch elevator is moving faster now, if you remember, what’s the role of work from home, so it’s your 22nd elevator pitch that you should tell your client to make him say I want to know more, really older so it so you need to staff literally you need to stuff a problem statement, solution statement benefits in a couple of sentences that you can clearly articulate in 20 seconds while you’re in the elevator with someone else, obviously, probably it’s probably never happens. But it’s good to have. Because people’s attention span is really, really, really small, is good to have this catchphrase that will help you communicate the product value what kind of problems is solving how and what’s the what’s the value was sent in from the client through a really short burst of information, the short Better. There’s no i didn’t come up with a specific limit. 20 seconds is just you know, pulled out of thin air. But it’s good to have that one catch phrase. Because here’s here’s the kicker, because every product needs a one pager. And in my book, the best one pager consists of two pages. Yes, it’s almost a joke. So it’s a two page one pager catchphrases on the top. This is the, you know, the product name, and this is what it does. page one is a problem statement, an elaborate problem statement, a solution and the outcomes. So it’s really, you know, as is, this is what we have today. This is the future and this is how we’re going to get to it, but how we’re gonna get to it and a future swap. So same sales presentation, as As always, as before, as well, you know, as well, no love. But you really defining a journey. This is what we are today, we’re going to do this. And this is if you bright future that we’re going to arrive at if we only do the right thing. second page of this presentation is really simple. It’s your commercials. It’s your who’s paying for it. Who was the target audience? How are they going to pay for it? What is the sales model? Is it a volume licensing? Is it the first seed? Is it a per year? Is that an enterprise license? Whatever it is, and how how do I get it is that a download SAS? Whatever it is, and how it’s going to be supported? It’s very important to include a support model in broad strokes general terms on this one pager. Because it’s not if the cost of ownership is higher than the cost of the product, then you’re not doing the right commercials. You’re not showing the right thing on all your commercials page. So customers especially again, in the enterprise world, but in in regular commercial world as well, they care about not just the upfront costs, but what is going to cost me to own this thing. Do I need support? Do I need people to come in and tweak couple of things every now and then and am I gonna have Do I have to pay for that. So, support models really important to

show to be present on that second page, because it shows demonstrates that you stand behind the product and again, it goes without saying that some people think that they are by Yeah, I stay behind my product of course, but the problem is, you have to tell it, you have to say it, you have to tell the customer potential customer that you do, they are not going to assume anything and if if the if they were going to assume anything, they probably will. They will, if they will, if they’re gonna assume anything at all. They will assume that, no, there’s no support support is separate and you’re going to nickel and dime them. So having that information, right on the one pager is crucially important, because you’re basically speaking their language, you’re saying, Hey, I’m going to solve your problem. This is how I’m going to do this, what you’re going to get out of this solution, this is how much you’re going to pay. And this is how I’m going to support you as you’re using my solution as you’re using my product. And that’s when the customer feels like you’re taking care of them. So that’s, in short, what the product ization calculus is. for the third time, I think, I’ll put it in the show notes on my website. Feel free to get there, download it, take a look and give me a feedback. I really, really, really want to hear your feedback. I would greatly appreciate it. Last couple of things, I want to keep This episode is short again, this is a long weekend, short days. I wanted to keep this shorter than usual hour and a half. Again, because I’m the only one talking. So a couple of updates. The This podcast is now available on pretty much every platform out there. Apple, iTunes, Google podcasts, Spotify iHeartRadio. Except Pandora, I submitted this. I think on March 1, when I released the first episode and still haven’t heard back from them. It’s been two months and change almost three months. So I don’t really care about that anymore. feedback on quality of the podcast. So if you are willing to provide feedback, please do let me know through the whips website. The grabbing of comm slash podcast or through the email asked a lot of the gramma calm. I want to hear about the things Want to hear about how happy are you with the content? Do you feel like you’re getting something useful out of it? Do you feel you’re not getting something useful out of it? And would you like me to focus more on? If there is any problem with the quality of the podcast, some of the episodes are more really hard to record because of some technical difficulties. We either on my side or on the other party side, sometimes it’s unavoidable, and we just can’t resolve it in the short time that we have to record the episode. So let us know if there are any issues that make those episodes. listenable that you just, you know, turn it off in this guy saying, yeah, it might have been interesting if I could listen to it, but I can’t it just hurts my ears. definitely let me know what hurts your ears. If there’s anything I can do better. I will. One more thing. I started uploading these on YouTube. And this is my first video where you actually see my face. So I’m trying to experiment with different delivery platforms. All the previous episodes will be on YouTube in the form of visualization. So just my face, some kind of a thing on the screen and really cute waveforms with the same audio track you could have listened to. I’ll try to produce video episodes. So it’s basically I’m recording the audio. This is the microphone and recording the video and we’ll see how that works out. If you’re getting bored with my talking head, I’ll figure out a way to get to show you some slides. Maybe some funny videos make, you know funny faces. So this these these podcasts are not only useful in terms of the information and knowledge but also in some way shape or form entertaining and I’ve asked this before early on Episode I

want to reiterate because it’s kind of a interesting question for me. Should I do Twitter? I actually never had a Twitter I had Instagram websites. A couple of other things for my photography hobby, but I’ve never really had Twitter ever. Should I start now? Should I not? These are the announcements of the new episode that produced usually come out on LinkedIn, my website and I created the Facebook page. Yes, I am old fashioned. I got Facebook page out of them. Other than that, I really don’t put it anywhere else. It’s just you know, regular broadcast distribution through regular channels. Let me know if there are any other ways you’d like to get in touch that I’m missing. That would be really great if you could share that. Outside of that I really don’t have anything else. I hope you’ve enjoyed this episode. I hope you’ve enjoyed listening to me alone. If not Just wait for the next episode. We actually in our company, we are producing really great series of events and they’re open to the outside world. They’re called z days.

Vlad G 44:14
I don’t remember length, unfortunately. But you can Google it. Sure, and they are open to outside. It’s not internal company event is the beauty event for everyone. We have a lot of interesting speakers coming in. We have a lot of interesting questions. So hopefully by sometime next week was basically weekend. Next weekend, I would have a ton of new stuff, new information, probably a lot of new people to interview and bring a lot of new stuff to these episodes. So hopefully, this is a you know, this is gonna get better. Really, really hoping you guys can provide me with more feedback than I’ve seen so far.

You’ve been listening to the real world’s product management and I’ve been your host Vlad Grubman. Until next time!

Real World Product Management – Episode 12

May 27, 2020 Vlad Grubman

In this episode I am talking to Kaushal, a fellow product manager, and we are discussing multiple challenges while working with the remote teams, ways to address these challenges, and engage teams effectively.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Hello everyone. This is yet another episode of Real World product management. And I have another guest for you guys today is kachelle Vyas kachelle. Could you please introduce yourself? Tell us a bit more of what you did and what you’re doing right now.

Kaushal V 0:30
Hey, hi. So my name is Kaushal. I’m joining today from London which is where I’ve been living for the past few years. I am originally from India. Some of my interests include behavioral economics, international music and cinema, mindfulness and playing cricket. Professionally, I’m currently leading product growth and monetization at a company called pocket. pocket is a mobile banking product for the funnel. underserved and it comes with a contactless MasterCard. I’ve been at pocket for just about a couple of months. But over the over the past few years, I’ve been lucky to have led various successful products. My my first product management role was with a Kids TV app called hopster. I was the first product manager there. And you know, one of the highlights of my time there was was that during all the years that I was the product manager, the app was recognized with the best of 2014 best of 2015 investor 2016 titles on the on the Apple App Store, which was which was quite an interesting achievement for a small team based out of London. After that, I spent some time with a company called game sis who, who are an online gambling product company. I was looking after a product called called unicorn unicorn apps, which was a Large platform product. And my demo here was also interesting in that I work with a really large team of about 30 people and we were split into four different products, courts all working on the same product. After that, I spent some time in the in the b2b space, I was with a company called Global app testing who were a crowdsource QA testing company. And, you know, I was looking after the supply side of the product. And yeah, beyond that, I recently moved into FinTech with with a company called pocket. And today, I’m happy to share some of my experiences here with with lead.

Vlad G 2:45
Cold, thank you. And my apologies for mangling the name, I’m horrible. With names. You mentioned that with your first engagement, your first gig as a product manager. You work with you work with the kids app. And my understanding is that the user and the customer in best scenario are quite different the customer, usually the distinction that I make is the customer who is the one who’s paying for it, and the user is the one who ends up using it. So your customers and adult and your user is a kid. Yeah. Did you guys how did you guys approach that in the b2c market? I I see this constantly in my daily life with a b2b products and that it’s normal. So how did you guys tackle that? In the b2c space?

Kaushal 3:33
Yeah. So that’s a great question. I’ll start by just giving a very brief context of what the company tried to do. So So hopster is is UK top grossing and top ranked Kids TV app, the vision of the company was to make screen time smart for kids. traditional television is is you know, littered with ads and is quite popular. And, you know, hopster wanted to give parents a better option. And the way we would do that is by having some of the best video content from around the world and pair that with with various learning games and storybooks. Right. So that’s the context. Now, coming to your question. So, you know, in the, in the early days of hopster, the app experience was designed around a two to four year old and chances are that a kid that young is, is probably not able to read or not able to make sense out of language. And so so the app experience was was, you know, without any text without any sort of information, and it was it was designed as an intuitive experience. Now, as a result of that, what what had happened was, you know, The apps abandoned rate was quite high. And what that means is about half the users who installed the app from the store would not come to the app again. And that was that that was a waste of our acquisition money, basically. And, you know, what we realized was that the apps first time user is a parent. So it is a parent who would install the app from the store and do a sanity check before the parent would hand the app over to the kid. And because we did not do a good job of onboarding, the first time user, we were seeing a lot of them just abandoned the app. And so that is when we realized that you know, we we need a different approach. Yes. You know, the app. The apps main user is a kid. But a parent is an equally important user because it Is the parent who makes the decision to buy the app and and and, you know, hand it over to the child. So, our approach to this challenge was was around building a timeline. Right? So we would we would think about all the all the touch points with the user and the customer and sequence them. And, you know, so the first time user is a parent chances are the second user is a child. The third or the fourth touchpoint could again be a parent, you know if they are renewing a subscription or if you’re if the Wonder child to explore some new content. So what we did to alleviate the the abandoned rate problem was basically build an onboarding experience that would give parents the confidence to hand the app or to the child and You know, interviewed some parents around what, what they would like to see before they would trust the product with their child. And, you know, a few things came out like parents suggested that they wanted to understand what was free and what was paid in the product. So the product was was a freemium product. So some content was free, some was paid. And parents wanted to know how that worked. parents wanted to feel that you know, the offering is is is safe, it is curated it is it has the kind of certifications that kids apps should have. And so yeah, like the, you know, the onboarding experience was was around trust around giving them the confidence around explaining them what the offering was. And in addition to this, we you know, we we also wanted to Encourage the parent to create an account so that we could we could get them into our CRM. And this was simply a business sort of objective around this because, you know, if the parent is not engaged the first time around, chances are it is really hard to engage them second or the third time around. So yeah, like the onboarding experience was around, giving the parent the confidence that the product is safe for the child, and also encouraging them to create an account in the in the app.

Kaushal 8:32
So yeah, so I first prototype with onboarding, addressed both of these concerns. And we saw that the abandoned rate of the app came down significantly. So it was half. So earlier, the abandoned rate was at 50%. After the first iteration, it came down to somewhere around 30 or 25%. And then we kept iterating over it eventually. It came down to about 10%. Pretty so so this hopefully gives some idea about how we thought about, you know, the user and the customer being different. And designing the app experience based on who is engaging with the product at which touch point,

Vlad G 9:21
though is this is interesting, thank you think of sharing this is really cool, as you keep saying they’re different, but they’re both engaged. I wonder in, in the current terms, I don’t want to really bring up specific frameworks. But if you’ve, if you use any specific approaches, I don’t know empathy, maps, journey maps, personas, jobs to be done. Or you just kind of intuitively built that timeline with touch points.

Kaushal 9:49
Yeah, that’s an interesting one, I guess. I guess, um, you know, at various points in my time at hopster. We’ve we’ve used Like some of the frameworks that you mentioned, jobs to be done is certainly one of the more popular frameworks around around product, building the product for the right user and, you know, helping them helping them do the right job with the product. I’m trying to think if there is like a singular framework that could, that I could I could talk about here, but yeah, like from what what I remember it was it was like a mix of of using user personas. jobs to be done. And, and the timeline based approach at at various points in time because, you know, like the onboarding, case study that I just talked about is is one of the many other use cases that we designed the product for. And for four different use cases, we would have you used different frameworks. So yeah, like there wasn’t like a one.

Kaushal 11:07
One size fits all framework that that we were using across

Vlad G 11:13
across all of the touchpoints across the whole product. Right. That’s, that’s, that’s, that’s a great answer. Thank you. It’s, it has to be the right tool for the right job. You can’t, you can’t really say yes, this is the only thing we’re using. Alright, cool. So, once you’ve left that company, you transition to game six, the online gambling platform. Yes. And I have I actually have a bunch of questions about this one. One is, I wonder, how did you approach platform as a product? How did the whole concept work? Was that an internal platform? Was that a platform you guys offer to the market? Can you elaborate on that a little more

Kaushal 12:01
Sure. So I’m again going to start by giving a little bit of context around around the company. So gives us is a is a leading international online gaming operator. It’s one of the one of the larger ones in the UK it operates brands such as virgin casino virgin games jackpot, Joy heart bingo and, and some others. I was a part of a department called player services and Play Services, as the name suggests, looked at everything outside of the games themselves. So placing bets or payment gateways or promotions or hybrid apps, load times registration, etc. Right. And the product I was looking after was called, called called unicorn unicorn apps and it was essentially a brand new platform for The company and what I mean by that is, is that, you know, the company had been around for a good 1015 years by then. And its main products were still a desktop first was still desktop first products. And this was 2017. And if you’re if you’re not mobile first, then then you’re probably not going to survive for long, especially in the b2c market. To my product was the company’s foray into the mobile platform. And it was also a platform for a quote unquote platform in in a in a different sense of the word for the rest of the company’s products to plug into. So yes, there was a front end aspect to it, but there was also a back end aspect to it in that a lot of the company’s other services such as payment gateways and KYC, and promotions and recommendation engines, and everything. would plug into that product, which would then be showcased to the user on the front end. So it was platform in like a couple of ways. One was it was a replacement for the desktop platform. And it was also a place for the rest of the company’s products to plug into.

Vlad G 14:21
Let me rephrase my question a little bit. You’re still you’re still product managing that platform? Yes. Okay. So this is this is just to give you a reason for this question. Yeah. This is something that a lot of companies are struggling with is coming up with the rationale. Why should we call the platform a product and what’s in it for us? I mean, it’s a platform. It’s not like, you know, a product that we can offer, package and sell. It’s our internal platform, kind of like a CRM and by the description, which you said is it It feels again, I don’t know enough to really call names here. But it seems like it’s a universal, kind of like a business. Can business processes boss or data boss feels like it. I’m not saying I understand what it is. I don’t have enough data to go on. But it just feels like it is a platform but not a product. How did the company rationalize that, hey, let’s treat this as a product. How did that happen?

Kaushal 15:30
That’s That’s an interesting question. I think I think. Well, yeah. I guess your question is also about the semantics of what is a product versus what is a platform? Yeah. And thank you. So

Kaushal 15:46
in, you know,

Kaushal 15:49
the the company’s model, revenue model or other the company’s business model was you know, having a solid white label gambling platform that could be spun off as you know, a custom made branded product for for brand a and could also be used, you know, you could apply another branded skin to it for for another brand and so on. So, I guess I guess it was a platform in the sense that it was

Kaushal 16:32
it was it was helping all of the other

Kaushal 16:37
Microsoft services integrate with with something tangible and the company’s rationale around calling it a product and you know having a product squad and product management resources

Kaushal 16:56
dedicated to it was was simply because

Kaushal 17:00
It was important for the company to to make

Kaushal 17:07
steady progress on that front

Kaushal 17:10
simply because the the internet landscape was changing right? desktop, a lot of desktop first technologies like the product was product used flash so the desktop product used flash quite extensively. And you’re probably aware that flash died a terrible deaths over the last few years were different, like all all the mainstream browsers have now stopped have either stopped supporting or provide a very limited support to flash. And so it was you know, it was it was in the company’s interest to to have a different product out there which could become the company’s main product, which we could sell to other brands. So, to me, I like it, you know, it made a lot of business sense for the company to get this product up and running in a matter of a few months, which is why it was it was given the title of product, but in the pure in the pure semantic sense, it is it is it was effectively a platform. It was not necessarily building new kinds of interfaces, it was just giving some of the old interfaces a new

Kaushal 18:53
place to be visible. If that, if that makes sense.

Vlad G 18:58
It makes it Yes. Thank you for explanation it definitely makes a lot more sense. Once you started talking about platform being a white label capability for for to be spotted for other brands and for other micro services to plug in. It’s starting sound is Yes, it did sound like it has it. It has all the rights to be called a byproduct. So yeah, it definitely makes sense. Thank you for elaborating. And it paints a better picture now in my head, at least in my head. The explanation. Okay, so you mentioned that’s that’s the place where you had 30 developers and you were split into the multiple teams, you called them. I think you called them squads. You have to work on this one product. So that’s another interesting challenge that I’ve seen in multiple engagements where product is large enough to warrant large development team but then you can’t really manage 30 4050 people you have to break them up into product teams or agile teams, however you want to call it show and challenges that I’ve seen. And this is this is kind of the question for you What have you seen, on your end Mike might provide what I’ve seen on my end was, it’s hard to come to a certain level of understanding, especially when teams are both varying the either old teams are very innovative and trying to get things done better. So they there look for opportunities in all the areas not specifically the areas that were assigned to them. And they, they have good intentions, right? They try to do better, or teams are very passive, and they only do what they were supposed to do. They’re not looking for ways to improve things. They’re not looking for opportunities to innovate and improve. And it all falls on the product manager to kind of run around and tell everybody what to do and do all the orchestration. So what was what is this what is your story? What is What was the situation with those teams?

Kaushal 21:03
It was?

Kaushal 21:05
Yeah. So because

Kaushal 21:08
so as I mentioned in my previous response, because it was a very important product for the company, and it was quite a large initiative within the company. We had a huge backlog. And this also meant that we, we we needed the all the technical muscle, all the resources we could. And so so the 30 people, we had included developers, designers, QA people from insights, business analysts, and product managers. And, you know, it’s like having a single product squad of these many people is basically not not practical, because it it makes all of the it makes the team unwieldly like having meetings with everyone involved. Coming to consensus is essential. It’d be hard. So it was just, it was purely for practical purposes that we were split into smaller squads. And each of these smaller scores about was about seven or eight people. And but all of us all of these squads were working on the same product. But because we had split into smaller groups, and because because each of these groups had their own ceremonies, you know, a couple of them use Scrum, a couple of them use Kanban. They had their own retrospectives, etc, etc, etc. It seemed that we we were working in a fragmented way. And we are sometimes pulling the team in in opposite directions as we were not fully aligned. We would

Kaushal 22:48
as I said, we would have individual backlog grooming sessions.

Kaushal 22:52
We that there wasn’t a complete sense of ownership. Like if if a bug would come up it it would take About about a day or so to figure out which score out of the four was accountable for that bug, and who would be the best person to fix that. So, so to summarize the challenges were around alignment ownership and And generally speaking of cohesion that was lacking among the squads. Now, of course, you know, as a pm This was making my my job really hard. So I wanted to get around these issues. So, so myself, a couple of the tech leads and and their Delivery Manager. We, we had a discussion around what we could do, and a few things came out, right. So this is, as I said, the problems we were trying to tackle were alignment, sense of ownership and a sense of cohesion, the way we went about alignment was was by creating a shared vision. Right? So I wanted us all to define the North Star that would guide everything we do going forward. So I gathered everyone in a workshop for a couple of hours and got every one of them to write their views on what we were trying to do on posted notes. And I, you know, I wanted to ensure that everyone was involved in that. I would encourage everyone to think about the why the quote unquote, why the purpose and

Kaushal 24:46
you know, like the, what came out of that was, was like

Kaushal 24:52
a singular statement which meant we want to help Apart customers quickly find discover and place bets on the games they love. Right? So this I mean, to an outsider like you, this may not be the most relevant thing. But yeah, like this statement gave us a sense of purpose. And this statement sort of was unique for all of us. So all the force courts would look at this statement for guidance. And on the back of this on the back of this vision workshop, I then had another workshop where we talked about our success metrics. So we’ve we’ve defined what our y is, we now wanted to define our water. So, so far for us, basically, we were looking for something that that would tell us that we are making progress on the why and to serve the country. out the success metric was was around time to wager. And what this meant is like from, from the point that a user logs into the product, how long does it take for them to place a bet. And the lower this KPI, the lower the time it takes for the user to wager, the faster it is for the user to find the game. And that was our definition of success. So we knew our y we knew our what and and this sort of guided everything that we were doing so earlier the teams are moving in opposite directions like some of them words. The some of them wanted us to pursue initiative a another squad was was quite passionate about initiative P. But now now that we all sort of agreed on you know, guys, this is what we are trying to do. It gave me a reason to do so. argue with them. And you know, like so if a certain team would say that we are passionate about initiative a, the next sort of counterpoint to that was, hang on, is this really taking us along the lines of where we want to be? So, so yeah, like this, this has the vision workshop and the success metric workshop. It helped

Kaushal 27:28
all of our conversations around alignment.

Kaushal 27:33
Now, the second bit was was, you know, around creating a sense of ownership.

Vlad G 27:40
I’m sorry, hold on one second. I want you to continue. But I have a question in regards to your Yep, go to the alignment part. So you’ll created the shared vision you have created the shared success metrics for the whole product. So you would have all four teams kind of driving towards this same direction. How long did it take them to agree with you or to agree amongst themselves to these common metrics? Was that just one workshop? Or did it take some time for them to come to kind of come to grips with reality? It

Kaushal 28:18
I wish it was just like this, these couple of meetings that you know, we all agreed on, on what what we want to do. But yeah, the reality was, was different. Out of the workshops like the the the main objective of the workshop was to was to get people’s thinking, right, get people thinking about a single question. Even if our thoughts were different, even if some some of the team members thought differently about about the vision, or the success metric, that’s fine, but at least they started thinking about it and it took me like, at least two to three weeks to sort of negotiate around around these two things. So, you know, like, it’s it’s really important for us to get the vision statement clear in everyone head in everyone’s head. And, you know, which is why the wording was quite important. So yeah, like it did take me a good few days or a couple of weeks to sort of have everyone agree on a singular statement. There certainly were different differences in the way we thought about what we’re trying to do. And the workshop simply brought all of that to surface. And it allowed us to start having a conversation around around this topic.

Vlad G 29:53
Okay, before before we could do one more question that I had, what would you say was the top What’s the top contention point or top tension point? There’s so in other words, you know, there’s usually like minor things that can be either completely thrown out or just kind of a compromise on or, you know, you can reach a compromise and something but there would be one or two pain points with each team that they really take a hard stand on says no, this is really important to me. I don’t want to compromise on it. But you know, I mean, I do want to compromise on it. But I want to I want to sell it for the heart. Sure, sure. So what do you what what were the points? without going too much into details? What were the points that teams didn’t really want to budge on until the last moment?

Kaushal 30:40
Yeah, so if I remember correctly, like one of the contention points, or rather, one of the main contention points was around what we should commit to as our success metric. As I said earlier, because it was a platform product, a lot have other products sort of plugged into plugged into our product. And we would serve the information we received from from from all of those other products. And so in some way our product relied on the information we would get from them. And it relied on the performance of those all of those products. So if if our KPI was say, you know, time to wage, or how long does it take for the user to place a bet from the time they log in it, it covers a lot of other things that happened between those two milestones. And for some of those things, we would rely on someone else’s product. And, you know, our performance was at the behest of their performance. So I remember that some teams were hesitant around committing to this as a KPI. Simply because we would be held accountable for this. And, you know, if not, if we did not have 100% control on what we are trying to impact, then is it really worth committing to that? And so yeah, like that was that that was one difficult conversation or other a few difficult conversations that I had to I had to have with the team. Makes perfect sense to me. I can see why they wouldn’t want to commit to that. But I see why. Yeah.

Vlad G 32:30
As a product manager, you would have to, truly so I definitely understand it. Okay. Cool. Thank you. That was that was really interesting. So you have two more left you have ownership and cohesion.

Kaushal 32:43
Yes. So

Kaushal 32:47
as I said earlier, you know, the ownership issue was mostly around, you know, if an issue came up with a certain feature, it took me a while to figure out who would be accountable to fix that. And, you know, I like to think that that engineers are highly skilled problem solvers. So, so I like to start by defining the problem or the opportunity, instead of instead of my saying, you know, here is a feature that we want to build. I prefer, I prefer presenting the problem first saying, you know, X percent of our users are facing ABC problems. And we want to pursue we want to explore solutions to these problems. So So, you know, once I’ve, once I’ve defined the high level problem, the solution exploration could be a team effort. So the solution does not necessarily have to come from the designer or the product manager or someone higher up the ladder. You know, if the team is invested, I think if the team is involved in the solution making or solution design process there there is an automatic sense of ownership like engineer, if an engineer is involved in in saying you know what the best way to solve so and so problem is such and such, then if in future an issue comes up with with that solution, the engineer is much more likely to, to sort of own up to that and and find better ways to fix that problem. So, so yeah, like the way we

Kaushal 34:33
went around creating a sense of ownership was

Kaushal 34:37
was mostly around how we would present new feature requests as problems who solutions needed to be found out. And so, that was around creating a sense of ownership and around creating a sense of cohesion. We You know, we a couple of couple of things come to my mind one was having a group backlog grooming session where instead of having like individual backlog grooming sessions we would have a single backlog grooming session where every every squad or at least one member of every squad would be invited and you know, we would talk about up and coming features which we would like to explore and what this helps with was what was that every team was aware of what was what was going to be worked on in the in the in the near future and by what team. So So that was one and the other was having a group retrospect So, so this is where we would discuss process improvements across the entire group and not within the individual squad. So So yeah, like having a group backlog grooming session and having group retrospective sessions, we were able to generate visibility about what the entire group was going to be doing. And also discuss, it also gave a forum for everyone to talk about how we could improve as a as a as a group, rather than individual squads. So yeah, like a In summary, the way we went about solving some of these problems were step one, create a shared vision. Step two, have a sense of ownership. And step three, involve everyone in you know, in a group retrospective and a group backlog grooming session, to have visibility and a sense of cohesion.

Vlad G 37:00
Cool, thank you, I might have missed this in your answer. But did you end up with the same cadence same ceremonies across all squads or they kept whatever they were using just came to the gel ceremonies for

Kaushal 37:18
the whole day. The The, the existing ceremonies that the squads had were not affected. Because they were they were anyway important. They were anyway required rather. And, but because we did not have a forum where the entire group was meeting, we needed that forum for visibility and for discussing process improvements.

Vlad G 37:43
Okay. All right. That makes sense. Okay, I mean, I folders fancy I kind of expected you guys to come up with a common cadence common framework, common kind of a job approach is something we in our organization are advocating for because it’s not, it’s not necessarily improves cohesion while it actually does. But that’s not even the main goal. The main goal is predictability of the results. And kind of alignment across, across multiple teams working on the same on the same product or the same project or within the same initiative, the same work stream, if you will. In our in our model in our governance model, everybody’s using whichever system, whichever framework, whichever agile approach they like, we don’t basically we’re not going to tell you which one to use, just as long as you stick to one. Everybody should be using one because it improves predictability and improves the quality of overall delivery. But it’s interesting to see you guys having different experience. And my understanding is because you didn’t do that it that doesn’t Straley it doesn’t automatically means you’re Right, you’re still you still weren’t successful even without instituting the common agile approach across different teams.

Kaushal 39:06
Yeah, I guess I guess it’s

Kaushal 39:09
there is there’s only so many battles that that that product manager can can fight. Right. And it was really important for me to to find the important battles. So yeah, like there is there was a, you, you you, you get someone you give some right. Yes, I wanted to get a few things done in the way I saw fit. And in return, of course, you know, the team deserved to do a few things that we they want to do them.

Vlad G 39:39
Okay. Okay. Make sense? Again, in the same full transparency, I’ve seen teams not having same frameworks, not even same cadence, but still kind of moving forward. Because they had different maturity levels. And and that’s perfectly fine in my book. I mean, I’m not saying they have to, but It looks, you know, from overall experience looks like this. This is a better approach, but I understand you. I understand the ways you have to find your battles. And I’m with you on that one. Believe me, I know what she’s talking about. Okay. All right, cool. So moving forward. Let’s talk about the third or the other engagement that you had in the b2b company that you mentioned. Yet again, I like how you give the overall background first, and then we dive into detail. So she’s gonna let you do that. It looks it works well for me. So she’s worked well for audience. And now we’ll dive a little deeper.

Kaushal 40:39
Yeah, so. So yeah, this this is about my time at global app testing. And global app testing is a cloud testing platform. Our our clientele included customers like Facebook, Google, Microsoft, Spotify at some point shell G, and so on. And what we do is we help these brands test their products out in the real in the real world with real users in in various parts of the world. So it’s a, it’s a classic marketplace like model where you have a demand side and supply side on the demand. on the demand side, we had these big brands who need to test their international products out in the wild. And on the supply side, we had a crowd of professional testers around the around the world. And if I remember correctly, your question is about the challenges like the challenges if relevant to product management, in this setup, right?

Vlad G 41:43
Correct. Yes. Since you’ve said yeah, full remote setup, b2b and this is before the pandemic. So how did you guys do that?

Kaushal 41:52
Yeah, yeah. So to working at a global app testing was an interesting experience in terms of orchestrating distributed teams. We we had product sales, marketing and finance teams in London. And all the other verticals like tech, QA operations, data science. We’re in, we’re based in different parts of the world. Even within my product squad we had, we had to work with at least at least three time zones. And, and this made it a little bit tricky for us to move together at at first. Now, the challenges were primarily primarily around engagement within the team. It was, it was it was hard to have live discussions, it often happened that it’s it’s lunchtime for some devs when you want to discuss something important, and it’s, you know, it’s it’s, yeah, it’s it’s not lunchtime in London. I know the challenge was that it was It was hard to generate team chemistry because we were not co located. And needless to say that English was the second language it was its third for me. And this didn’t really help at the at the start. So our meetings early on, they were quite dry, you know, with with little to no participation from the team. And this led to again, a lack of ownership, slower velocity and, and, and communication gaps.

Vlad G 43:35
So

Kaushal 43:37
in order to come around some of these problems, we we made a simple, simple start, we had the team members lead the recurring meetings like stand ups or retrospectives. And after some time, like after if, you know, a couple of weeks or so, the team members who are leading these meetings, They started seeing that that everyone else responded well to their instructions during the meeting. And that gave them the confidence to participate. The the concerns they had about about English being their second language early on, they were they were less of a concern now because they saw that they were able to successfully lead some of these meetings.

Vlad G 44:26
So allow me to interrupt you for a second. Yeah. Was that just one appointed person specifically, per team? Or did your rotate that facilitator role across different team members to kind of see who would who would do it better?

Kaushal 44:39
It was. So I like to think that they’re all of these groups are like especially the startups and the retrospectives they are for the team, rather than for the product management person or someone else. And I personally think it’s it’s great if the team of volunteers to to host Any of these meetings and but in our case, what was happening was it they were being orchestrated by the product manager. And as I said, it wasn’t helping. So we, we started going around the team. So it would be a single person leading the standards auditor just retrospectives for a week or a sprint, and then they would pass it on to someone else for the next sprint.

Vlad G 45:26
So rotating. Yep. Good. Yep. Thank you. So,

Kaushal 45:31
yeah, so after that, we we also switched to having asynchronous communication. And, you know, async communication is when when you send a message without expecting an immediate response. For example, you send an email, I open and respond to that email a few hours later. And synchronous communication is when you when you send a message and the recipient processes the information and responds in Immediately so in person meeting in person communication like meetings are purely synchronous communication. Now, the way this helped us was it, you’re probably aware that engineering requires deep work right in that you would want to work on on specific agendas for unbroken, uninterrupted periods of time. And having asynchronous communication allowed us all to do deep work. It reduced interruptions. It gave us time to think about what question was being asked explore a response before sending it and you know, plus, because we are in different time zones async communication was was innovate inevitable

Kaushal 46:58
but

Kaushal 47:01
My order the product management specifically saying that, you know, we don’t need to respond to questions live, I think that went a long way in, in alleviating any of the concerns that team members had around around communication or life meetings. So, three I like moving from sync to async communication was another thing that that helped our communication challenges. We then beyond that, we started having virtual coffees you know, with with with a slack bot, bearing three members of the team and scheduling a meeting for them. And if you remember I said earlier that you know, we were lacking team chemistry because we are not in the same location. So, when when you are in the same office, you can probably go out for lunch, or go out for a drink after work or go out for coffee with you colleagues and you know, you get opportunities to know your colleagues or your team members better. So yeah, which is why we started exploring having virtual coffees. And we would, we would do celebratory lunches. So after launching a feature, we would all have lunch on a video call together and talk about food, talk about what what we just launched. And we also talked about stuff happening outside of work. And, you know, all of this allowed us to know each other better, and, and help build some sort of chemistry. And lastly, like one of the most effective things that worked for us was epic ownership of feature ownership. So we wanted to cultivate, as I said earlier, a sense of ownership and accountable In the team, and we started having the system where we would make an engineer and owner of a feature alongside the PM. So So this engineer would be involved in product discovery for that feature. The product discovery would be led by the PM, but yeah, the engineer was was had the option to participate in that process, or be present in the conversations with the stakeholders or with customers. And, you know, this made them much more engaged. And I guess, I guess, you’re at the end of the day, the engineer is the one who’s going to be building the solution. And the closer the engineer is to the problem, the more effective the solution is, at least in my opinion. So So yeah, like it was a mix of these things like In summary, you know, having having the team lead the recurring meetings, then switching to asynchronous communication, having virtual coffees and celebratory lunches, and having epic ownership, like all of it was a combination of all of these things that that helped us get into a good rhythm and help us build some sort of chemistry as a team over a period of time.

Vlad G 50:32
Thank you. That is that is really interesting. I just have a question. I didn’t want to interrupt you. But I had a question when you said you had an engineer, as a single, as a single person, being the feature owner or participate heavily, heavily invest in property discovery around that particular feature? Yeah. How did that align with a business continuity We’re all people when they’re standing up engineers may come and leave, you know, people have, you know, their problems, they may go on extended vacation or extended leave. Things happen. How would you it was there any backup? How would you alleviate this this risk from from a business standpoint? I need the I need to make extensive changes to that particular feature why someone was on? I don’t know, two weeks vacation.

Kaushal 51:27
Sure. Hmm. That’s an interesting question.

Kaushal 51:31
We, so, this was this was purely, so epic ownership was was

Kaushal 51:41
we would we would ask engineers to volunteer rather than we would rather than us saying that so and so engineer is the epic owner for for this epic or feature. And so, so the the early conversation we would have with with the squad was mostly around You know, who’s number one who has the right skills for this required for this feature? who is who is available over the next quarter or next couple of Sprint’s to work on this. So, so yeah, like some of the some of the concerns around ensuring that there is a continuity in in that the person who is who is who’s capable of building the solution is available to build that solution for a good amount of time. So that that that was addressed at the start, in the case where, you know, the the engineer had to take a break or or they leave the the left the business. If I remember if my memory serves me, right, it was, the engineer would I mean it would it would be a standard handover process where the engineer would basically work with One of the one of their colleagues, and, you know, write up stuff that that would make sense to this other colleague, and then pass on the ownership to them. So yeah, like it was a combination of of ensuring that the engineer is available right at the start before we begin the feature. And then in case of the engineer leaving midway through the feature, it would be like a regular handover to to a colleague.

Vlad G 53:27
The reason of, well, there are multiple reasons why I was asking this question, right. But one of the things that I have experienced in some of my previous jobs and engagement was that there was a particular feature in the legacy product, which was the main product offering of a company I was working for, and there was one and a half people who knew what’s going on and half half of a person not not the real half person I was a business analyst who kind of understood the business processes and some of the technology involved in solving for that particular problem. But then there was the one engineer who built the whole thing end to end. And when that engineer for whatever reason became not available, I quite honestly don’t remember what happened. Just because it was it was a remote development team. And I just at some point, I just realized that the person is not there anymore. And I started asking questions like, Hey, we have this change coming up. And we need to we need to ensure that there is a you know, we can apply those changes. And when you figure out what to do without that subject matter if I see it, yeah, it turns out that what again, I don’t know exactly what happened. He didn’t leave on good terms. And he was not available. He was not available for any kind of even for consulting and they’ll became a big problem for the company because they didn’t have anybody else. The It was a legacy code and would take them, I don’t know, two months to even figure out what’s going on because there was a lot of patchwork, and logic and business logic on top of technology logic, spread across multiple store procedures on different databases. It was it was a nightmare. Every time every time something like this pops up this this nightmare scenario pops to my head. Are you guys gonna deal with that?

Kaushal 55:28
I mean that that does sound like a terrible way to end things for an engineer.

Vlad G 55:35
Just sometimes it is what it is. And then again, it I’m not really I don’t really remember what happened and they really care at this point. I’m just saying this is the business. This is the business scenario that whenever you are thinking about assigning responsibility, that’s something that’s a risk you need to alleviate somehow.

Kaushal 55:56
They’re like on the back of what you just mentioned, I remember that You know, the the tech lead or the architect was was always consulted on on whatever feature or or or a solution we have building. So even if like an engineer was the subject matter expert for a certain feature, they would have consulted the the the tech lead or the architect and, you know, so there was always another brain who, who would have some context of what the solution was so, so that was our like Plan B if that happened, but thankfully we did not have a situation like the one you mentioned.

Vlad G 56:38
Yep. That Lucky you. All right. As we coming up to coming up at the end of the hour, I have a couple of questions. They are regular kind of same thing I asked every guest on the show. And I started I started offering a choice which one you want to tackle first. There are two questions. One is in this new world, and I remember, you know, you said that you just started a new job a few months ago. So it sounds like you started your new engagement union job right when the pandemic hit or thereabouts. What How? So the first question is, how were you affected by this new norm or this, you know, new situation? And the second question that I always ask on this podcast is, do you have any questions for me? So this is your chance to turn the tables around and ask me for whatever whatever questions you have. Hopefully, we’re not going to, you know, discuss world politics or you know how to solve the world hunger or develop a vaccine because I have no idea I’m not the life science, less as expert, but something around product management will do nicely. So either one you want to you want to tackle. Either one you want to go with first or second. Let me know

Kaushal 58:03
Yeah, the second is is is fine. The second one is okay. About the first one, you know, like,

Kaushal 58:10
I’m just wondering if there is

Kaushal 58:15
yes, the crisis has had an effect on the business. I’m just wondering if if I would be okay. Talking about it on a public forum. So, if I can probably speak about how it has affected the way we go about our day to day rather than the impact on the business.

Kaushal 58:44
Does that does that sound okay?

Vlad G 58:46
Yep, sure, whatever, whatever. Again, this is more as more of a recurring theme. We’re all living in this new world. So, and some of the previous guests started with how they go about doing forming their job responsibilities. And we always slip into, you know how this affects personal life because now we are working out of our personal spaces, not workspaces. So it’s it’s inevitably, you know, merging affect each other. So however you want to tackle this, by all means.

Kaushal 59:21
Yeah. So, tell us I’ll start by start with the first question around how this crisis has has affected day to day work and product management in general, at least for me.

Vlad G 59:36
Yeah. So,

Kaushal 59:38
of course, you know, large majority of the companies have have have encouraged their employees to work from home. And pocket is no exception. In fact, yeah, like we, even before the government announced a lock down. Our company internally encouraged everyone To work from home, if that was possible. And again, this this came as a little bit of surprise, of course, because there wasn’t like a two week heads up that that this was going to happen. So initially like the first week was or other first 10 days or so we’re, we’re we’re challenging simply because

Kaushal 1:00:28
for a few reasons like number one being

Kaushal 1:00:32
in the office, I asked, you know, the designer or some some of my other colleagues they would, they would work on like multiple screens, they would have bigger screens to work with. And because all of a sudden we were working from home. It was and not everyone had like the like home desk, or home office, where we will have the luxury of a largest screen so today I like getting used to a smaller screen getting used to your own laptop was was a little bit of a challenge in the first few days. beyond that. I guess as as people who work in the technology space, working remotely is not a big issue like ever since we had outsourcing ever since there was this concept of outsourcing. You know, engineers have been working remotely so that has been like for a couple of decades at least. So yeah, like having having meetings or having calls over the internet is not a problem at all. But sometimes he has like the, you know, when you’re in the office and when when you have some of your colleagues sitting just across the across the hall. It’s quite easy to, to engage them in spontaneous conversations about work and You know, like you You have the luxury of using a notepad or a whiteboard to explain you know your question or your query and have a discussion around around that diagram or or whiteboard presentation that you’ve made. But yeah, if any are behind the screen when you’re when you’re working remotely, you you don’t have that luxury. And yes, you have, you know, tools like Miro and whatnot to create, like, online whiteboards or, or, or virtual workshops and whatnot. But again, you need to set them up, right? You need to you need to schedule them in people’s diary. You need to give people a heads up that you want to engage, engage them in a conversation like that. So there are some natural hurdles to having spontaneous conversations which which does take

Kaushal 1:03:00
We which does impact your productivity to an extent

Kaushal 1:03:05
but you know rightly or wrongly we can’t change we don’t have a have an any other option right we we have to get used to this new way of working at least for a few months, if not more. So I guess

Kaushal 1:03:25
at least my team has taken well to it

Kaushal 1:03:28
you know, our conversations over slack or or our video meetings or zoom or or any other tool that we’ve been using the the they’re, they’re going just fine. People do miss having spontaneous conversations around work or you know, even beyond work. We do miss going out for a drink after work or or having lunches together in the office. We sometimes have like virtual coffee sessions with with my team Where we just talk about what’s new on Netflix, and we talk about all sorts of things over there. So yeah, we are trying to find ways in which we can engage more spontaneously and talk about things outside of work. It is not the same as being in the same office is certainly different. But I think I think we’ve learned how to get around them. And I think it has become the new normal now, at least for my team, or at least for me. And, like, sometimes, I think, how it might be like getting back to, to the office. So So yeah, like, those are some of my my thoughts around how how the current crisis has affected day to day and like the work that I do.

Vlad G 1:04:55
I see. Thank you that that is interesting. We’re welcome. And I’ve been working from home for the past two years outside of just outside of the time that I am on engagement where I have to be face to face with the client. So I agree there’s nothing, not much difference for me. I was equipped even before that happened yet what we’re seeing is actually more of a impact when people are deliberately not setting themselves up to be to be able to work from home kind of making that mental compartmentalization working from home working versus being home where they’re not working. And once this delicate, and there are some cultural things where people not used to working from home not because they don’t want to because they’ve never done it before and you know, homeless for the home. True. But it’s interesting because again, as you said, I this is a recurring question I keep asking is all of my guests. I keep getting different answers. I love it. So you know, everybody

Kaushal 1:06:00
They like what you said they’d come up in one of our recent office surveys where we asked for people’s thoughts on on what’s working and what’s not working, when it comes to remote work, and there were some concerns about people scheduling teams, sorry, people scheduling meetings with their teams, and outside of office hours. And we later found out that these were people who are outside of tech, like who are not familiar to the idea of remote working. So so I’m sure you know, like for for people who, as you said, it’s a cultural thing for I think tech teams are used to this culture of remote working and sort of compartmentalizing their work and life, whereas some other departments are not. So yeah, I hear you like sometimes it does happen that

Kaushal 1:06:54
people outside of your

Kaushal 1:06:57
team may not be sensitive towards How working from home might work for you versus for them.

Vlad G 1:07:04
Yep. All right. Cool. Thank you. That was interesting. Last last thing on my list is the the turn turning of the tables if you have any questions for me, again, hopefully around product management. This is this is the chance that if you don’t have any questions, that’s fine, too. It’s, it’s up to you if you want to take this opportunity.

Kaushal 1:07:25
Yeah. Sure. Like so as

Kaushal 1:07:31
how do you think of this podcast as your product?

Kaushal 1:07:37
You know, if you put on your product manager hat rather than the head of the host, what, what what problem are you solving out there for the user and like, what what is your goal with the podcast?

Vlad G 1:07:53
That is a really interesting question as definitely a new one. Haven’t heard that before. So You’re right. And I do approach this as a product rather than I’m a host your guests, let’s talk about things. because inevitably I have to because of the nature of my own work. It’s a occupational hazard, if you will. Sure. I, I have a couple of businesses before and my major responsibility in my current role is go to market for the products that I’m working with. It’s not necessarily developing of the products is going to market was either the current versions of the prototype or the MVP and then based on how market responds, continuing the product, discover continue the product development. So having said that, as the product manager of my own podcast, I am trying to go to market, my current problem the if you’re familiar with one metric that matters. My current metric that I care about is adoption. I need more people to listen to it so that’s that’s what I’m doing. I’m trying to build up on two things I’m trying to build up on the content having more content because one thing with podcasts is it’s not live translate it transmit it’s not live broadcasts You don’t have to be in front of your TV or in front of your radio with a specific our crew and everybody can listen to any episode they can they can they don’t have to start with episode one. They don’t have to start with the latest episode. They can start with episode number five and then go to 12 and then go to one so every episode is it’s a little different from episodes in your favorite sitcom or your favorite TV series. Every episode has to stand on its own on its own two legs. Interesting and and and so that’s the content front it you know expanding on content, having interesting episodes by themselves is important then the second part is expanding. into as many platforms as possible. Unfortunately, I can’t do live videos, I can’t do videos, you know, with my poking hat and a guest talking head, because not everybody is equipped and people don’t necessarily want to be seen. So it’s a challenge I sometimes I talk to people who don’t want to be seen. Sometimes they talk to people who do want to be seen, but they don’t have necessary capabilities. Even you know, if even just recording the audio stream is challenging, think about recording the video stream. So I’m trying to kind of sidestep those things and still expand into video platforms like YouTube and others, but mainly trying to get to YouTube right now. So I started the channel on YouTube, I started posting episodes, there was just you know, playing visualizations, but at least now you can listen to these podcasts on YouTube as well. So, again, since this is not my full time job I only have so many hours I can dedicate to producing, getting the guests on having the actual interview than post production, producing the episode of creating all the necessary meta work around it and then posting it all over the place. So, you know, with I’m working with what I have, I, I don’t think I’m doing everything I should be doing it just because I don’t I’ll lack enough time and knowledge but as I keep working through this, either iterative approach to product development. Sure. I’m sure I’ll get there. Yes, I like the topic. I like doing what I’m doing. I like product management as a as both science and art. I keep saying this, the product management as much science as it is art. So you can’t just go and get an MBA and become a product manager and become a solid product manager. Yes, you do need education. Yes, you do need to learn as much as you can, but you also need have real world experience making things happen in whatever branch of, you know, responsibilities before you can really become a good product manager. So, yeah, that’s that’s, thank you for the question. That was good. And that’s that’s the answer. Brilliant.

Kaushal 1:12:18
Thank you for for for sharing your answer.

Vlad G 1:12:22
All right, thank you so much. We’re slightly over time. I don’t think it’s a problem. Thank you very much for being guest such an awesome guest on the show.

Kaushal 1:12:32
Thank you for having me. My pleasure.

Vlad G 1:12:34
Absolutely. It’s a pleasure. Thank you very much and see you soon. See you. Bye. You’ve been listening to the real world product management and I’ll be your host Vlad Grubman. Until next time,

Real World Product Management – Episode 11

May 18, 2020 Vlad Grubman

In this episode, I am talking to Erin Wood – a project manager at NASA. We discuss how and where product management has to give way to project management, and vice versa.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Alright, Hello, everyone. This is yet another episode of Real World product management. And I have Erin Wood with me on the call today. Hi, Erin. Hi, thank you for finding time to be on this episode. Can you please go ahead and introduce yourself? What is it that you do? What is your role?

Erin W 0:34
What do you work here, and I am currently employed by Northrop Grumman. I’m contracted in NASA Glenn Research Center in Cleveland, Ohio. So report up to our headquarters location. And I am responsible for production planning and scheduling for a variety of projects that are all destined for the International Space Station.

Vlad G 0:58
That is awesome. Thank you. When I was a kid, I always dream to be an astronaut. But unfortunately, you know, being born in the Soviet Union and not having a perfect eyesight kind of closed it down for me.

Erin W 1:10
So, believe it or not, I never planned on working at NASA Glenn Research far I grew up in the Cleveland area. So this has always been in my backyard. And I am third generation of women supporting NASA Glenn Research Center, which is kind of unusual. My mother in law was employed there for 30 years from retired not long before I started, so I have the interesting perspective of seeing what it was like for her as a woman in the 70s and 80s. You know, really getting her feet wet at NASA Glenn Research Center. Prior to that, unbeknownst to me, in my youth, my grandmother’s cognitive Annie Glenn, Glenn Research Center, so I had got to learn a lot about what it was like for her to be a woman supporting NASA on research. Via her husband, john. And that’s also been, you know, an interesting glimpse of womanhood from the 50s. Say, it’s a lot of fun.

Vlad G 2:09
So to me that okay, that is that is even, is even better. So my obvious question here would be well, did you what what were the differences and what were the similarities, if any, between these generations?

Erin W 2:26
Oh, well, I mean, Annie obviously was married to john lennon, who is one of, you know, the United States best astronauts, of course, and it will always be my favorite for obvious reasons. But, you know, she was more of an in a supportive role. You know, her responsibility was the, you know, Americans trophy wife. And she was really really good at it. I have to say I have her rice pudding recipe and it’s the best you know, what she did was just as important although, you know, In a totally different capacity than what women at work would do today, my mother in law was one of the first ones to kind of venture into the workplace. She worked in document management. That’s what we call it today. So she was in, in more of an administrative role and and more of a secretarial role. And she got to deal with kind of the first brunt of women working in in an engineer, engineering environment in their research environment. And it was much different than where I come from today. I have a master’s degree in project management specifically. So in my position, I just have a little bit taken a little bit more seriously. I think that she was more of an equal partner, I think then than some of my predecessors were, although that said, there’s some advantages to each and I think that the women of today definitely have something to learn from and admire from every generation that’s come before us. Thank you. Wow.

Vlad G 4:02
Thank you. Okay, so tell me a little bit more about what is it? I mean, in within the allowed space? What is it that you’re specifically working on? And why does NASA need project managers? What is it? What is it there that project managers do?

Erin W 4:23
Well, I work in the background of project management. And so what I do is work with either projects who are in the development phase and are planning out their work and building a schedule, as well as projects that already have that defined and are in more of a monitoring and controlling phase of their project lifecycle. So I want to report up their their milestones to headquarters. I work with our cost team to correlate between schedule and cost, as well as our risk team to correlate between schedule and risk. So the three of us are, are kind of an integrated discipline in addition From a, from a project planning perspective, I incorporate a lot of Lean and Six Sigma techniques in order to remove constraints in advance and help things move really smoothly.

Vlad G 5:11
So this is is this have anything to do with a specific space programs flying space? Or is that some kind of internal initiatives that are staying on the ground?

Erin W 5:24
Well, specifically for me, I work in in code MSI, which is the physical sciences and human research for experiments going to the International Space Station. I mean, of course, at NASA Glenn Research Center, there’s lots of different different programs going on. But that’s the program that I specifically worked on.

Vlad G 5:42
Okay. All right. So your stuff flies into space, am I Am I understanding this correctly?

Erin W 5:48
Yes, as long as everything goes well, ever it will fly into space. It will be launched up on a shelf shuttle, taken up to the International Space Station in the form of experiments. And when It’s that experiments turns and the astronauts on the International Space Station will perform experiments and then send all of the data and sometimes the materials and the physical experiment itself back down to earth

Vlad G 6:15
as a project manager, and you’re This is so let me step back a little. This is a product management podcast right? So why do we have a project manager? The reason why we have you here is because a you work on some some of the most advanced and most amazing stuff ever. At NASA. I am I’m really amazed that everything that has anything to do with the space and second reason is because although I am a product manager, I have been the product manager for almost 10 years now. I still think product mindset should coexist with Project mindset. So every you know, there’s a tool for every job and to me, it seems like launching shuttle or launching stop on the shuttle into space is probably one of the very most wonderful things ever. Because once you launch the shuttle, you can really, you know, recycle really reiterate and do this, you know, oh wait, this didn’t lunch the way we want it. Let’s, let’s do this, again, doesn’t quite work that way. So that’s what we have here because I think we should be product product and project managers, we should be friends not you know, we should not we should not create the gap between devices it should, we should we should think about how to make it work together. And one of the questions that I had based on what I know is it seemed like the shuttle launches are pretty hard deadlines like there that’s at installs, you know, you lose your window. Things fly away, you know, weather changes of other any reasons. And again, in product management, we are way more flexible with this certain Things are hard, hard sets hard and stone. But in most cases, we’re flexible. We can play with a scope we can play with deliverables we can play with the deadlines. What is the situation with you guys? At NASA, you know launching things into space?

Erin W 8:18
Well, the launches are pretty set in stone. The launches are scheduled and they might fluctuate a little bit to the left or right depending on atmospheric conditions, and shuttle preparation. However, that shuttle is going to launch when they decide it’s going to launch with or without our materials on board. And in our materials don’t make it it’s going to launch with a different material. But it’s like going with or without us. So if the materials are not provided to the on dock location, with plenty of time to spare, they will not they will not get on the shuttle. And they always have backup experiment, ready to go so they experiment with the most Already, as long as it’s ready and on dock, it’s going to get to launch. But if it’s late, the launch goes on and you miss your window. And that’s just it. If you’re lucky, and it’s a high priority, maybe they’ll put you on the next launch. But that’s not guaranteed.

Vlad G 9:16
Wow. So I like again, this is this is both similar and different from how in product management, you would, you would build a roadmap. If you don’t make it this release, you can always say, we’re going to do it, the next release is there’s a lot more flexibility. It sounds like each launch in your case has its own queue, and all experiments that potentially can get on it. And if you’re missing, if you’ve missed your own window, there’s no guarantee you ever get into another queue.

Erin W 9:46
Is that absolutely. And you have to keep in mind each window has a highly specific amount of math and date that can get on the shuttle. So in addition to the time can Joining, we have to make sure that our material will fit into each individual shuttle. So each shuttle has its own dimensions and weight capacity. And so if your experiment is larger and need more space and more needs, then you have to be on a per diem, you have to be on a shuttle that will fit that. And not all shuttles can’t fit all experiments. So there’s there’s some additional considerations and some additional constraints and very little wiggle room.

Vlad G 10:28
Yeah, that’s that’s another argument about project management still not being not being obsolete. I keep hearing in some of the product management community that product management is way better, it should replace project management. This keeps telling me it shouldn’t there are different tools for different jobs.

Erin W 10:47
Well, I think when people think of project managers, a lot of times they only think about software. A lot of people think about software is the primary use for project managers. And I think that’s probably a similar thing that you get Product Management. However, I am one of the rare ones that have never worked in software, I actually am terrible with anything it or software related. It’s just not my field of interest. So I come from a background of working in construction project management, in manufacturing, project management and now in project management. So it’s important to keep in mind that, that, you know, that war might exist within the software community, but not necessarily in all these other industries. So I think it is a lot like what you said is the best tool for the job. And a lot of what we do in our as far as data analytics is very, very similar.

Unknown Speaker 11:45
Now, what do you mean the what would you do in data analytics? Can you elaborate on that a little more?

Erin W 11:52
Well, in in developing a research project, we’re also building a business case, a lot of times we have external partners involved whether it be Countries or businesses that have a financial interest in the data that we’re producing. Our main stakeholder is the American public. So, you know, everything that we do we answer to the American public as our as our primary stakeholder. And obviously, that’s pretty broad, broad group there. But we also build a business case, we go through development phases, where, you know, we would call it pre phase a NASA. So our project lifecycle will go through, you know, concept and development and a very high fidelity level of testing, primarily safety related long before that product and ever project is ever ready to launch. And that’s very similar to what you guys do. You build a business case, you have a concept, you go ahead and develop it. And then you would you would market your your product. For us. Our product is that data that’s coming back from the International Space Station. That is the product that we’re building towards network gathering to provide to the American public.

Vlad G 13:08
Right? Makes Yeah, thank you that makes sense. The only difference would probably be that product development, especially in software, product development is way, at least the the right way of doing things is a lot more iterative. So like I said, if we didn’t get things into this release, or launch in in other terms, sometimes people do colon launches. Now we can always hit it the next and the next, the next we can always reprioritize we’re a lot more flexible. In that sense.

Erin W 13:40
It does kind of twist me out when you say the word launch because you’re talking about a product launch and I’m thinking of an actual space shuttle.

Vlad G 13:49
Right, I’ll keep I’ll keep it to the releases. As a matter of fact, no, let me correct myself. You’re right. I’m not using this term correctly. We do use the word launch. When we Launch the Major Leagues like for example, we had, in my experience, we had an idea, we’ve iterated on it for for a number of months, we have figured out what we wanted to do, how the product will look in the market. And once we had the product in the market, we actually do the launch. It’s a one time deal. And it’s at you know, you can only do this you know, so often and it’s when product is presented to the market. It’s a go to market internally, but externally it’s it’s a product launch, we’re now making this product available to the market. So that’s that’s the that’s the proper way of using the term I bought that said okay, so from and as, as I was going before, I my goal in this podcast and reason to have different people is to understand not the glorious stories of of, Hey, we came in Everything started working properly. But talk about the real world challenges. And given that you’re not it, and most of the folks who have been on the show so far are from it. I am, I’m with a particularly very curious to hear what are the challenges? And you mentioned some, like missing the window not being ready. What are their challenges that you guys are facing? When you’re executing your projects? What are the challenges? The real world challenges not, you know, we came in everything started working, the stories that may get in the way of making things happen.

Erin W 15:42
Sure, I actually look at it and I’m not even going to be as specific as talking about my experiences at NASA because this is true of my experiences in construction and manufacturing environments as well. As an analyst one of my responsibilities is to do the math. That goes between cost, schedule and risk. And some people that will convert that into Earned Value management. There’s a lot of level in between in terms of integration between those those metrics and, and lots of different things that you can do to compare one against the other in order to map them. And so that’s my, that’s my area of specialization is looking at these metrics, and finding a way to translate scheduling the cost and cost into risk and all of them together. The number one problem that I have found in any industry is that there’s there’s typically a culture very much against data management. Math is is very specific, it’s black and white. You look at it and it gives you an answer. And that answer does not care what you think. It doesn’t like you or not like you. It’s it’s completely, you know, objective it It just has no opinions one way or the other. And in almost every environment where I’ve worked in, there’s been a culture against that people just typically do not like that level of accountability. A lot of people, project managers would would rather have the ability to, you know, make decisions based on what they think would happen or what they hope would happen. And when you show them the math, sometimes it doesn’t necessarily correlate to be the same, the same outcome. The math might say, hey, our current projection shows this is going to be the most likely outcome. Teams as a whole, you know, engineers in many fields, many industries, including research are brilliant people, and they do amazing work but they’re not always known to be the best planner. So they tend to resist a level of planning that that sets up for data management. They don’t always like that because, you know, there’s a there’s a high level of accountability. You know, when you’re looking at that, and it’s not always A welcome thing. So the biggest struggle that I’ve had in all of these industries is creating a culture in which there’s acceptance toward using data to make decisions. Because people are typically fearful that that data will ultimately be used against them, for example, you know, if they’re confident something can be done, and they kind of take a leap of faith going against that math and turn out to be wrong, you know, maybe they’re afraid they will click a guru in front of their boss or, you know, that, that that some for some other reason, they just don’t want that data being, you know, spread up the pipeline, they’re, they’re afraid that it’s going, they’re going to be micromanaged by it, or they’re going to be held accountable to it. And there’s just typically a culture where that’s not very acceptable. Wow.

Vlad G 18:47
That is so much different from what I’ve seen. Okay, let me let me correct that what I’ve seen recently be the data driven design visions and data driven development is one of the core principles of the product mindset. And I would argue that there’s actually a discipline within product mindset where you’re only making decisions based on the data. And as a matter of fact, if you look at the way Google builds things, they are very data driven organization, they build their products, they make their decisions based on the data and data alone. And I actually, I’m actually arguing that it’s not the best way of doing things. But because there are certain things that data cannot tell you or you can only make decisions as good as the data you have collected and granted. Organizations like Google or Facebook or others collect mongos amount of data and so they are, you know, within their right to make those decisions. And those decisions are mostly correct. But there there’s there’s an edge case When you don’t have enough data to make that decision, for example, you’re you’re debating whether you should develop or not. Or you should make certain things happen or not. And you only have two customers, one saying, Yes, I want it and another one saying, No, I don’t want it.

Erin W 20:18
What would you do? Well, I think the most important thing to do in order to create a culture where data management is, is encouraged, and I have to say, NASA is by far, you know, the greatest level of Project Management Maturity in any organization that I’ve worked within. And they are by far the most set up in order to be successful in this. So I have, I have full faith that, you know, this is, you know, the data is successful at NASA. But the the greatest way to build that on a team level when you’re working with people that don’t traditionally have, you know, a high level of project management knowledge, you’re working with people who are not project managers or project analyst by trade. You know, the greatest thing that you can do is just keep it simple. You know, sometimes we put such incredible math into the metrics, and we develop KPIs that are so complicated that nobody could possibly understand where this information comes from. And that’s I think one of the greatest things that lead to rejection is that we just overcomplicate it, and people don’t embrace things that they don’t understand. And we don’t always need the most complicated math in order to provide the data to make decisions. So by keeping it simple, you know, using, you know, metrics that you can explain where the numbers come from, is the greatest thing that we can do in order to create a culture of acceptance, in terms of data analytics, in terms of data driven decision making tool. You know, sometimes we just make it too complicated. We put big fancy words around things and they don’t mean anything. There’s so many different metrics that we can use. You know, Harold Kirchner is somebody that I really admire probably one of my favorite authors. I know, it’s kind of dorky that I’m one of those people that have a favorite textbook author, but I do. And, you know, he talks in his in his books about how, you know, sometimes we overcomplicate the metrics, and we have so many different metrics for so many different things, that it’s just, it’s too much, you know, just to keep it down to a handful of simple metrics that people can understand that, you know, you can really communicate openly about is probably the best way to create, you know, a culture in which a team can fully embrace data driven decision making tools.

Vlad G 22:47
That is actually very true. And one of the things one of the first things that I learned about how to talk to executive suite when I was just starting in product management is go break Your whole analytics report, bring me a couple of KPIs that the C level would understand. major stakeholders would understand they don’t want to look at your Excel spreadsheet if they want an Excel spreadsheet that will ask you for. But if you when you come into the meeting, bring in top three KPIs that you care about, that they care about. And as a matter of fact, one of the things that one of the approaches in product mindset and product management is one metric that matters. So you’re not tracking a number of KPIs. You’re only tracking one. For example, if you’re developing an app, your product is an app on mobile phones, then you can safely say, hey, my one metric that matters is adoption. How many users I have on the app? I don’t really care about anything else. I don’t care how long it takes for a signal to travel back and forth from app to the server and back. What’s the response time? How long it takes to transition from one screen to another. I don’t care. Okay. about is how many users are using this. Because ultimately, that is my KPI for usability, right? If the app is not usable, my adoption will drop. If my, if my app is slow, my adoption will drop. If something is wrong, if there’s there any issues, my adoption will drop. So that’s the one that truly matters. And it’s also indicative of the underlying things. So that’s something you can communicate. And that’s something people understand and the key Deacon behind get behind it. And and this is kind of like, how you how we in product management, how we get our stakeholders to love and accept the data driven decisions. And when you have when you have a very loud executive, it’s really easy for them to overpower everybody in the room and say, No, I want this to be done. This is right decision, blah, blah, blah. But if you tell them, hey, you doing this because you care about whatever one metric that matters, your adoption from the periods is Apple, and this is what’s going to happen. If we do this, but this is going to happen if we do something different and it’s going to drive up the adoption, it’s really easy to communicate surely demonstrate the benefits or disadvantages of one way doing one thing over another.

Erin W 25:13
I’m a firm believer in dashboards.

I believe in at a glance, put, you know, a one page graphic in front of someone, and they should be able to immediately figure out what they need to know right now. Yes, of course, I would expect that they they glance over all of it and give all of it consideration. But you know, the highlights should all be right there. And so if you create a dashboard to summarize your information, and it’s 20 pages long, you’ve overdone it because there you know, who flips through to page 20. So if you keep it all on one page, at a glance, all of your you know, your primary metrics, of course, your supporting data, you know, some people will want to see it, but most people really just want to be able to have dashboard in which they can look at it and use that as a key decision making tool. You know, I think a lot of people have so much incoming information every day. And, you know, the details sometimes are too much.

Vlad G 26:12
I completely agree as, as a product manager who created and managed probably about six or seven products in the last five years. There were nothing but glorified dashboards. I completely agree with you. I completely agree with that. Because, yes, that’s what that’s what enables people who should be making decisions to make the right decisions. And yes, there are ways to preview things up. But ultimately, yeah, nobody, nobody, nobody looks at the second page of search results. Why why people would expect executives to look at the fourth page of our desk world. I don’t know.

Erin W 26:57
I don’t even think it’s really a dashboard now. Money.

Vlad G 27:01
Right, right. No, I mean, I’ve seen I’ve seen I seen that I’ve done the dashboards that were 20 pages long. There was a reason for it, and we were asked for them. But I still think it was the watch. I and I agree. This is one of the things that I agree with you hundred percent. Yet sometimes too much is too much.

Erin W 27:21
I will say it, it’s kind of interesting. We’re working at NASA because most of my teams the first time they see a dashboard or report near on unusual budget in which they will want to go through and check the math. They want to understand lawyer that information comes from how it’s devised. So most of my teammates will at some point, go through the data and, you know, make sure that they can mentally process as far as where that information comes from, they really want to understand it, and very, very few exceptions to that and NASA They really don’t buy into a dashboard. And until they’ve seen the data at least once after that they’re usually fine with it. But perhaps it’s just because I work with so many brilliant minds, and, and we’re all very mathematically driven. So, you know, they they kind of want to see, okay, where does that number come from? They don’t usually take anything at face value the first time. And what a gift to that is that I work with people that really value the data and the value the math, they want to understand where where this information comes from, so that they can best use it to make decisions.

Vlad G 28:34
Right? Yes, and it makes perfect sense to me. Again, not always, this doesn’t always happen in the enterprise world, or in the IT world. Sometimes. You have when you have enough data, those those cases, sometimes you just have to trust your data engineers that prepare that data that they built that is kind of like one of the challenges and one of the products I’m using Managing. I was managing last year, when we show you the results. And we argue that, hey, these results were produced by algorithms that were developed through, you know, blood, sweat and tears of all our employees and in the past 15 years, and then when they want to see the math, we can’t really show it to them, because, hey, it’s not. It’s not it’s not really math. It’s just, you know, specific algorithms and rules that we’ve come up with based on our our real world experience. But I get what you’re saying. And I would agree that, you know, once you’ve proven that your approach is working, then just look at the at the end result. So I, I’m just curious, just just spend maybe a couple of more minutes on this. So were there any cases or are there cases when you have to make a decision or you have to make people make a decision when there is not enough data because One of the things we’ve talked about before, in this on the show is product managers gut feeling. And it comes from understanding the business domain, understanding being subject matter experts in certain fields, understanding how things work, in market on the market in the market in a specific vertical, basically knowing, you know, having this situational awareness of things. So we’ve thought through this several times that we’ve kind of agreed that gut feeling Product Manager feeling, Product Manager gut feeling is a thing. My question to you is the guise of cases when you don’t have enough data, but you still need to have a decision. And how does that work?

Erin W 30:43
Even apps are set up in a way in which we have so many tools available to us and we really can can find almost anything. Whether it be from historical data and lessons learned, or from future projections, we really, we really have a lot of tools that are disposal in order to use in analytics in project management exclusively to provide that data, I’d say that the hardest thing in this environment and being in a research environment, especially from a scheduling perspective, is that you’re building things that have never been built before. And you’re doing things that are just so outlandish Li crazy in terms of, of what we’re building, one of the products that I was one of the projects that I was working on, was to build a nuclear reactor, they could power our future moon base or a trip to Mars. I mean, who would ever think in a million years that that would even be a thing that we’d have to think about? From a time perspective? How long will it take to build it? What partners do we have, you know, we have the Department of Energy and we have NASA and we have, you know, all of these different stakeholders available, all of these different contributors available and some of the world’s most brilliant, you know nuclear physicists working on this project? How do you schedule the build of something that’s never been built before with materials that have never been built before for use, it’s never been built before. And a lot of these things depend on lightbulb moments, you know, you’re talking about brilliant people, and you’re trying to schedule how long it will take for them to create the uncreated. And that would probably be the greatest challenge that we have is is Eklund Research Center. A lot of the things that we’re doing have just never been done. We’re not quite sure how things are going to go because, you know, once they get up into a microgravity environment, you know, everything paves differently in outer space than it does on Earth. So, despite all of the extensive testing and everything else, you know, how do you schedule that? How do you how do you schedule somebody lightbulb moment in which everything comes together? A lot of times, you know, you think it comes together, and then it goes into testing and come to find out, there’s some rework involved, or some some safety considerations that need to be worked out and revised. So if you can’t see those coming in advance, and the same way that you would say, if you were building a cell phone, you’re building a cell phone, there’s lots of other cell phones, you know, it’s probably going to behave pretty similarly, it’s going to have a lot of similarities in the software similarities in the hardware, in what we’re building, we’re building things that are just just so outlandish that they’re not even going to be operating in an environment in which any of the scientists are familiar. So so that would probably be the hardest thing is that, that you can’t see the report coming and it’s just, it’s really a lot of trial and error.

Vlad G 33:50
In full transparency. When you started down this down this path, it started feeling a lot like product development, and just hearing me out Not because, you know, you’re doing similar things No, but because of the approach and, and that’s kind of like one of the reasons why product mindset too cold in it because there’s a lot of industry disruption, there was a lot of things built, that weren’t built before. And as a person who’ve built things that never been built before, several times by now I can, I can assure you, I’ve been through, I’ve been through the pain of having to report a schedule of building a thing that no one in the industry has built before 90% of the company that I was working for me even believes it could be built. Imagine you’re not building a nuclear reactor which is something we kind of have an idea how to do but building an anti gravity device right? I read a lot of science fiction so go mind. Imagine doing that and everybody knows it’s impossible. Everybody Billy tells you it’s impossible. You’re wasting your time and and you have to come up against all these people. And not only Tell them, oh, yes, it’s possible, but also tell them, this is the schedule and this is what we’re going to deliver certain results. So I think I understand the pain. But I still, I still want to hear how you guys do that. Because I don’t think I don’t think you told me what the problem is. But how do you face this challenge? I mean, how do you guys do that?

Erin W 35:23
You just eat the brownie one bite at a time.

That’s the way we all do it. So that’s probably the same thing. That’s probably the same thing that you do. You just break it down into manageable pieces. And then as you gather more information, you break it down a little bit farther. You break it down a little bit farther and you just eat it one bite at a time until you make your way through the whole pan. And and that’s just kind of as probably very similar to what you do. You start off in big chunks and as you get more information you you further you know break it down into into the next level chunks to them. I think the formal word for it is rolling wave planning. But, but that’s just really all you can do as you get more information. you revise your plan.

Vlad G 36:09
Yeah, we’re eating elephants here. But yes. Whereas the brownies were saying how do you one of the one of the recurring thing is how do you eat an elephant? Piece by piece? Same, same thing, but not not the brown.

Erin W 36:24
I prefer brownies over elephant.

Vlad G 36:29
Right. Okay, so, obviously, the next question is, how does it feel when you actually make it happen? And I’m not deliberately not asking specifically about the nuclear reactor or something like that. How does it feel for you? It probably, I would imagine it would be different. Because we, in product management were more kind of more used to this feeling when you know, you can’t make anything happen. And we’re more used to saying, Hey, you know what, it’s not good. To work, we’re we’re experimenting a lot and we’re a lot more acceptable to be a little more accepting to dropping an experiment that it’s not working. Okay. Not gonna happen. Let’s find another way. How do you guys deal with it with consistent failure to achieve the goal? Like, again, imagine where you’re doing the antigravity device, and you’ve spent the vital year, two years, three years, okay, it’s not not happening. You can’t, doesn’t work. Or you spent two or three years and then you know, the the regular scheduled lightbulb moment happened and you suddenly figure out how to do it. What How do you what happens at that time? How do you deal with systemic failure or unnecessarily regular failure? And that one time when everything starts checking out?

Erin W 37:53
You know, I’m not one of the scientists on a project so I can only really speculate how they must feel about it. The credible sense of pride. I’m just a numbers cruncher in the background. So. So I mean, of course, I’m I’m just as excited when I hear about a launch. I’m thrilled to see which of our experiments were on it and what’s going up into the space station. And then, as things go, I’m excited to hear about what comes back. But one of the most delightful things about it for me, is that it is it several of our experiments are really applicable in a real world environment. For example, it’s not projects that I’m on, but it’s a project currently going on at f1 Research Center, is it’s an experiment about a fire extinguisher system on a very high level, because it’s not my field of expertise. You know, the overhead of fire extinguisher systems that we have in buildings in which you know, dropped a line out of the ceiling and the ceiling comes down and it extinguishes any fire that is in that room. Well, that wouldn’t work in space because nothing goes down right? So they’re working on a new hire extinguishing system for the International Space Station. So in the unlikely event that there’s some fire on board, they can put it out my husband’s a firefighter. So I can excited to share some of the data and some of the some of the foundation level of experiments. It’s actually just recently launched in about decode operation. I get so excited to share that information back with my local fire department. We have another one called Slayer. And they basically take tools that we already have and that are common Say for example, calf bar, which is used by the United States military all the time and safety forces, of course use capital or pretty often, and as well as some new materials that aren’t currently available in the American marketplace like yet, and they basically light it on fire and they drop it down. It’s huge drop tylar Tower, and materials go so fast that it simulates an antigravity environment. So they can kind of start to get a prediction for how these materials will burn in space. In, in contrast, of course, they burn them in an earth environment as well to have the full data set, so I’m able to go back to the fire department and say, Hey, this is cool thing. They lit this on fire and dropped it today. And this is what happened. And you know, the community as a whole, you know, my fire family community gets excited about it. And so just being able to talk about what we’re doing and find on earth application for some of these experiments that are going on, how can we use this? Not just in face, but how can we use this information on the ground? Like, what is this hold on us about what’s going on? I think that’s probably where my source of pride comes from, as well as hearing the wonderment from the people around me in which, you know, with having such a huge Research Center in the backyard, you know, as a community, we all kind of talk about what’s going on and the fun things that are happening there. And, and we get to dream together. And so I think that’s probably more where my source of pride comes from at this point than necessarily the way that the scientists themselves must feel about their experiments. I mean, I’ve seen people working for as long as my teams now have been working on a project for 11 years, and it’s finally about to launch and so they’re in kind of the 11th hour crunch and are working really hard and they’re doing amazing things. And so to see that source of pride and, and, and to have that, that pride in our community as a whole, it’s just really cool.

Vlad G 41:45
Thank you. Yeah, that it makes sense. And yeah, I descend to say we will, what do you mean by this? Wow, okay. That’s for sure. Pretty cool. Again, as I mentioned, I was A lot of sensation. I just have to ask this. And I apologize if it’s self care. But how far are we from from the anti gravity and you fast fast travel to, at least to the solar system?

Erin W 42:14
Oh my goodness, I have no idea. Well, I appreciate that you’re into science fiction. I have to admit that I’m not. I have never read. I’m going to tell you the truth. My mother in law is going to kill me. She is a huge fan of Star Wars. Huge fan. Like her whole house is full of Star Wars memorabilia, like the shelves, everything. Everything challenge is Star Wars. And I have never told her that somehow in my whole life, even now that I work at NASA. I’ve never seen it.

Vlad G 42:50
Yesterday was May 4.

Erin W 42:53
I know. I can’t even begin to tell you how many jokes I got emailed to me about May the fourth be with you and me me Really cute, but I mean, I’ve never seen it.

Vlad G 43:03
That is that is very believable. And I apologize for the reference. I’m sure you haven’t seen that. There’s this military science fiction series. He was running for 1015 years. Stargate is you on and there the main character is a is a US Air Force Colonel. He, his point is he doesn’t like science fiction. He likes Simpsons, but he hates science fiction and, and he goes through the Stargate to other galaxies to other worlds explores other planets and he hates science fiction. That’s kind of the thing.

Erin W 43:43
I mean, I kind of feel weird at this point in my life, not having seen it. But that said, now I work in an environment where I live it every day. So it doesn’t, you know, when I was one morning I would watch the movie and think it’s not realistic. Or that, you know, I live it every day. So it just probably wouldn’t be that much fun for me. Why would I want to watch a movie about daydreaming about getting to other planets when I don’t have to daydream, and I work in environment every day with people who are trying to get to other planets. Like, it’s not a daydream, where I work. It’s just a reality. This is this is what the goals are. And so I mean, of course, I understand it’s different. But at the same time, I mean, I when I have a dream, I’m doing it at work with people who are trying to make it happen. So it’s just it’s kind of fun the way I have it, and I don’t want to ruin it.

Vlad G 44:42
Oh, yeah, definitely. No, I completely understand. And believe me, I appreciate what you guys are doing. Making my childhood dreams come true. Even if I don’t get to go to space, at least I see people go into, not far, far space not I mean, even Moon or Mars, I mean, the whole solar system. And I do appreciate it. I do appreciate the work that you’re doing. Bringing that a little closer. One one other one other questions that I had. And I want to go back to the the pre one of the previous things that you’ve said. He said, You said that you guys are working on projects that collect the huge number of data. What What is the overall sentiment around the data? I remember you mentioned that there is your kind of like final product that belongs to one of NASA is a public organization and one of the main shareholder stakeholders. So that’s who the data belongs to.

Erin W 45:50
Yes, most of the data is is what I consider to be the final product. So once an experiment is launched, and the information comes back down to earth, and gets ordered agonized and final reports get written, that’s considered the close of the project on my end. And that’s kind of the that’s the product that we’re looking for. That’s the ultimate goal. And all of that information alone, not all of it everything that doesn’t have a national security interest is is the property of the American people and and most of the information and most of the experiments that we do, you know, that information is accessible to the American public and not a lot of people know that. It’s not, not my information. It’s it’s the information of our country as a whole. So, you know, if there’s an interest a lot of a lot of times people will submit requests to get access to the information. And that’s not just the common citizen, but also companies interested in the in the results of our research have access to that information. We oftentimes partner up with other countries. I mean, a lot of people are familiar about the partnership with SpaceX and Blue Origin, especially now with our human launch campaign. That’s very exciting with the partnership with SpaceX. So, you know, any of our stakeholders have access to our information. And those stakeholders aren’t always just simply finance ears. I think in product management, I always imagined it coming from a marketing perspective in which, you know, your stakeholders were were of course, the people buying the product, but also the people financing the creation of the product. Whereas it’s a little bit different in a research environment in this research environment, because our stakeholders might be other companies that we’re doing business with, or other countries that we’ve partnered with, or the American people as a whole, just your average citizen who’s interested in the in the results of what we did. Everyone has access to that information. As long as like I said, there’s not a national security interest.

Vlad G 47:57
Right. Well, that part is understandable. I mean, I was Imagine myself as an average american citizen interested in really cute wallpapers from mess a website. Since you’ve since you’ve mentioned since you’ve mentioned and thank you for bringing this up, since you mentioned product management, just to clarify, it’s not just marketing. The product management is actually responsible the product management role or a product management discipline is actually responsible for a full lifecycle of a product that begins with ideation, hey, we have this cool idea is there something to be had there then you have experimentation, prototyping activities, where you try to build something that makes sense and try to build something that has merit that can be profitable and where value can be extracted from and then you have a specific point in time when you make a decision. Gotta go no go decision. Do we have ability This or we abandon this as this is this has this as a potential, then you move on into actual development of a product. And then you launch the product. And that’s exactly when the launch word is being used. You go to market with the product, and then you keep supporting the product, the product keeps evolving. One of the examples they keep bringing up about the product lifecycle, or product as a whole is Microsoft Office, it’s a family of products, each product does something, I’m sure you’re familiar with Office, and each product does something and if you if you if you’ve seen it long enough, you’ve seen how we’ve evolved it certain things come in, certain things are being deprecated certain things are improving, but the product leaves on and and and the company that develops it or the team that develops it keeps their hands on that same product, nobody’s walking away from it, once it’s on the market. Actually the opposite It happens once they launch the product and throw them on the market. And you start collecting the feedback from the market, you start applying more effort to make the product better. Usually the since you’re you’re doing the math behind behind the cost risks and schedules. What happens to the resource plan after product is on the market? What happens is the whole resource plan is reassessed. Now having a feedback have recently started receiving this continuous feedback from the market, we can now say, Oh, hey, you know what we have, we don’t have enough people, we need to double our team. And that’s usually what happens. You start seeing teams growing and product being split into specific capabilities where individual human can work on a specific capability rather than on the whole product. Again, as an example, let’s look at the Microsoft Word. There is a spell checker special checker is a capability that is called separate teams working on outside of Microsoft Word. So that that’s how the split happens in software like behind the scenes.

Erin W 51:13
Okay. The only difference I see there was an project with project management is that one of our project closes, we can use that information, obviously, you can kind of be a building block for our next projects, it might that information might lead to our next project as opposed to an iteration. I mean, I have seen cases in which case there have been iterations or you know, further developments with the data on the same project. Kind of like a like an option. If things go well then we’ll do this next. But for the most case, most of the time in the scientific community, the results of one data, batch or one experiment will end up leading to more experiments later down down the road. But that would be a new project.

Vlad G 52:05
Correct. And building up on my, my own experience being the project manager in IoT world, it’s very similar in the it. So you have a project you have school project, you have budget. Once it’s over, it’s done, nobody comes back to the project projects closed, eith will may and probably will lead to different project using the information that was collected from this one. But it’s just like you said, it’s going to be a different project. And the reason why product mindset showed up to the scene and kind of started to go over in a key is because it’s a very inefficient way to build software, project by project. It’s a very inefficient way to build software, because there’s a number of reasons. One of them is you may lose financing for something, at least for a period of time. You lose financing for people Define to continue developing the product, or an application or software application or something. And in that time, you need to make rapid changes, but nobody’s available because they don’t have developers who don’t have money to pay them. And things go south with product mindset where you have this concept of continuous lifecycle of a product where work on the product doesn’t stop. It’s just another iteration, the other iteration and iteration, but the product leaves on. That doesn’t happen. There might be adjustments to the budgeting, there might be adjustments to the scope it adjustments to the resources available, but the product moves on until it’s it’s deprecated until it’s no longer needed, until there’s no longer value in the product.

Erin W 53:45
So on the user, and I would see that as being like when one of my affricates an update and there’s new features available, or fixes and things like that.

Vlad G 53:54
Correct? Correct. These these updates or bug fixes or new features are available because the product leaves on. Once you stop financing product effectively dies, people stop using it or number of issues with the product increases, because you know operating systems evolve, the things evolve and you no longer can sustain. You know, the product doesn’t doesn’t work anymore. I’m trying to come up with with a good example off the top of my head. I can’t think of anything except Windows Phone. So I apologize.

Erin W 54:28
But it’s a really good example. How about like AOL? AOL was something that had lots and lots of iterations until its life cycle finally ended although I admit my mom still has a while email.

Vlad G 54:39
I think I still have it. But it was, it was my first one. My first one was Yahoo. But it they’re still alive. They’re just in this hibernation mode when were you still have certain things working Windows, Windows Phone is really good because they stopped making them. They stopped making new versions of the software for the phone and they stopped making the phones themselves. There’s no more Windows Phone, you can’t unless it’s something from 2011 you can’t really buy it anymore. There’s no polls available that that’s why I brought this into the picture. They released it, they were building certain capabilities into it, they were supporting it for some time and then they stopped and then you know, your your regular apps stopped working one after an hour after another after another. Like I had my Windows Phone. It had really good camera. I do have a hobby of photography. So I practice photography with false cameras everything and had really good camera and I used it. I use it a lot. And I started noticing that Facebook’s not working anymore on that phone. Instagram is not working anymore and of course so I even if I took a picture, I now need to drag the phone back home hook it up to the computer to download the images. Things stopped working. And at some point, it was easier to use another phone than to keep using this one because it became literally obsolete. So that’s, that’s why I thought it would be a good example.

Unknown Speaker 56:13
Okay,

Vlad G 56:14
so that’s that’s your life cycle. It was a very hot phone with a startup. Everybody was interested, everybody was buying it because it was super duper camera on it. And then you know, it was supported for some time and then eventually died out. And that’s, that’s, that’s how you can think about the product lifecycle. So yes, it’s definitely different. But then again, in, in the, in your world, maybe there would be placed for a product manager product approach. I don’t want to say product management, mindset or product approach, who probably did better terminology of how to iterate through different cycles of product lifecycle.

Erin W 56:55
But it sounds like in our core, a lot of what we do is similar in terms of building a business case and, and keeping track of cost schedule and risk and things like that. It sounds like there’s a lot of similarities between the two pack.

Vlad G 57:08
Oh, absolutely, yes. And if I had my co host Irina, on the on the on the phone, he’s just not her out. She’s She’s in the in the Europe so Slifer she was, she would actually disagree with me. And that’s why I love having her because we always disagree and she puts a different perspective on things. She would say that inside every product, especially inside every enterprise product, there are different work streams that we treat this project. So projects and products are literally go together every time and and i i slightly disagree with that. But I want to make sure that I talk to that point, since she’s not here and I think it’s kind of like it’s worth having that point of view voiced out because in many cases when you start splitting words It needs to be done to build a product. For budgeting for scheduling purposes, it’s easier to wrap it in a project in a project form rather than in the product. There are not that many companies that completely abandon the project mentality when they building software products, and will full on with product mentality. Most enterprise companies still keep project mentality around they just have them kind of like a queue or conveyor of projects. When one answer, the next one immediately begins. So there’s no gap in coverage and financial coverage and resource coverage. It’s just like one project after another after another. Okay. Does that make sense?

Erin W 58:42
makes total sense makes total sense. You know, when you’re talking about your, you know, business intelligence, a lot of the terminology that we’re using is very, very similar. Things like just the lifecycle structure is a little bit different.

Vlad G 58:55
All right, so we’re almost at the hour. I wanted to ask if you Have any questions for me? At one? This is one of the things that we’re doing. We’re asking a couple of kind of standard questions. One is, how do you guys work from home? Are you working from home? How does the current pandemic affect you? And the other one is, if you have any questions for me, so feel free to start with either one that you like.

Erin W 59:22
Sure, I’ll share it with I’ll start with a pandemic. Yes, I’m working from home. I’ve been home now almost two months, I have been very impressed with how well NASA has done with moving all of us outside offsite, coming up with a framework with some with some metrics that would establish at what level of a lockdown we weren’t going to be subject to, as this moved on. They were very, very early in the game. They were very, very transparent with with how they were closing the centers and what decision making what data they were using. In order to make their decisions and what we could expect as as employees moving forward, so I was grateful for that. The first week or two after we all moved home, it was a little bit chaotic. We all adapted to teams, which was new to many of us. You know, as well as juggling just the life that was happening. You know, many of us have family, children at home. And you know, a lot of a lot of my colleagues have both, you know, both parents working from home with toddlers on the split, so you’re sharing a living room with, you know, your spouse who’s also on calls and your toddler. And so so phone calls got interesting there for a little bit, but everything seems to have really calmed down. I feel like as a whole, we’ve adapted very well. I find that that work has gone on pretty seamlessly. I don’t see very much disruption in my day to day work responsibilities or routines. The only difference is that I don’t have a commute anymore. On a more personal note, you know, and I have two teenagers at home. And so they’re able to do their remote learning somewhat independently, they don’t need me hands on the way that that many of the people with young folks do. As a family, though, you know, once the initial Shaka and all pass, we all kind of have have really grown a lot closer. I’ve spent more time with my kids in the last month than I think I have in years. And so overall, I don’t mind this one day. I mean, of course, I’d rather nobody be, you know, sick or dying. And I’d rather not have this fearful thing out there in the world but, but on a small scale in my own family. There’s just been a lot of good that’s come from this. And I feel like working from home has really given me the opportunity to juggle my life and manage my life a lot better. So that I don’t have to be either at work or at home all the time, I have a little bit more flexibility to kind of go back and forth and, and prioritize my life accordingly. So I have no no negatives in the telework situation at all, I’ve really, really been grateful for it. Awesome. I know that’s an unusual perspective, most people just can’t wait to go back to the office. I do miss the social aspect of it. But I feel like we’re all using technology in a way that is blended to that. In fact, some of the other schedule analysts that I work with, we set up like a Wednesday night, happy hour, you know, when we were kind of getting on a, you know, a call after hours of video chat after hours and, you know, having a refreshing beverage together and just kind of socializing. And I don’t think we ever did that before the telework situation. So I’ve gotten to know them a lot better. And I’m spending more time with, you know, some of my more distant colleagues than I did before. For so I think we’ve all embraced it well and and I’ve really enjoyed the the relationship building that’s come as a result of this. I miss the socialization I miss just seeing people smile. But that’s, that’s, you know we do our best with technology. It’s just an unfortunate, you know, unfortunate side effect is that we don’t get to see each other’s smiling faces very much anymore

or we do but it’s only the video and I don’t think that’s really the

Vlad G 1:03:34
nature of my work is I almost always work remotely and I’ve been working from home for the past couple of years, aside from some time that I wasn’t on the assignment so for me, it’s a little bit more. I’m a little bit more used to this, but I’m happy to hear that. You find positive things in it. I I’m a big proponent of working from wherever I’m a big proponent of working from home You know, remote, anything, doesn’t matter as long as the person gets things done. So I’m really happy to hear that you guys have to have it figured out and hopefully

Erin W 1:04:13
even I think overall we really are getting gone if anything I think we’re getting more done now than we did before. As strange as that sounds. I feel like we’re more productive as a group and we’re accomplishing more at home than we really did in the office on a variety of different levels. I, I wouldn’t be hesitant you know, of course if they called us back now I would be hesitant just because of this eg situation, but I have to be honest that this has just been you know, really a gift and I couldn’t imagine, you know, that, that this wouldn’t be a more embraced lifestyle moving forward. I think I think that probably companies are going to find out that he we really are getting stuff done and our costs have gone down a lot. Our employees are a lot happier. So I couldn’t imagine you know, a world in which this didn’t become just our new normal.

Vlad G 1:05:07
I completely This is another thing that I completely agree with you on. I think it’s it’s the new It’s the new normal, working from home or working from wherever. Because you are more productive you don’t dread the commute and me being from New York City, believe me, I did dread the commute them and if I pretended I didn’t, I I’m not enjoying. I wasn’t enjoying. So I do think we get things done. We get more things done and we get things done better overall, provided we can work from wherever we like.

Unknown Speaker 1:05:43
So

Erin W 1:05:44
that’s great. Yeah, I’m sure to commute there’s a lot different. I mean, where I live, um, it only takes me 23 minutes to get to work. So it’s really not that big of a it’s not that big of a commute. It just now that I know that we can do all this remotely It just seems wasteful. And so it’s a waste of, you know, time albeit not as much time as what you would be wasting. And just a waste of resources, a waste of fuel and wear and tear on my car and everything else. Not to mention time away. No right now, no, my kids are teenagers. So they know when when the office door is closed, monitor meeting and leave or be. But they also know that in between meetings, I’ll pop my head out and check on everybody and make sure everything’s moving forward and going the way it’s supposed to. So you know, I just don’t i don’t see very many negatives. And I would probably feel differently if I had a baby at home or if I was at home alone and didn’t have a family that would probably make me feel differently about it.

Vlad G 1:06:48
Right or, as as I have recently learned, if you had way too many people working then you just don’t have enough privacy or enough space to take the calls and not being disrupted during those days,

Erin W 1:07:02
yeah, extra, I’m really lucky that I have a home office and I have a door closed. So and it’s also separate from, you know, my otherwise living space, not in my bedroom. It’s not in my dining room. So it’s one way we work is contained. And that’s kind of my way of keeping my sanity. Otherwise, I think work just encroaches into your life and life encourages me to go to work, and it’s just kind of chaotic. So having an office has been a huge advantage for me. But, but overall, you know, even without that, I don’t know if I’d ever want to go back. Yeah, I guess I’ll have to cross that bridge when we come to it. And what was your commute, like in New York City,

Vlad G 1:07:43
so in New York City, it was about an hour on the subway with one transfer, which wasn’t that bad. On in New York City, not bad. Or you have to deal with pack trains and in rush hour, and all that Prior to that I work outside of the city. I live in city I live in Brooklyn, I worked outside of the city and was hour and a change drive in traffic back and forth. So it wasn’t. It wasn’t as much the stress of driving it was more of a stress of wasting a good hour, hour and a half each way. hour and a half each way of your life where you can’t really do anything at all. The only thing I have available was to listen to audiobooks and podcasts. That’s really all there was there for me. Otherwise, it’s two to two and a half, three hours lost everyday just driving.

Erin W 1:08:39
Yeah, that’s so wasteful. Like there’s so many things you can probably do now that you weren’t able to do before recreational things, you know, things that are good and healthy for you. Well, at least I have now I have

Vlad G 1:08:49
to have a podcast.

Erin W 1:08:52
So there you go. I’d say the only struggle in my house is that my kids are constantly streaming Whether it be their schoolwork or YouTube or whatever it is, they’re doing their video games. So there’s never enough internet to go around.

Unknown Speaker 1:09:11
Yes, there’s just never enough internet.

Erin W 1:09:13
So and I, from what I hear, that’s like a countrywide problem. We’re all dealing with that, as we have, you know, sometimes four devices going at the same time streaming different things. And, you know, for me, my work computer, I’m going on VPN, but then I’ve got teams going, and I got email going. And so, you know, I’m constantly using, you know, internet as well, and the children are two because they’re doing their schoolwork. And of course, my husband is too because life happens. So it’s, that’s probably been the biggest, the biggest struggle is just that there’s not enough internet to go around. But I, I imagine our infrastructure will catch up with that eventually, and probably not as big of a problem and

Vlad G 1:09:55
it’s still it’s still he is a little bit I mean, Depending on which level of service to subscribe to, but it is a problem because isn’t one of those commercial things where you oversell your capacity because no one is ever going to use it to full capacity, like I purchased shortly last year, and I purchased the highest level possible, or the finest Verizon FiOS is almost a gigabit connection. And it still chokes off when more than two people are streaming. And as a matter of fact, I have instituted this rule in my meetings whenever I’m in the meeting, and I’m presenting, I’ve always I give them five, seven minutes to do the video sharing and then I say I tell people, hey, not everybody has the fastest connection in the world. Kill the video so we don’t have interruptions in audio and presentation. So please, you know, shut off your camera. It would free up people’s bandwidth to see other things in it generally kind of work. Now you can add it to a shot. If you want But how is the pandemic situation in city? How is everybody? Not? What is poor conditions like right now? Well, they’re they’re two camps in New York City, specifically, two camps those people want one camp is very strict. And as soon as you pop your head out, they go hysterical, like stay home, with all the explosives in the world attach to it. And the other camp is, there’s no pedantic or we don’t care. And I occasionally get out on the supply runs or just for a walk. Because they can’t, you know, sit in the chair 24 seven, and I see a lot of people going about their business like there’s nothing they will do is they will dogs, they just, you know, taking a stroll, not wearing a mask not ignoring six feet separation. So they pretend that you know, it’s over. And that’s basically what was happening in your city.

Erin W 1:11:58
Well, I think that’s probably what’s happening right now. Were we honest? That’s exactly what’s happening here. You have two camps. Those of us who were were pretty strict and those of us who are just following on with life so this has happened and they’re just like, resentful of ours are closed.

Vlad G 1:12:13
Yeah, yeah. I mean, hopefully, it will, it will start being better soon. I’m not I’m more I’m very worried about people dying, especially in New York City. We’re all here. We’re all affected. I know people who were affected I know people who passed away because of the virus. But I also worry about if we prolong the strain on the economy, although we may not emerge back may not survive the way we were before. So coming back to the they may not be coming back for because it would be no work because the economy is gonna start crumbling under stress. That’s that’s my concern.

Erin W 1:12:55
I’ve been concerned about the economy as well, especially recently, as pies has become, you know, a bigger concern. You know, today in the news, they had Wendy’s, you know, the fast food restaurants 20% of them are quotes because they can’t compete. And I just never imagine a world in which that would ever even be possible in which food supply would become a problem in the United States. I just never, I never even imagined that it was possible that I would go to the grocery store, not be able to get whatever I want. So that it’s a bit of a shock, you know, to the American public. I think it’s probably a similar feeling to how we felt about 911 in which, you know, wars happen and bombings happen and terrible things happen to innocent people, but it doesn’t happen here. And then all of a sudden it did. And so that’s kind of the way I’m looking at the food supply right now. And I couldn’t imagine what it’s like to be in this city, where you don’t have access to you know, Grow Your Own or, you know, I live in a I live in a state But of course, it’s a much smaller city outside of Cleveland, but I have enough land that I have a, you know, a big garden and, you know, my husband hunts and and so you know, we have some of our own supplies and I couldn’t imagine being in a world in which that wasn’t even a possibility.

Vlad G 1:14:17
Yeah, I mean, I didn’t

Erin W 1:14:19
ever imagine if it all happened here

Vlad G 1:14:22
coming from Soviet Union, I did experience that and I don’t want to scare anybody but it starts looking very much like it. We had, we had pretty much old stages of, you know, just not having food all the time. But it was possible to get it than not having food and not being able to get it because there’s no supply of food coming into the city. And at some point we had the voucher system or card system when you had a card that allows you to get you know, specific items specific numbers. So you can only purchase, I don’t know, two pounds of meat per week. And that’s it. That’s your norm, kind of kind of similar like during World War Two people had in in the occupied or in the cities where supply of food was very limited. So they only were able to purchase or even get without versus there’s no money or no money you can get it. You can only you know, use specific allotment of of stuff. Again, I don’t want to scare anybody, but I really don’t want this to start looking like it.

Erin W 1:15:34
What do you see is the greatest difference between product management and project management. In terms of methodologies overall,

Vlad G 1:15:43
I think I think it’s the is that difference that we’ve discussed this the difference between project is having this finite timeline. You start you build your finish product, you’re having a Life, which is why we call it a life cycle. Yes, it’s kind of like a human life, right? You, you you’re born, you learn, you grow up, you get into your prime, you plateau at some point and then you start, you know, you’re you start descending down, you’re the products have become outdated or or obsolete and eventually the products life has ended Unless, you know, you you come up with another idea or another replicability of the product, but by then it’s probably, it’s probably a child now. It’s another product is so that’s a new product that was built on top of the previous one. I think that’s one of

Erin W 1:16:46
the big, very similar, very similar, I mean, obviously, projects have a life cycle too. But it sounds like the difference between a project and a product is that the product would be managed through its operations. until it becomes obsolete, as opposed to a project. Once it gets to operations, then it goes off to the operations people. Sounds like it would be like the project management methodology might have a more like a division between development and operations where as product is development and operation,

Vlad G 1:17:23
core development never stops. Yes, that’s Yeah. Yeah. You nailed it on the head. Yes. The development okay. doesn’t stop until unless product is dead.

Erin W 1:17:35
There you go. Now, I learned something new today, too.

Unknown Speaker 1:17:38
glad I could help.

Vlad G 1:17:40
Any, any other questions. Anything else? I can? I can answer. No, no, I’m good. All right, then. So thank you very much for being on the podcast. Really enjoyed the conversation. Hopefully we can do this again, with slightly less technical issues that we had before.

Erin W 1:17:58
I would look forward to that.

Vlad G 1:18:00
Thank you very much Aaron would. Thank you. And that’s that’s it. That’s all there is for today. Thank you.

Erin W 1:18:07
Thank you talk to you soon. You’ve been

Vlad G 1:18:12
listening to the real world product management and I’ll be your host lon Drummond. Until next time,

Real World Product Management – Episode 10

May 5, 2020 Vlad Grubman

Irina and I are talking to Nikolai, a design lead with a lot of experience working with various products in healthcare and finance. We talk about the examples of when making something happen isn’t as easy as it seems, having education as a prerequisite for a Product Management role and a few more things.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Hello everyone. This is another episode of the real world product management podcast and we have Nico he solid line here today with Irina. Akiko I can you please go ahead and introduce yourself?

Nikolai 0:26
Hi Ron. I’m Nikolai Chesalin and thank you for invitation, I work at EPAM as a systems designer, but my actual role more like a product architect, and today I want to share with you some insights from my experience. During the time of my career, in napalm and before working on different product products, I will cover some insights from mobile applications for Like fitness tracker and like application, healthcare application, a lot of products related to financial industry and some innovative things that we build the inside the EPM system.

Vlad G 1:16
Cool. If you don’t mind me asking, Can you start with healthcare because I think this is one of the more relevant of things that we all care about right now, given the current situation with the COVID-19 pandemic. So talk to me about your healthcare related products. What did you do? How did you do it? What were the challenges? It was very interesting experience. It’s one of my favorite products I’ve been working so far. It was a mobile client for the large healthcare institution in New York,

Nikolai 1:52
probably one of the largest Cancer Center in New York, and they had a problem. Actually, I will describe to produce We’ll start from the mobile, but it was like an open door for another one. So they struggled with the situation they have. They had like a portal, tons of clients, tons of users. But they realized that people don’t use features that they had as a product. And furthermore, they need to push more data and make sure that people gain this data at that moment for them was crucial to make sure that they will have like remote experience like when you check in with the doctor, check your prescriptions, etc, etc. Today with all the entire situation of current team, it’s it’s become essential, but that that moment, it was like six or seven years ago, it was kind of a new thing. And I just want to give you some Overview period of that time healthkit wasn’t introduced by Apple yet. Think that we use them that product for checking real insights from end users like testflight wasn’t bought by Apple. So a lot of things that today is the standard. They were completely new. And it was interesting, but we had problems like every single day, even everything was like, designed properly, I would say appropriately. We had a standard, good process of cooperation, etc, etc. we resolve new issues almost every day why it’s happening. And this is probably one of the reasons why I don’t believe any senior specialist can say hey, and very senior I will be able to ideate and bring product to life without any serious problems. I don’t believe to this because if you deal with more or less in your thing, you have a lot of data. And you don’t control the data and correlations around this data. And as a result, you have a number of issues, for example, how you can handle different business logic around things. And it’s not only how to find the proper way to handle the things. It’s also always how to do this on a computer level coding, designing, etc, etc. but also on the business layer, how people transform or interpreted different rules. So let’s say we have an exception that this temperature way far away. Bob normal, the the regular program say, hey, he had 200 degrees temperature, what’s the problem? It’s not a division by zero, I can show it, no problem at all. For doctors, obviously something wrong, it’s kind of impossible, almost impossible situation. And for the business, I mean for people who are running this institution, it’s even bigger problem because they have a number of obligations, legal things, etc etc. And as you can see, very small thing can

I can trigger a much bigger amount of work just to resolve a simple situation with validation of one single parameter. Obviously, not everything so bad as this one. But this is a typical situation where you need to find a quick solution. How to deal with this on all layer. And I can say that it’s it’s the case in many situations, but it happens very often. And in reality, what I believe when people say, Hey, I don’t know all recipes for all kinds of situations, but I haven’t experienced how to dealing with them. Product Management as many correlated areas like user experience, for example. Very interesting can the same time very difficult areas because when you come to interview in any organization, expectation, on the other hand, that you already have some serious experience, and most questions, they came, hey, Okay, tell me about your experience dealing with this thing. And you can have a perfect indication Good, good score in your diploma but practical experience. It’s something that is essential. And these type of things back to my example of the healthcare, healthcare application. It’s very interesting how people who work on a product, try to observe it from different angles. And here it’s very important, I believe, to establish governments of resolving all kinds of questions or discussions at some point. Because if you don’t have established governance, of running processes, or even some meetings, eventually you will came to the chaos or just stronger voice will win all the time. So this is a kind of experience. I’ve got working on that Healthcare application, make sure that you have a governance, don’t be afraid of problems. And we had a problems like everyday. And I’m talking about just things related directly to the product. technical issues a separate topic, which is also interesting, but not a part of this conversation.

Vlad G 8:23
Right? So you’re saying you had a product design problems around this product? Or were they more around agreeing to what the design decisions would be or agreeing what the product decisions would be? Give me like a couple of examples of what these problems

Nikolai 8:40
Okay, a couple more examples. For example, when you have a results, and let’s say, you have a, your cardio results, right, EKG results, and this was a part of that product that you have Big. It’s not only about the product decision, it’s also about details of the product decision. And let’s say on a very high level, somebody says, Hey, we want to provide some overview of AKG to our clients to our patients. No problem. Can you do this a mobile application? Yes. And this is a kind of a high level decision. In reality, when you want to make sure that it will be feasible in the product, you need to go much deeper. And when you go deeper, you realize what kind of problems you have, for example, what kind of definitions you need to provide, to make sure that you’re okay that you are in compliance with the standards that you like. text messages like please, this is just for you and your doctor, security things how you can deliver This thing. And when you start going deeper, it’s important to understand your border border so that you don’t want to go too deep. Otherwise, you will be just working on one perfect feature of the product that will never be able to release. And for me, it’s about balance, how you can deal with it. Let’s continue with this example. So I want to have AKG in my mobile application. Good. We have some sketches, designs, workflows. What are the typical workflow for the user? As I mentioned before, we even had a chance to run like early test tests. So we use a test flight. So we have a small group of adopters, so alpha testers, of applications, so the good thing that we were able to gain Feedback very quickly, but run after that we realized them you when you deal with the data, you need to have more than just a group of 100 people. And this is a one of the important takeaways. Then when you work on the product with the data you need to GM and defy common rules and validate validate more or less common theories, because it’s not always good when Oh 100 people say that works perfectly. What what happened with another 100 or another 1000 and here we are, slowly but surely jumping in the area of the scalability of the product. And this is a problem. I believe that many applications many products facing when the starting on some Most small small use case and rapidly grow. And here it’s important to understand the difference between scaling in terms of base of users and scale in terms of data that running through the product. And how this data will affect results that we see through the product.

Vlad G 12:25
I think I’m sorry to interrupt, I think you’re you’re actually distinguishing two different things. One is the actual scalability, scalability as we will understand it, and the other one is scalability of adoption. So the more people adopt it, yeah,

Nikolai 12:42
exactly. Sorry for a computer probably. And this is the will the perfect example, what kind of problems you will have, because I believe a lot of people who are dealing with a product so on different stages, they have a situation okay? We pass all our the checks create testing, positive feedback from initial wave of users. But when we can predict something, and, again, life science or healthcare products, for me means a lot of data and a lot of regulations. And we have similar areas like geolocation. It’s not something that is very restricted, but very similar problems. When you bring product to the life, and even initial feedback from relatively large group of positive, just in couple weeks, you can have tons of negative things. Because once you have the bigger thresholds of active users, results and feedback can be completely different. And I just want to give you a couple of well known examples that are related to my situation with health Application good Apple Maps, the initial intro, introduction of Apple Maps, it was something like big revolution and initial users, they the results of the initial test focus groups, etc, etc. was super positive. People were very optimistic. But we should remember that the most of these test users they were from Bay Area and berry initially in FMS were very good. But when people start using this apple map on a regular basis, just in couple weeks, a number of negative feedbacks, it was outrageous. The same situation we potentially could have with any application, especially if life science, when we be very cautious how we define the process, how we define the future, what helps and not us.

Not going to tell only negative or very scary things about being involved in product development. It’s how you can call our significant amount of negative scenarios. And there are a number of checklists, the best practices what you can do in a specific domain areas. And this is where the the statement I’m booting this domain area matters. So when you become when you real subject matter expert, or you know, good people who can help you with this. This is a crucial moment where you need to connect this people.

Vlad G 15:48
Yeah, I kind of agree with that because a lot of times people think that it’s easy to figure things out and a lot of times they think they know, but they don’t And as a specific person who built if you healthcare products myself, I can totally relate as a matter of fact, I’m actually curious. I was curious to hear your your experience in the European Union. Because in my back of my day when I was building health care products, we had two separate UT teams won, the team was strictly focused on health, health related data to make sure we’re not having patients waiting 17 grams or, like in your example temperature of 200 degrees. So that data makes clinical sense. And the other UT team was a focus on user experience. So they didn’t care if the temperature was whatever or the weight was, whatever. All they cared about is that it shows up in proper places with proper notation with proper labels so that when doctor is used to seeing temperature in you know, I don’t know, top right corner always it’s Suddenly moved to be swapped with a pressure blood pressure in a, you know, bottom left corner so that if he’s looking and it’s, you know, it’s those things that are really important to people are out there, they’re out there, they’re truly users, they’re not going to give you ideas about how to make things, right, because they that’s not where the mind is. And it was really interesting to hear how you guys handled it.

Nikolai 17:28
Yeah, and just to summarize, I want to go through the different layers of my particular case. So our audience can imagine what could happen with a particular case in vape products. So as I mentioned, EKG from technical perspective, good data, draw the illustration, couple nice functions. That’s it. Let’s try to look at this all from the product perspective and your responsibilities as a product. Manager what you do? What kind of question you ask, do we have enough data to really be able to provide the data or the things how people will be able to read this data and more important questions, what it gives to our end users, what kind of failure what kind of cost of operation for us is the owner of the product that will be how popular we think. And all of this question is related to a specific area, like marketing, financial analysis, etc, etc. So the same thing, but it could be relatively easy from technical perspective. And from the product perspective, it become as a separate I can say like a separate discipline, but here even one feature could be extremely complex. When you observe this feature from the product perspective. And of course, when it’s quite complicated chances that you will fail, at least in the beginning, extremely high. So

Vlad G 19:12
let me let me interject here and ask you this question. Because I think I think it’s an interesting thing to understand. And it’s not a tricky question. It’s really about your experience from the previous some of the previous episodes, we’ve talked about the data driven design data driven decisions, or even you know, product managers got feeling from your experience, only that we all agree that you know, doing things by the book is not always the best way to do things. What is the way to make decisions about building features in the product or about ability products, like if I like this example, because you’re talking about features that sound easy for so if you do burn ization based on how easy it is versus how much value it is it probably would be it would be picked up. But you’re saying there might be, you know, dangerous, dangerous waters up ahead if you do that. So from your experience, how would you decide? And I deliberately say decide not prioritize? How do you decide what to build and when to build it?

Nikolai 20:19
And I want to kind of divide my arms for my personal opinion and reality because I believe for our audience, it’s interesting, not just one or two opinions, but like some statistics from the reality from our experience working for different large companies. So from my experience, I will go like 80 2080 based on the facts, like data analysis, marketing analysis, some research, deeper search results, probably user’s cognitive results, etc, etc. I still keep some room for for the, for the hatch. So what my guests tells me based off of working in a specific industry, or dealing with things, because I’m, I always remind myself when we build product, it’s always more than just set of features. When I build products that for a huge group of people, it’s important to keep in mind that you need you need to build a conversation and community around this rather, if you will not be focused on this and will be focused only on the features. You have a great chance of that you fail here, because product could be good done by the book, but nobody will use it and we know these examples from from the world and we know also some examples in our company, then one product become more Popular than another one. And it’s relative. So my, my personal opinion 8020 and 80. It’s like cold analysis and data, and 20 just emotions. It’s more like your gut feeling, what will work and what will not do? So this is the first part of my answer. Second part of my answer reality. In reality, you can come up with the perfect justification of the decision. Why is this feature should work, but it’s important to understand what is your weight? And what are the who are the players on the table? who make the decision? Is it another single ultimate person who made the final decision, or it’s a kind of a conglomerate of people who put the like a number After that you have kind of a process, the threshold of voting should you go with this feature or not? And fortunately, and unfortunately, in many cases, the ultimate voice is ultimate decision is based not on your scientific research or even on a hunch of the previous 2015 years of experience, but it’s based on something that completely irrelevant to this product for some company strategy, or some personal preferences, and that is why we can see some unexpected things in the product and verify often we see them big companies and not only big companies. They fail with the obvious things. Because

even if I will not work in the product management area, I will ask why they done this. They are smart people. This is a reason why it happens. And we can observe these every single day, when we ask ourselves why they crave this interface in the car like this, or why they put this an application or why they eliminate this feature. Very common, what we can observe when people decided when management decided to remove some features, and then return this features. This is exactly how it happens. It’s not about numbers, small table, different opinions and just somebody have a bigger, bigger voice than yours. That is why it’s very important when you do a product management or do a user experience, design or redesign of the product. It’s important To have a support or at least understand preferences of the people who made the decision. And you need to understand this preference not to try to adjust to this preferences, but just to keep in reality, what will definitely not go all how you can provide your justifications, not just things by book, but taking the current reality of management.

Iryna M 25:33
So let me jump in over here and start asking slightly different questions in your world, and in your experience, you know, we kind of really started talking about that, but still, you know, looking back at your experience that you’ve got, in how many physicians and how many different companies, engagements clients, so whomever you worked with, you actually had the chance to meet a decision on the particular parameters in the role of Product Manager. And in how many of those positions you actually will resume to ensure that you’re orchestrating or making sure that the decision has been made. But then there is some big bodies sitting at the table who are making the final call. I guess at least one of their discussion points that we heard here earlier, is really product manager who should decide on versus and is it actually happening, especially enterprise world, when somebody in their own product manager can decide on the pirates? So looking back at your experience, How was it for

Nikolai 26:40
you? Uh, um, for my personal experience, I would say that I was quite lucky in terms of more number of positive decisions. So a number of good products good, I mean, useful products that bring good good value for end users. bring money for the Creator. So of his products were built. I will try to answer this way. If we’re talking about smaller organizations, small companies. It’s it depends on another small companies as a matter of fact, it’s very depends on engineering and culture in general inside the company. So some organizations, huge financial institutions, they have a rule that every single voice should be heard. It’s not always good, but at least you know this rules. And in this case, of course, they also have ultimate voice, in this case, ultimate voice make a decision based on some good estimations, the opposite to this. I have relatively small number of cases, when people even I mean product People even didn’t try to do some proper things by the book, because they were aware that management do what they think they should do. A management said that we have smartest people in the room, I would say, based on my experience, and it’s more than 15 years in industry, and a couple dozen giant clients and the same amount of smaller, I would say 20% it was about making decisions by management without any consideration of product people. Yeah, it’s quite sad but not so worth statistics. And for for the rest amount of cases. It was relatively fair decisions, but these decisions were by the Choose already good enough options. And they made the decisions based on some higher interest. But again, important point here that they made this decision by, and they choose the solution from already good enough least.

Iryna M 29:15
So I guess what you’re saying here that about in the, in about 80% of the cases, pro the people has brother people have the power to make the decisions and to kind of dictate the purchase or define the next Captain song.

Unknown Speaker 29:31
Yeah. And it’s important to understand that even if you think, Oh, I’m just a regular Product Manager, and I have a manager and that manager has another managers, we need to understand that this triangle or moving from data to wisdom, right, if you put your best effort to make sure that your results clear enough understanding on a higher level, and by the way, this is another very powerful Don’t think why many things are fail, that very bright people, they just can’t communicate on the level that higher the people stands on a higher level in the hierarchy of the organization. They simply couldn’t understood what people say. So if every single product related person will do his job on the best possible way, even if he can see this, this moment, here, he intentionally or unintentionally effects on the final, final decision for the product. Even even if you think all I told them, this feature should fly. And after a number of meetings, his manager tell him your feature will not fly but you’ve done a great job. What the hell does this mean? Potentially it means that somebody just didn’t like your solution, but I was I was able to observe a number of cases when solution was so good enough that management decided to put this as a additional killer feature in the next major release of the product to make sure this great feature will bring not only satisfaction to the clients but will be good thing for money wise

Iryna M 31:32
and probably will really last one here for me, do you think that that that would have freedom would depend on the domain on the particular area on the particular company as your business that we are talking about because, as you said, you know, you’ve been working with quite a number of different clients and companies, our products And I’m just wondering if you ever had a chance kind of to sit back and compare those and see how are the managers and they are free though and where they’re allowed and not allowed to do is somehow differ from industry to industry? Oh, yeah.

Nikolai 32:21
It’s funny that you mentioned this thing. Just recently, I tried to compare again cultural perspective and cultural traditions in different companies, some of them our clients, some of them not, but I try to put the different organizations in the clusters, where product development, culture is high enough, developed enough to, to deal with this, but in some organizations, this Culture is exists, but it has a different interface. And this is why I started from the data point, the data driven products very standard thing from one perspective, but very few people don’t really understand that it works in both directions. I mean like on the product side, a non engineer can you can easily focus on the product features, but continue your conversation on the data development language with the people who are involved. So, I will try to say that

I will try to provide a couple examples. In some organizations, you have a right to say, Boy, I want to continue to work with the features and you can face higher decision Just the decision of your colleagues. That way, we don’t want to go with this feature. And you can always have a choice. I don’t want to work on this. I want to switch team or I okay. I disagree with you. But I commit and this thing like disagree and commit is very crucial and important to make sure that our organization will continue to work efficiently. From my experience, the number of fail, where because people disagree, they didn’t commit and they feel they felt like a personal failure that they suggestion of the features where they stay on a shelf on the table and didn’t move to the product. This is not like In one case, this is not only the case when it happens. But this is very often case, when people when the results of this simple business decision, probably not very fair business decision was the trigger for problems with the product development.

Vlad G 35:25
Okay, thank you. That was that was interesting. So I want to move on a little bit, because we seem to have stuck stuck on just one example of healthcare. Let’s Let’s go with finance. Yes.

Unknown Speaker 35:38
I want to talk in this case, just briefly about the example and you tell me Is it good enough about the application that was very interesting for me. The application was like a core core component of very huge private investment company, which is also based in New York, and may Other major cities. And the intention was we just want to do a lift and shift. And this was a kind of a moment of an add No. charisma stupidity. When I was able to say to the Senior Director of this company, well, of course we can do this, we are professionals. But this is the wrong direction. You will spend another X amount of money, you’ll have lift and shift of your product, but you don’t solve any problems. And somehow I was able to convince the senior person from the client side that we need to spend some time on the discovery. And here’s the problem when you can lead somebody, especially on the client side, that you can provide something better than it was before, you really need to do this. Because one of the assumptions that wherever consulting firm wherever product managers will come to us. It will not help us significant because we know better how to do this. And partially it’s right. If we’re talking about financial institution, obviously, you will not be able to understand financial market better than they are until you’re on the same level of understanding, but you need to find the framework, how you convince people on that side, to provide you a chance to prove that your process can bring some value. So this is what was done by me and my team. We explained that instead of trying to do like lift and shift on a very huge application, I believe it was more like Couple hundreds of cream, we need to understand like essence of the process to understand your users and what they actually use. And in the end, it was very elegant tablet ready web application. In total, it was like six or seven screens, imagine a couple hundred screens. And after that, like just a couple less than 10 screens.

Nikolai 38:30
And the funny thing that people were really happy because now learning curve is so small. Yes, we eliminate some functionality that most people didn’t use, and it wasn’t essential, but it was a kind of a happy story from one and now I will tell you like the real deal will be extremely hard work and it’s not something quite high. We’ve done a great analysis. And we made the magic. The Magic was like hard work and the hard work that every single day I need, I had to fight for another day. Every single day, I had to fight to have an opportunity to continue work on the thing. So for the first two weeks, I literally had to fight to be able to continue work on this product for another day. After the first couple of weeks. I need to fight less often a couple times a week. But still, the pressure was so high that it’s still one of my most difficult projects and corresponding products for me, because the pressure was on the top level. People didn’t believe to me that it could possibly happen till the very end I will say even till the first couple months of production, use of the product. A significant amount of positive feedback. So they didn’t read me I’m sorry, throughout, I just want to understand this, what the struggle was the struggle was that they did not believe that you will be able to build better solution provided what they had. They just wanted to replace it with something more modern, not not just that they didn’t believe that it will be possible. For them, it was like a pure adventure, that they spend some amount of money because they point was to prove that I’m wrong, and they don’t need to apply the same principles. Father,

Vlad G 40:39
what was your what works or what were your winning arguments?

Unknown Speaker 40:45
My winning arguments was I don’t have a silver bullet. I don’t believe it. But we have some good recipes that we can have long hanging fruits and as a result, or we Hearts of the end users. And to show other opportunities, I didn’t promise them that will solve all kinds of problems that they have. I just promised to them that we’ll be able to help in a specific area. And I try to be very not pessimistic, but my prognosis was like, hey, and my play game was you need to spend X amount of money not to solve all your problems, but this will improve your situation here and there. But we will use not like lift and shift which will be just a little bit cheaper that I need to help you. But it will open for us opportunity to go deeper to more essential problems that you have. So my point that my fight was for small when the fight was for a bigger And the fight was against mine. And invented sounds like wow, you build a product that people really appreciate. And as a matter of fact, this company’s still our client. And as far as I know, they just recently asked it ditional help for other products. But it was a crucial moment how us can survive. It was like three months of non stop work day by day. So in the end, when we very often listen, oh, successful story. behind any successful story, there is a number of small alleged failures. And this is reality. My takeaway from here for our audience, that even if you need to fail a number of times during the day but it’s still aligned with your strategy, and you don’t have Red flag that you need to stop, you need to continue. And you need to continue to work harder on the things because very often when we see a night white papers, videos, oh, it was so cool project we came, we see the problem was solve it and we bring like 20% of savings. That means that most likely not often, but most likely people try to solve a bigger problem and potentially they spend much more money. This savings will help you in the next five to seven years. So it’s important to check the reality and understand what was really done and how difficult it was. So my point there is almost no quick and simple wins. In product management. I

Vlad G 43:56
would I would actually prefer that as someone who’ve been fighting these battles daily. Most of my time being the product manager, I can totally relate to that. One more thing that I wanted to kind of stress and that would be my next question for you is, you got to have a certain level of skill and certain level of chops, if you will, to be able to fight these battles to be able to fight these battles every day. And to me, I mean, you’ve been around a number of years, I’ve been around a number of years in a production world, we kind of take it for granted. But what I’m seeing in a lot of cases are people who think they need to read a couple of books or you know, get an MBA, that’s my personal pet peeve. And they’re golden. They they can go and do product management in the mosque. So what is your take on that? What do you think they lacking or what do you think is needed? And Irina, if you have anything to add, feel free to chime in? So yeah,

Unknown Speaker 44:58
let’s move This way to have an MBA is definitely a plus. And it’s, I want to be very, very specific here. Any good education is definitely plus intelligent person always can utilize this experience, this experience in education, or let’s say you’ve finished a number of certificate programs is good, you gain something new. But it’s important, how much of this knowledge you can apply in reality in the situation, because you can have all knowledge and you basically can stand freeze, and not doing anything at all. And this is how we differentiate one group group of let’s see product managers who can act right now and another group of products managers who don’t have or they also could have my point, have education is not a bad thing. It’s not a guarantee that you can solve real problems. So in terms of the like, what I recommend to do, again, it depends where you are in your career paths right now, if you already have some experience, let’s say in engineering, or marketing, and for some reason, you said, Oh, I want to be a product manager. I saw that cool video on Netflix. I want to be like that guy. It’s a good inspirational point. But definitely you need to go A to Z at least couple times. And you need to be ready that you fail a number of times before you will be able to do something that will bring you success satisfaction. Oh, I am a very successful product manager. I’ve done this and that. This is why many companies who hire interns of fresh gradiated even like position intern for a product manager position, they understand this, and in many cases, and they provide for you some work that will be useful for them fresh blood, new fresh ideas. But then at the same time, I don’t believe that any products manager who has enough weight in organization will put his take on a person who could be extremely bright, but didn’t done this before. At least some amount the number of times, of course, he will take in consideration his experience. But product management, it’s an area when you need to have some statistics, not just because people think that you’re not smart enough, just to make sure that your decisions are not just based on academics, and they based on a real production experience. Again, tricky point and mechanics. Very, very thin border here. It’s how you transform things. And I will try to provide a couple examples here. Or I need to build a need to provide a new product to have new users for this and that and you start building amazing products. You do a lot of research. Just like you started in a good university of yours, and everything by good and if you look from the first side, you see Huh, not bad. It’s actually great plan, but it will work. First if you have enough budget if you don’t have a problem. And that’s it. It’s like comparing pilots of airplanes. One, if you don’t have enough hours, you can be bright. You can have all skill, all essential foundational skills, all knowledge. But before you pass like golden amount of hours, people will not be able to believe that not just believe they will never give you a airplane was 500 passengers. And they will tell you now you’re responsible for this. And the same situation here. If we’re talking about small student project, why not? Yes, of course. If we’re talking about Product project and corresponding product with a huge budget and potential effect for entire organization. This is exactly the situation when you don’t have a second chance. Yeah,

Unknown Speaker 50:14
I just wanted to really quickly to chime in over here. So lots coming back to a question about the education rating and NBA for product managers. Being the hiring manager for both for product manager or small company, I can tell that we didn’t reach out to any single person because he or she didn’t have an appropriate dedication. And NB definitely applause but this is this is so doesn’t give your sign that this candidate will do better than anybody else. If part of that we were doing research, whether there were assist tender certification of tender education that you can take product managers similar for like BMI, for example. managers, for example, that would be recognized all over the world and that guarantee that you kind of token to the person who knows what the product management is about. And unfortunately, this research didn’t take us to the conclusion didn’t take us to the physician when we found one certification or one program that would give you this recognition as their world wild Product Manager. So with that, I would say that education and the right education is definitely a plus but still a practical experience and the records that you have experience of launching products would matter much more during the interview during the hiring process comparing to the education itself. So MBA, yes, if you can do that. But if you think that MBA will take you to the great product manager position than now on Fortunately, this is not true.

Vlad G 52:01
Yeah, I agree with that. Now that I’ve asked this question, I probably should provide some some sort of response to that as well. Since you both have chimed in. Personally, I can say I don’t believe in the education but i believe i believe more in a practical experience. So practical experience in my book wins, wins over a day and night over just having even the really good education. And I agree, any good education is good. I need education. That’s good, but nothing beats practical experience and in my book,

Nikolai 52:34
and I just won’t put that there are common again, as a practical takeaway from this conversation for the people who listen us or will read us that there are some good recipes that helps like popular communities that bring attention have successful product managers are people who somehow affect product growth, product creation and creation, growth, etc, etc, everything related to the product, it helps, because in this case, you don’t need to run your own products, like a number of products and you still can gain insights from the people or like from the practical experience, it’s one useful thing. Another useful thing, it’s like, not a single book, but to keep kind of your your own. Your own. I don’t know Bible or navigation guide, what what you’re looking for afterwards. So for example, when I work for specific industries, three or four specific products, I always have like, hey, these top 10 guys are these top two firms. For some reason I think that they are leaders like real rhythms in this area, it doesn’t mean that they do everything correct. But at least reality check with them is a good starting point for validation any of your ideas, if that makes sense.

Vlad G 54:15
Just for the record, we’re almost at the time. So I want to start wrapping up. And as one of my regular questions, I’m going to skip one because we’re all working from home. One of my regular questions is how does it feel to work from home? We’ve been doing this for quite a while. So I don’t think that that is big anymore. But the other one is, again, it’s a regular question I asked every time on my podcast. Do you have any questions for Irina or myself? kind of turn the tables around let you ask questions instead of answering them.

Nikolai 54:51
Oh, I will be happy if we can extend your podcast to some kind of a workshop and bring questions from, from the audience, or do follow ups that way we’ll do that. Eventually, when we have enough questions, we actually did one of the episodes we did only around the questions from the audience. So that the next one might be in order. So this one is just for you. If you have anything to ask what is your favorite? What is your favorite trailer from your experience? Like quick example, in Tableau in couple sentences.

Vlad G 55:31
It was over engineered solution for and yet another manager for our clients of our product. So we had a product was a community around the product. And we thought about building yet another medicine messenger as both of the channel for critical alerts and marketing messages fail gloriously because nobody needs another messenger.

Nikolai 55:57
And what was your takeaway from that? imaging.

Vlad G 56:02
My personal takeaway was, it’s good to experiment because we spent two weeks of part time work and the problem about $50 on this experiment, instead of spending several months and if you thousands of dollars, thousands of dollars on building the prototype that we wanted to build. Cool.

Unknown Speaker 56:26
And I have a quick question for Irina as well. What drives you to keep running? Because we work for the same company. I am familiar that Rena is doing amazing job not just as a practical, professional, but also as a very proactive person in a product management community and what drives you to work in this area.

Iryna M 56:57
Spreading the knowledge, maybe They’re making Product Manager, product managers and product management as a true real discipline and practice. You know, I, I started this as a developer myself, I moved to business analysis, and now I’m in product management. And people tend to think that when they’re in product manager position, they can just sit back, relax and do nothing. So they are seeking that as a goal to dream over their career paths. And to me, I would like people to know what are the trade offs of this profession to know what the real product management is about? And that’s one of the reasons why we’re doing this podcast and probably, even if it correlates with the name of this podcast, and I would like to have more experienced and more skilled product managers Iran myself in order to learn from them, and not just for the knowledge and indicate them so yeah, we all would like to grow. So we all would like to learn new things, educate ourselves and having the right people around is basically the easiest way to do that in my mind.

Nikolai 58:23
Amazing, thank you!

Vlad G 58:26
Right, thanks, everybody. It’s been really great conversation. And we’re right at the time first, best six or seven episodes. So I don’t think we should be watching that closely anymore. So Nikolai, thank you very much for being our guest on the podcast.

Nikolai 58:42
Thank you for the invitation. It was a real pleasure. And I’m looking to join your big events, with workshops and our audience full of questions.

Vlad G 58:55
Hopefully, we’ll get there. Irina, thank you so much for helping me with this episode. Really. appreciate your being here.

Iryna M 59:02
Thank you and talk to you next time guys.

Vlad G 59:05
All right, thank you guys and talk to you soon. You’ve been listening to the real world product management and I’ve been your host Vlad Grubman. Until next time

Real World Product Management – Episode 09

April 28, 2020 Vlad Grubman

In this episode I am talking to Ksenia, a data analytics and BI specialist. We had a chance to discuss a data-driven approach to building products, the impact of AI in the near future, and what product management would look like when AI comes a-calling.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Alright, Hello, everybody. This is another episode of the real world product management and I have senior Kalashnikov on the line today. I think Irina will join later if she can. She has other arrangements at the moment. So Excuse me, can you please introduce yourself and tell us a bit more about who you are? What are you working on? What is your role?

Kseniya K 0:38
Okay, sure. I’m I’m not sure whether I’m supposed to say hi, everybody, but Hi, everybody. Currently, I’m working in EPM as senior business intelligence analyst, that actually implies that I’m working with both business departments and the architects who are building the solution for the enterprise. So it’s More on the edge of business and technical perspectives. And that would be nice to share some experience in how that is actually working for businesses and technicians and people who are interested in both of those.

Vlad G 1:16
And that’s why we have you off today. Yes, yeah. So So I love the words business intelligence. I actually I like intelligence as a whole and business intelligence specifically. So talk to me about the data. Why is it important to have business intelligence? Why is it important to have data collected and processed? And we know it’s kind of expensive and gets expensive over time. So why should businesses invest in that?

Kseniya K 1:49
First of all, I feel the urge to mention that business intelligence is not a tool, and it’s not a thing. It’s more of a set of approaches technology. GE architectures, whatever it takes just to get the data. So when we say that we need business intelligence, that’s just like the very, very beginning of the very long road. That means that we are just getting started. And it’s I’m saying that not to just scare away, it’s just to say that there are a lot of ways how to do that. So basically, when companies are introducing business intelligence, they think that it is something extremely huge, very complicated, very expensive, and probably we don’t need that. Why overcomplicate stuff, what the heck. But the key thing is that business intelligence actually gives you more insights out of the data, which you either already have or just planning to ever gather. You can pick just a few very starting small approaches and small techniques, and they will already give you much more insights and ideas about what to do and what to start with. So I’d say that business intelligence is great to start with. Because it helps you both work with the data that you have. gather more data even out of the sorry places you’ve never thought you’re going to get data. And also, because it can help you build out the bright future for both your company and the environment that you’re working in, so kind of works for it. After all, even being these complicated as it sounds.

Vlad G 3:30
I like I like the bright future reference things.

Kseniya K 3:34
We like future references.

Vlad G 3:39
Oh, yes, definitely. Yes, indeed. So question, um, we kinda know and I’m with you on this one, as much as I like to disagree with my guests. One of the things that I noticed since we’re talking about a company’s building, bright future and all that will usually happens in at companies and I’ve been I don’t want to say I’ve been a victim of that, but I happen. Companies have want to use advanced analytics, bi, maybe even machine learning artificial intelligence on the data that they have or they are able to collect. they assess the costs or the assess the expenditures, the resources that they need to process the data. They think it’s expensive, they end it is just hiring data engineer, or data scientist and somebody who can collect process and understand the data. They look at all this and they say, you know what, that still expensive I we can’t afford this. Now we’ll probably do this, you know, q3 next year, which is basically never and they never do it and they obviously they’re leaving money on the table. So what is your given your background? What would be your kind of idea or recommendation approach to how should companies really approach this? Since since we both understand that we both want to build a bright future for all of us? How would you approach that problem?

Kseniya K 5:13
Well, referring to the thing that you said in the very, very beginning, I’m not sure whether it’s the right expression to us right here. But it’s like, a lot of the company’s approach business intelligence and advanced analytics, as you’ve mentioned, them is like, you know, they’re trying to cuca hair before they actually catch him. They just entered the market of business intelligence, they see how cool it is, and what a great experience other companies had with working with advanced analytics, and they’re like, Yay, let’s do that. But the key thing is that advanced analytics is gonna work out the way it’s supposed to only when you have like the data infrastructure already prepared, and you cannot jump into that just clicking your fingers, you should prepare First of all, and in a lot of cases This preparation takes way more effort, much more money than expected. And it’s way more complicated. But the key thing if you do that, right, advanced analytics is just going to smoothly slide in. So I’d say that to start implementing that in your company and just to go data driven, if you are planning to, at first you have to fix up the mess that you already have right now in your company and fix that at first before you start talking about business intelligence, advanced analytics, machine learning and all this kind of stuff.

Vlad G 6:38
I’ve I’ve had a few projects, kind of even before they became products, I’ve had a few of those kind of shut down because of because of this always expensive rationale. My personal take on this is yes, it’s expensive. Yes, it’s complicated. Yes, you need to clean up the mess but you have to start somewhere you have to stop At some point in in present and keep pushing towards the future and kind of way I, the way I see this was that, hey, we don’t have to buy the whole machine learning thing from day one. We don’t have to invest in the full blown artificial intelligence approach from day one. What we need to do is we need to understand what we have today and see how we can use it. We need to move on slowly to regular just look at the statistics, look at the regular analytics, move on to advanced analytics, move on to the AI and then eventually we’ll get to the machine learning because by that time, we’ll be prepared we’ll be ready to have the data will understand the data gaps. So, you know, data that we’re not collecting today, but we will be collecting based on what we see instead of just stats and analytics down the road. So that was kind of kind of the approach I was thinking. Fortunately didn’t take so but that that did workout in less less place to try it. I think whether sent that data driven approach is important. So is it just a buzzword a trend? Or is it the real deal? Talk to me about that?

Kseniya K 8:12
Well, I just wanted to make small comments regarding the project that you just mentioned when it did not kick up. As much as I hate to admit that and I hate to say that even out loud, I hope people aren’t gonna hate me for that. But um, a lot of the companies are working pretty much fine using just XML files and like simple reporting tools, not something very complicated, even without, you know, and they are able to perform the analysis they need and to gather all the information that can help them out. So the key thing is that Yeah, you’re absolutely right that we have to start with a small steps and small steps can be as simple as it is just make clear business rules in your Excel spreadsheets. Make obvious restrictions onto the reporting dashboards that you’re using. And that would be your first step to seeing the gaps then to introduce a new approaches then to apply more complicated more advanced analysis onto that and then moving further to where you would want to so totally agreed on your point to that. Simple steps are just okay here.

Vlad G 9:23
Yeah, just just one note, one note in what you’re saying I it’s not that I don’t like Excel love Excel is just that after in my career after seeing two gigabyte files that run whole hedge funds, I would say find something.

Kseniya K 9:40
Yeah, I hear you.

Vlad G 9:43
Don’t abuse the thing.

Kseniya K 9:46
And one of the most recent projects I have been involved in there was a very complicated constructional work done when the company has been trying different combinations of various equipment and There were more than 100 combinations within one stack. And it all was handled in the XML file. So when I saw that file, it was like really? Do you do you really use this? Is that okay for you? So I’m not judging them is just the, you know, the depth perception. So, maybe there’s something better for you to do that. Kind of maybe. Yep.

Vlad G 10:24
Yeah, yep. I mean, I’ve seen I’ve seen people using Microsoft Access. And that’s kind of the whole the whole story about the road it people who can get things done by their own IT departments so they start learning Microsoft will they already have Microsoft Office they already know Excel. They maybe learn Microsoft Access because it’s slightly better in its actual database, even though it’s you know, reduced significantly from sequel but still basic database and that’s how you get the whole departments riding on rope databases because Yeah, things things are not things are not there. So, back to the original question. So data driven approach is a thing. We want to use it as much as we can, regardless of whether it’s a simple Excel file. If it’s something more complicated or if it’s an advanced analytics suite, we’re still we still want to use it. For what, what do we what are we going to do with the data? How are we going to how we’re going to use that data? I mean, do you have any examples of using the collected data for real world usage? I don’t know. Building Products, changing products, anything of that sort?

Kseniya K 11:42
Yeah, sure. Well, first of all, I would say that your question is still connected to the story about the raw data and people trying to use axes themselves. That the key thing is that in the majority of cases, people are doing that out of despair because they as you said, they cannot reach out to IT specialists and They have to solve the problem themselves. So they just invent new ways. But the key thing is the business intelligence kind of gives you the information, it makes it more democratic sort of thing for you. So you don’t have to invent your new wheel, you just use the already simplified and comprehensive information for you. Once were going data driven, and we are willing to do that to the BI, I would say that, in a lot of ways, it would make a lot of sense to just have the hypothesis and you know, like the first statement we’re starting with, but in a lot of cases, people do not have that on the project. Just to give an example, prior to working in EPM, I’ve been involved in the banking products, and we’ve been implementing a new functionality, which actually was not articulated properly, it was just aimed to do the job better. So just do that or give us Your best shot sort of thing is like a classic example of not articulated. task which has to be solved. And there’s no particular request, we have to deal with that and come up with some adjustments based on the data data is like the first thing we can use to actually do that first step. So we were working with a user friendly, user user friendly application, as we thought, and we were testing it trying to make it better and more, more mobile, more easy to understand for whoever is going to use that. And also there was the increasing number of information people were uploading into their application, so we had to handle that as well. So I’d say that given that we were not given a specific goal or specific restrictions on how to use the data we were given. We just started collecting Seeing the data which was most to the eye. So the most logical thing would be just together everything, you know, like get all the data you can get. But we were more aimed into getting the information which is more obvious. So let’s say for us, it was the quantity of transactions people were doing in various periods of the month. So obviously, when you have to pay bills for your apartment, that’s like one date and the month, when you have salaries, that’s another important date in the month and so on and so forth. So we were gathering a now analytical information data from this specific dates. We’re trying to get, how many people are online, what steps they’re taking to get to this information, what kind of transactions they’re doing? Are they getting any bills? Do they print out some information? Do they probably decent many populations inside of their accounts, etc, etc. So As a first step, it would be good to actually have the goal and the purpose articulated from the business department. But as long as we did not have any, we just went with the most obvious thing that we thought was is just to get the information which was mostly used and on the surface. So I would say that if you are like, in the middle of a huge ocean and you don’t know where to swim in the land of data, this is the third thing that you can do. Just grab the information which is on the surface and makes more sense to you than anything else. And that can be a good start.

Vlad G 15:38
Okay, so please go on. So what, what happens then next, you take the data, you apply it, you make changes. How do you assess whether you’re successful or not? I’d say that we’re actually missing one more step. before we’re assessing whether we succeeded or failed. We So how we’re applying this information, so it’s not enough to just gather information, you know, like, okay, hundred of people were using the application at BBM yesterday, like perfect. So what like, how am I going to use this kind of information? What does it tell us? Exactly?

Kseniya K 16:15
Yeah, like, what is it about what can I do to the So basically, once we gathered this most obvious as we are going to call it for now information, we have to do something to it. And current situation in the data driven approaches and in the BI trends, particularly, a lot of analytical thinking, and understanding is still done by the humans. We’re going to get to that, but let’s assume that it’s still humans job. And using the information that we’ve already gathered, we have to at first, understand where it can be used. So let’s say I have a number of people who were using my application yesterday. How can I use it like to wear clothes Basically, I cannot use that to the currency rates because it makes no sense. But I can use that to see how stable my application was. So what was what was going to happen that if the 200 and first person is going to enter the application, and everything is going to crash, so this is my point where I can use this number. So once we gathered information we have to think through and in a lot of ways from the business perspective, the part where this information can be used, and in the future applied to make some conclusions and build up some models even on the hypothesis level. So let’s call it this way. So as you said that we need to implement that like optimize it somehow to later figure out whether it’s a success or failure. So let’s say that we figured out that okay, that 200 and first person to answer the application is going to crash it Perfect. This is a valid information, we can use it. So let’s use it, how we can use that. So according to this, like the most obvious thing we’re going to do is just work with our technical support part and make the application more stable. So we can handle this, this number of people using it, maybe we can increase it even more if we know that the functionality is going to be expanding or becoming more popular and the sort of thing. Also, we can come up with alternative ways how people can enter the application. So let’s say a lot of people are using the four digit numbers, combinations to enter the application like kind of code lock. So what if we’re going to introduce the fingerprint scan, that’s going to increase the speed of login in and that would just reload the system a little bit because this an alternative way how to do that. So once we’ve found out the ways we We can use and apply information we gathered, we can actually come up with the solutions, how this information can help us. And what can be increased, because we’re basically working with how to make things better.After we’ve implemented this kind of things, simple, simple, old fashioned testing, so you always have to test out on people. It can be you yourself, like on the first party, the development team, they might be a little bit subjective. But anyway, that’s good to test with them in the first place. Then the beta testing, the alpha testing, all this kind of stuff that we know. So this is the optimization and implementation part that you’ve been mentioning. And only after that, we can actually figure out that Okay, so the, the application did not crash. And people are satisfied with the alternative way of logging into application because well, aside from state Typical ways, there’s always the satisfaction factor, which we cannot ignore a no matter what our business is connected to. So if we say that Okay, so this kind of information helped us to improve our application, people are happy statistics are happy system is okay, then we can consider that to be a success. He, for example, people rejected the fingerprint scan, they say like, it’s too complicated, like my fingers are frozen, it’s not working, it’s just given to me bugs led me into the app. So it can be considered a failure, because it did not give us the optimization that we’ve been intending. And it actually gave us the decline in the satisfaction rates, so people got frustrated with us, so it can be considered a failure. So I’d say that before we actually label it onto white or black, it would be good to also have those two additional steps prior to any like finalization of the data usage at this point. Okay, that

Vlad G 21:01
that is that is really interesting. Thank you so much. Yeah, well, I get that. You need to you need to process the data. And obviously, you can’t just throw data at the wall and see what sticks. I think what you’ve described is improving usability and performance optimization using using the data. I was wondering if there’s anything else any other issues that you can stumble, when you are looking at the data, like, for example, in your in, in the story that you told about optimizing the performance and potential use improving user experience, instead of needing to type in the password on a small phone or tablet keyboard, maybe it’s just easier to use a fingerprint, how accurate that data was, or rather, let me rephrase that. How sure are you that The data is actually telling you the truth. And what do you usually do? How do you make sure it tells you the truth? And what do you do if it doesn’t? And I think you mentioned, you mentioned something, your story about data being collected by human by humans. One of the things that I know from from my past experience is, even if the data is not collected by humans, it’s still you know, in in real world, it’s still being generated one way or another by humans. So you can’t always rely on that data being true. Because there’s always you know, room for room for error on the data side. So talk me through how you guys deal with this.

Kseniya K 22:45
Well, I would agree on to that part, because human factor is always there so people can be tired while aggregating information they can be distracted so they can make him stay or use the outdated position just because they forgot to To update them or something. So, it’s always understandable why data can be false because a was processed by the human after it was aggregated by the machine. So it’s more than obvious. I do agree with that. I would probably say the very obvious thing and not, you know, like, not as sophisticated to you, as you were probably expected me to say. But um, I would say that the first thing you need to do to figure out whether information is lying to you is just to use common sense, you know, because when you see the information, which obviously, like confuses you or makes you question stuff additionally and more than it’s supposed to, there might be something wrong with that. Good example of that also from my banking experience, so we were preparing the it’s like it a little bit different, but still from there. So we were preparing reports to present to high authority Once a month, and the person who was in charge of generating those reports was using various coefficients of currency rates in Belarus, where I am from and where the bank was working the currency rates has two approaches of how how to calculate them, like statistically, a algebraic clique, let’s call it this way. So various ways how to calculate them. They are approximately the same, but still different, basically, because the approach is different. So the key thing is that during preparing this report, the responsible person used the coefficient which was calculated using the different approach than it was used in all the previous month. Obviously, that created a difference like the huge delta between the past months values in the current month values so the regulator actually called and said, Hey guys, what Are you doing there? Why the Where’s the money? Why everything changes so much? So basically the same that could have been done in here is one more additional checkpoint link, which would figure out you know, like this edge cases, the Delta like okay, that differs way too much than he was in one month before two months before it cetera, or where did you take the coefficient from please show me that and Oh, the value does not match. So just setting some common business rules and restrictions for validation would help you to cut to catch up the mistakes that were done by humans, would you do not come into the common sense? So like the most obvious things, you can do that manually with your own eyes, the ones which are not as obvious but still questionable can be caught in business. drills and how to Elsa secret whether data is lying to you. Um, I’d say that he would always say you probably will always have the way to compare that. So if we work in with statistics with analysis and different levels, like if it’s just the reporting or the dashboarding, or the cloud analysis already, there’s always the way to figure out the trend. And you would see that some things are changing, not the way they’re supposed to not according to the modelling that you’ve been building previously. So you can just find them yourself through common sense. So I say that to find out the obviously false data use common sense, aka human brain. business rules, aka validation and restrictions, and three trends and proceeding business models. So like this three steps can actually give you the insights whether the data is wrong or not really.

Vlad G 27:05
Wow, that that is that is interesting. I had and I’ve talked about this product in one of my first episodes, I’ve had a product that I was building that relied on data collected from points of sale. So they itself was not generated manually, but the process was, and the fun part. Again, maybe I’m I misspell or miss pronounce the steps that you’ve mentioned. The funny part was that business rules would be okay. And the salesperson in the store that was using the boiler sale was actually following the business rules. But they would find ways to game the system and we’ve expected about You know certain certain percentage of transactions running through the system and in depending on the store, it could be from one or 2% to as much as 15 20% of transactions to be fraudulent, meaning they would adhere to business processes, they would adhere to business rules, otherwise the system will not process the transaction at all. But we will do something else about that transaction that is fraudulent that is wrong. And if you just look at the transaction it like you said, the human use common sense and human eye, it would never you would never see that it’s a it’s a wrong or fraudulent transaction. And you we actually developed a number of separate rules, if you will, or additional algorithms that would look through the data, match it with the business logic and then match it against a number of additional rules. To see if they just had transaction was fraudulent to give you a a removed example not not from that industry but something that everybody understands. So this is one of the examples that was given to me and what is what is a fraud at the point of sale system. For example, you want to sell someone a bottle of champagne in a bar, right and this is ideal bar right? You just you just work there so you don’t really care. And so use your friend you can charge them in the club and the bar is really expensive bottle that usually costs $20 make us 300 so he’s your friend, you don’t really want to charge him $300 but you want to give them that bottle of champagne at a lower cost. So what you would do is you would take a chocolate chocolate or you know, something less expensive. you swipe it on the barcode scanner and you put a bottle and thought Have it so you don’t use the camera that tracks you. And there are cameras that track the cash registers, it will see that you’re scanning a bottle of champagne. But the barcode that is being scanned would actually be from a chocolate so you pay, you know, a couple of dollars for a bottle champagne that you’re supposed to be paying $300 this way, transaction is correct. You know, business business rules are are observed. You just scan to a piece of chocolate and you know, you’re paying whatever 510 dollars or should I get worried, you know, this methodology way too good. Yeah, because, because I as I said I was building product, it was actually analyzing the data and highlighting the fraudulent cases. So it was one of the uses for the product was to catch the fraud. We’ve one of the pitches we did was that if you I’m just saying To remember the exact phrase. So we’ll we’ll show you how much money we’re leaving on the table by analyzing just the fraud. And

it was it was something to the extent that please, we’re not going to charge you any fixed fee where they charge you a percentage of the recovered money based on how much further we recovered based on how much in efficiencies were recovered. And people usually in the couple of months after, you know, two or three months then once they’ve seen that the amounts that we’ve recovered and potential they ask we’re asking to switch to a flat fee no matter how big it is, because you know, any reasonable flat fee would be still will still be less than, you know, the fraud and the amount of money that that we were recovering. It was really really huge. The the ways people were gaming the system, the ways and this was the telecom industry. So think selling iPhones at $1,000. Price mark. So if you can, you know, game the system and sell two or three iPhones at thousand dollars a year commitment and gain commissions from it, it’s a pretty significant amount of money. It’s pretty big deal.

Kseniya K 32:12
Yeah. So I would say that this is the beauty of this kind of approach that no matter how intelligent your system is, and how perfectly verify your business rules are, it still can give you the wrong data. And no matter how qualified and experienced your people are, they still can make a mistake. So always combine those two. And they can give you at least a little bit better result that each by themselves. So that pretty much is what would work here that business rules in the camera and people’s attention probably could have given you the way how to catch the person who’s actually scanning the chocolate bar, but it did not. So there’s a lot of gaps in their overlay.

Vlad G 32:57
Yep, yeah. I mean it It was funny that even you know, the store owner would come in and say, or a bar owner or business owner would say, I know they’re stealing from me, I just can’t understand how they’re doing it. And these are the people that have been that have been in the industry for many, many years. So if anybody knows how to steal from their own cash register would be them. But yeah, that was, it was interesting how data can solve problems cheaper, as a matter of fact, then installing very expensive, very expensive security systems, closed circuit cameras and all that, like I really don’t need the camera, I really don’t need anything. I just need to see the data and I can tell you the outliers and I can tell you that this transaction has something something’s off or you know, the number of transactions does not match the number of items in your inventory, which usually different things in many systems for whatever reason. All right, this is this is interesting this is really great, thank you. So what, how else? In the real world again? How else can we use the data? Assuming we can figure out we can clean it up and figure out that the data is correct. What else can we do? Can we use the data for what else can we do with the data? Can we use it to plan the products? Can we, I don’t know, create new products based solely on data? I mean, I would love to have certain things right? Look at the data and say, Hey, you know what this feels like we need to build a product and that something similar did happen that exactly that. And that’s what I’m saying. I would love. I would love for this specific thing to happen.

Kseniya K 34:45
If I may, that reminds me on an off topic of how I met, your mother said come it was like we were told we should totally open a bar. Yeah, we should open a bar. So it’s here we have the data. We should totally start up a new product. We should do that.

Vlad G 35:01
That’s good. That’s good. That’s a very good reference. Thank you.

Kseniya K 35:07
So the key thing in here, probably the not obvious thing, like when we have the data, and we have the urge to make the earth a better place, you know, like to create a better economy and this kind of stuff. That only the obvious way we can go. But I’d like to give an insight on the nuts so much of an obvious way how we can actually use the data when we have it. It’s incorrect, and we want to use it somehow. I would say that one of the ways we can use data is to start developing the data culture among employees. What do I mean by that? It’s not like data driven culture, as you’ve probably heard the term but like the data itself, culture is that the employees who would learn how to kind of be hasty with the data and treat either Right wait. So in I’m pretty sure in a lot of your projects you as well. So how people are just dumping the information onto some weird spreadsheets, do not save them properly, do not update them once it should be updated, turned into garbage data pretty much. So the way that we actually were able to figure out the data out of this dump and see the value in it can be one of the stimulating ways for the employees to actually start treating data right. So let’s say you come to your employees and say that Okay, guys, so we had to analyze a lot of information through like these means and these means and whatsoever. And we were able to figure out that let’s say the, the number of purchases and the quantity of the box in the storage is Ashley. God basically from this Publication not from the others. And in the future reporting, we’re going to use information used only from this application as our main source. So that would actually give people the understanding. Okay, so if the main source of information is now this application, there’s no need to support, let’s say, this other folder of data, because it’s going to be obsolete, not used, not even for historical purposes, is just going to be useless and overwhelming. So they would create a habit of following the data and the trend in the data in the organization and the company, which is actually give them the profit, they would give them. The initial, you know, like, discipline sort of thing. They would be able to figure out like, what data to store and which one is not. And another way is to like, when they’re going to be understanding like what data is used, that would make them more aware of probably the mistakes and not how to say that probably not accurate usage and storage of the data. So they would say like in the particular example that I’ve been given earlier, so the person was preparing the reports once a month. So it’s not as often and as urgent as you would think. But in this particular way, it is extremely important. So you have to be like super concentrated, super focused and very, very much into the stuff that you’re doing. So it will be probably easier and more understandable for the employees how to separate and delegate different tasks of a very complicated reporting or a dashboard or whatever to different employees. So the so that the cross check would give them the most correct information, and each of the employee would developed their own responsibility for the piece of information they’re preparing. So as one of the non obvious ways of using the information gathered is sort of giving your employees the inner data discipline and data culture in treating numbers and various information correctly. And the way it can bring value to the company through themselves already, not after some smart machine or some smart algorithm is going to transport a some way and give us information. So like you are responsible for the data you’re given. So be nice to it.

Vlad G 39:40
Funny, it’s nice to look at it like that. I like the term that you’ve used the data culture of more than once I’ve seen people not really eager to collect to help companies collect the data because there’s this stigma Hey, this data will be used against me and they’re not wrong. That’s let me just say that there’s definitely a way how the company can use this data to. I don’t want to say abused, but take unfair advantage of the workers. And one of the examples that come to mind and if you if you feel you can comment on that was the recent discovery, I think would be the proper term that the algorithm was optimizing the roots of Amazon delivery and Amazon warehouse workers. But what was missing from that algorithm was the account that these are not robots These are real people. They need to you know, sometimes have a sanitation break. Oh, that’s just it was and it was there was a lot of there was a lot of complaints. There was a lot of materials published online, all over the place. In regards to Amazon, not taking into account the actually not Amazon, but the algorithms that they were employing, not taking into advantage. The fact that these are real human beings So for example, if you’re supposed to work an eight hour shift, it would plan literally all eight hours down to minutes and seconds. Make sure you’re the most efficient there’s was no account for you know, bathroom breaks, no accounting for just stopping and catching a break, stopping and taking a breath. Same thing if you’re a delivery. You’re driving the mouse on track to deliver goes to individual customers or from warehouse to whoever ordered it. You they will optimize the route without accounting for traffic and it’s fine in rural areas. But as you go into larger cities like Los Angeles, San Francisco, New York, Boston, Chicago, the large lommers it becomes a really big problem. And Amazon did not account for that, or whoever built the algorithm did not account for that. And as much as we love, you know, being efficient as much as we love making sure that everything is working as smoothly as possible. There was no way for people to meet the quotas or there was no way for people in those large cities to meet the expectations set up by the algorithm. Yeah, right. So in your, in your term data culture, I love it. I am making a note I’m probably going to use it. It’s some some points in my work. Thank you for that. We’re gonna make sure that we trademark please. Absolutely. So what I wanted to say was the the culture is is it the way street Yes, we want to encourage workers employees to save Data collected data preserved the data in the right way. But we also want to make sure that it’s being used responsible. That’s where I was getting it. By the way, since you’re an expert, I have I have a rookie question. Question. Is there reasonable? Can I reasonably expect the algorithm to learn? And I’m deliberately using artificial intelligence, machine learning terminology, because I don’t want I don’t want to say, you know, what is their statistical approaches or analytical approach? I want to use as generic as you know, as Internet’s terms as possible. Is there a way for the algorithms or whoever’s working on them to learn from real humans so that they, their their inefficiencies are built in you can’t optimize to remove those inefficiencies. They’ll be there just because we’re, you know, living breathing organisms and not not machines? Is there a way to account for that? On the algorithm or on the data level? Not you know, when I’m planning things, I kind of feel that you’re expecting the yes or no question from me, then, no, actually, actually, no, please don’t please elaborate as much as possible given, you know, given what we know now.

Kseniya K 44:24
So, if I may, I’m like a fan of weird references. I’m going to give another one recently been reading The Hitchhiker’s Guide to the Galaxy by Douglas Adams, and you probably familiar with the concept of the deep sauce computer, like the smartest computer in the world ever created and all the universes so humans gave the computer the task to figure out the answer, the ultimate answer to the lies of the universe and everything, and the computer was versus in disinformation. Through all the statistics through all the assumptions, risks, complications has been processing for 7 million years. And then the answer was 42. So I would like answering your particular question with this reference, I would say that the machine is able to like if we’re using the buzzword is able to learn as much new answers as possible. And use as many details and assumptions that take risks into considerations to the most possible way to participate in preparing predictions or forecasts or some analysis, that I still believe that the decision maker is anyway the human. Aside from all the science fiction and all the fantasy world stories where the intelligence is too intelligent to overcome the human And I strongly believe that the final word is upon the human and as advanced as the technology in machine learning and artificial intelligence and all the processing of the advanced analysis, and deeper bi in specifics is getting with time, that is still going to be more like this invest in that. So what next? So the person is going to be the decision maker, I’d say that the machine is able to learn a lot and to act and answer a lot of very complicated questions. But it would not be completely correct to leave the machine decide. So the decision making is still left to the human brain and to the subject matter experts who are humans and people and who are actually like people with the business. insights, deeper understanding, empathy, if you may. So all of the emotions and emotional intelligence, which are machines that are lacking and hopefully will keep lacking.

Vlad G 47:13
Thank you for the reference, by the way, I absolutely love the books. And both, I think both movies Well, one of one is the BBC production one is the actual movie. So funny that you said that will leave decision making to humans because as of last year, if I’m not mistaken as of last year. And I’m just going to generalize it because I don’t remember the specifics. So don’t quote me specifically on that. There was an experiment where scientists built AI, predicting or analyzing the symptoms and predicting the diagnosis in patients. With a very specific niche of the medicine, I can’t remember for the life of me, and I don’t have time to look it up right now, the specific niche, but it was one of the general things like seasonal flu or, or allergies or something like that. So it’s a very common case. So there’s a lot of data. And they did the analysis they, they ran, they created the algorithm, they’ve built the AI around it. And To their surprise, I predicted the correct diagnosis. Better than the human doctors, it was not marginally better. It was actually significantly better. So this was not like a marginal error. It was, hey, if this was the real deal, we should use computers rather doctors because computers are distinctly better.

Kseniya K 48:56
Yeah, that kind of reminds me the way that kind of Same project project I heard about when they were given the LRS algorithms, a lot of X ray files to analyze and to figure out whether it’s a tumor, or some kind of oncology coming up in human. And that gave actually the exact same result. So the doctors were missing out information and making less correct diagnosis then rather than the machine dead. So that sounds kind of the same to the situation that you gave.

Vlad G 49:30
Yeah, that sounds similar indeed. So that’s my point. And and we’re, let’s be reasonable. We are very early in stages of building algorithms and building machine learning and artificial intelligence capabilities. Because we’re not as as humanity. We’re not cultured, we’re not collecting enough data, we only now started to realize all the benefits of collecting the data. So we’re probably going to build additional you know, we’re going to need additional several years to build up the enough data for some serious algorithm usage. And that sense, we probably going to see even more even larger gap between what the doctors can do. And since we’re using this, this case, be what the doctors can do and what the computers can do. And in that sense, it feels like at some point, we’re just going to give up it’s really easy way I see. It’s really easy to give up to say, Hey, we don’t need the doctor to make the diagnosis. We just need the doctor to validate the diagnosis. So it makes to basically confirm that the computer was correct. And that’s it. We’re basically as far as I like to call it another called shoulder reference I I for one, praise the coming of our future robot overlords.

Kseniya K 50:58
So

Vlad G 51:01
Possibly thinking I’m getting there I, for once I like to think that computers will help us will definitely lift some of the load off our shoulders in terms of processing, analyzing data and giving us some kind of a, some kind of an outcomes, some kind of food for thought. At the same time, I feel feels like it’s too easy to say, hey, computer knows better, let’s let it decide.

Unknown Speaker 51:30
Well, you know, that kind of already happened to us remember when the first computers were so advanced in all the calculations and how fast we were doing some mathematical actions. So at some point, humans just had to admit that okay, we cannot do math as quickly and as awesomely as computers do. Well, it’s just that okay. And do you see people actually started stopped making two plus two now people are still Using that they just admit that okay, that computers can be better in this case. So I would, I think that it can be the same way with a further development of intelligence, self computers. So isn’t the example as you said, okay, the computers are given the better and the more accurate diagnosis to the patient’s into complicated cases, okay? So it’s just the way we can learn from them. So we can just analyze the information that they given us and try to find out like, what did we miss out? How can we can become a more advanced specialist, an expert ourselves and just keep treating people and just becoming a better version of ourselves. It’s more of the human way how to deal with that. So the computers just can get more advanced in the actions that are performing. And the humans can just become better in all the other aspects which are close to four machines and four algorithms. So Do not give up on humankind so easily, we can still use computers and even if they’re smarter than us, we can still use them and grow more from them and get more information from them and become a better versions of ourselves that positive vibes. Okay, so your

Vlad G 53:17
your outlook is is strongly positive. optimistic we were going to get better at this,

Kseniya K 53:24
you know, as much as we love Terminator, honestly, like every part of it as much as I love it. I still think that humankind is having something that machines are never going to have. And it might be a little bit, you know, like philosophical question, and which is not a part of this particular discussion. But in the particular case that you given. I think that even though machines are getting smarter than humans in some of the ways, attention in some of the ways not in all the ways it’s not the end of humankind era, and it’s not the limit for humankind capabilities, okay.

Vlad G 54:05
That’s, that’s really optimistic. Thank you. I appreciate that. All right. So we’re getting closer to the time. And I have to ask those two questions that I promised you before we started the recording that I’m going to ask. So one is, how does it feel? Or what do you think? or What is your opinion on working from home? More or less permanently? And as a data expert, I want to throw in another curveball, asking, how do you think, since it provides more data to be collected, like how long you’ve been online? How long have you been coding how long you’ve been, you know, sitting behind the computer? Would that affect our understanding of workplace as a whole would that be would there be any kind of consequences from that on on the data so the data collection Analysis side.

Kseniya K 55:01
Okay, so answering the first part of the question about remote remote working for most of the time, I’d say that, thanks to again, the growing intelligence of computers and the way they’re proceeding the information that we’re given, like this specific case, like we’re exchanging our voices for the ocean, over across the ocean, and we’re still doing that and all the data is going clear. And okay, so I’d say that remote work in 21st century is becoming less of a problem in its core, then the human work itself. What do I mean by that is that we do have all the means to do that like intranet, notebooks, remote workplaces, etc, etc, etc. But it’s more up to humans to adjust themselves like their natures upon this particular environment. So for them, to be more data driven to be more you know, like Simple example more attached to the schedule. So let’s say this particular hours, you have to be online, with your team having meetings, no matter what happens. And if your cat wants to have some extra food, you just have to be online and get more into the technology. You’re doing more data that you’re getting like those texting codes, and different comments in the GitHub, like different pushes and rolls and stuff and stuff and stuff. It’s more of the human kind way to adjust to this way. From my perspective as one standalone person, I’d say that it’s a in the technical and in data driven perspective, it’s not complicated for me, because I’m used to working with remote teams, and just being very, you know, like responsible onto the way I’m doing, but from the human part of myself and from the emotional intelligence And feels kind of lonely because you’re facing the machines only not the real humans you do not hear them laugh You do not see the faces so it’s a little bit complicated into the emotional part.

Vlad G 57:12
And as the data specialist I can you please remind me what was the core of the question like what data how the data we eat okay. So it was a data we collected from the remote from people working remotely How would that affect us? In the long run it would we collect something useful we collect something dangerous to us, those types of things.

Kseniya K 57:37
Okay, again, tricky thing I’d say that if you’re using like this monitor things like lot of the programming stocks are using like they’re keeping the movement on your on your computer that you’re actually like moving and typing something in, that would make the relationship between employer and the employees more mechanic and more techniques. So it’s my ruin. In the relationship and like the core of the work, not to ruin but just like, bring some damage onto it in the long run, because both employers and I’m going to be trusting enough, like, what are you doing there, like why your mouse Mouse is not moving, I see that you’re not coding there. And people are not going to be trusting of that, okay, I wanted to go drink some water, but I have to be here, right into code, otherwise, otherwise, the data would be collected wrongly, and I wouldn’t be paid properly. So it might bring some distancing, uh, you know, like mistrust in between people in some ways. But among the positive consequences, I would say that it would just show in funny thing, unexpectedly more managerial questions and issues like you can get the data in numbers like analysis and statistics about how your team actually was performing, then you would see that okay, I’m actually having the specialist who is not Okay with the tasks I’m given, or I should delegate one task to two people instead of one, as I do have it now. So probably the data that we’re going to have from working remotely, might give us insights onto how to optimize our managerial part of the work. So that’s from my perspective. And I think Interesting, interesting, now that you mentioned that the manager role part and the mistrust.

Vlad G 59:29
As, as the person who used to be a developer in the previous lives, I actually think it was create way more mistrust than I mean, my my overall outlook is that is going to do more harm than good because and that’s the one of the things that I read online, is, there’s a couple of systems out there, that track freelancers work, and what happens is they take several pictures, randomly You know, in a specific, a specific unit of time, let’s say five minutes, 10 minutes, it takes several pictures and counts the number of keystrokes. Then take screenshots. So not only it takes the picture through the camera to see if you’re in front of a screen, also take screenshots of what you’re working on, looks at the applications that are open currently on the computer, and so on and so on. Wow. And violation alert. Well, apparently, you know, you have to consent to that before it starts. So you can if you can just start doing randomly, if you take on the project, then you consent to this information being being taken from you. And I think up work is one of them. There are a couple of others. And I know that because we’ve I’ve used the artwork in the previous we hired people off the artwork in the previous job that I had. Yeah, well, I work as well. Mm hmm. Yep. So let’s One of the things that can happen here is that, you know, it’s it’s almost like spying on the person. And even then you’re you have the best intentions, even if you’re like, Okay, I really like this project, I want to work on it. And I think, you know, judging by the result is fair, but if if this is what’s happening, and the system automatically says, Oh, you did, I didn’t get enough screenshots of you working in this 10 minute period. So you’re not going to be paid for that for that 10 minute period if you’re being paid by the hour. So what happens with people actually end up working way over to make up for those missed 10 minute periods when they were not accounted for. So instead of an eight hour work day, people ended up working than 12 hours to make up for those little bits and pieces that were missing. And, again, as a former developer is a person they used to do that. I think it’s a pretty dangerous path to go on. They’d rather keep track of a result rather than the process itself. And then I mean, ultimately, I, I keep saying this in multiple situations, people don’t buy a drill the buy holes in the wall. So if you deliver the result, I don’t really care if you worked in 10 minute increments, or if you just, you know, Sprint through the whole thing in four hours and then spent another four hours trying to relax and bring your mind back. Yeah. There’s, there’s, there’s that idea that, you know, it may create additional mistrust and hostility. And a last question that I have for today for this episode is if you have any questions for me, and again, let’s keep it up to me being able to answer the question

Kseniya K 1:02:46
Don’t worry, I’m not gonna challenge you with anything like to artificial intelligence Lee because I don’t know that either. And so the key thing I actually like generated this question throughout our discussion. So if you are like you’re As I see that most of your positions are more of the product manager side. So you are more from the business perspective from the overall situation tendencies and like this part of the project. Thank you. Yeah, bigger picture of the project. Exactly. So how do you think he is there might be maybe, to some extent, in the future, these threats to you to what you do to what you’re analyzing from the intelligence and from the business intelligence from the data driven approaches, and from the algorithms that are as we figure it out, advancing way too quickly. Well,

Vlad G 1:03:44
that’s a great question. Thank you. I I was the right word to them. We just said I didn’t think of this. But now that you’ve asked this, I think it’s a beautiful question. Thank you. No, I don’t think there’s a thing I think we are actually under less threat of from AI than musicians are. And I’m sure you’ve seen cases when computers write music that is more pleasant to the, to the year than humans. Yeah. And I, I’ve got I kind of I was subscribed at some point to a source of computer generated music, it just was not generated. That particular person was not generating it in the style that I liked at that time. So I lost that the thread I should probably go find it. But since computers write, you know, compose music, they write lyrics, they write, create paintings, it’s natural that they will one day start doing product management work. And my rationale is product management is as much art as it is science and computers can probably do both at some point, but in the same way They probably going to be more supporting rather than replacing role. I think we get to have a lot more insight into why people do things, and how people do things, which is how we create, and how we enhance our products, and the whole data driven approach to building your product. And that you remember you remember, I alluded to that as is there? Is there a way to look at the day they say, hey, this looks like a new product to me. Definitely, I definitely want that to happen. Because this is one of those cases when we can we can create products that create products, kind of like we’ve we had plants that manufacturer tools, and the way that those tools were used to build other tools, and then we were building add products. So it’s the same way though you can you had robotic mechanize them their robotic tools so that you just tell the tool what you want or like a 3d printer at the end of the day. So think about it this way instead of giving the 3d printer a specific dimensions or specific things, specific instructions how to print a certain item, you just tell them hey, I need something to open this cat. And then artificial intelligence would analyze that hey, this is a you know, aluminum cans, so you probably need a knife. So you probably need a knife with certain level of sharpness and you know, being sturdy and from specific type of metal so you can cut the aluminum and it would make all these decisions by itself and then print the 3d print that knife. So that would be Oh, by the way, there’s a great literature reference if you know it. I think it’s rubber Shaklee The product was called confabulate er, so you would tell him what you want. They were 3d printed or or created from something. I heard of that. Yeah, it’s look it up by I think Robert Shockley had a series of stories. This is one of the short stories. And one of the problems that because yobbo confabulate er had was he had he had a consciousness so it never made anything twice. Mm hmm. It never made any any of the thing fly. So if you ever asked it to build something from steel, it would never make anything made from steel like ever again. And that was kind of just kind of like a premise of the story. That’s why how people got into the trouble. So if you want rid of that, and you know that that’s kind of a that’s kind of a way I see this happening, so we’re definitely gonna get there eventually. I’ll probably retire by them. But I still want to see this because it makes your life a lot easier. You can stop thinking about little details and you actually can start thinking about bigger pictures and you pictures just gonna get bigger because you Now you can, instead of thinking, Oh, hey, we’re going to put you know, Part A, insert Part A into detail B or into item B, you can start thinking, Okay, so I want, I want a bicycle, right? You know, it has to be this tall or this thing. So you can, instead of writing requirements for every individual piece, every individual tiny little thing that comprises your product, you can start recording or writing requirements for the whole product. And then whatever that is to fill, show intelligence, we’ll figure out how to build that product. So that would be really cool. Yeah,

Kseniya K 1:08:35
well, honestly, I wasn’t expecting such an awesome answer. So thank you very much. That’s actually very cool to hear, like how you see the, you know, like artificial intelligence and like the data driven future from from your early business perspective is really very awesome. Thank you.

Vlad G 1:08:54
That by all means, yeah, I read a lot of science fiction. I still do. A lot of crazy

Kseniya K 1:09:00
That’s what I was implying but never mind.

Vlad G 1:09:06
All right, this has been great. Thank you so much, Sonia. I appreciate your being on this episode of the real world product management. Thanks for having me. Absolutely. It’s been a pleasure. And I’m hoping we’ll have you again, I think you have a lot more to contribute to our knowledge and understanding of advanced AI and machine learning and artificial intelligence. So I’m looking forward to talking to you again,

Kseniya K 1:09:31
Thank you for having me, thank you for today.

Vlad G 1:09:37
You’ve been listening to the real world project management and I’ll be your host, Vlad Grubman. Until next time

Real World Product Management – Episode 08

April 21, 2020 Vlad Grubman

In Episode 08 I am talking to Jon Janego, who shares his experience of being a product manager in a software security industry and the challenges of a young product management organization.

Transcript (courtesy of Otter.AI)

Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!

Vlad G 0:07
This is real world product management.

Hello, everyone. This is another episode of their real world product management. And I have a John Janego. Am I pronouncing your name right?

John Janego 0:23
Yep.

Vlad G 0:24
Okay, so John, is here with us today. Thank you so much. Can you please introduce yourself?

John Janego 0:31
Sure. So my name is John Janego. I’m a product manager for a company called Vera code based in Chicago, Illinois.

Vlad G 0:41
Thank you. We don’t have a very specific agenda. So it won’t be like some of our previous episodes when we will where we only talked about data driven decisions or very specific presentation, then do the q&a. We just want to explore with john his career path. I think there’s some interesting things that I keep seeing that, personally I’ve never seen before. And he has a very interesting experience of being Product Manager in these company. And he has a really interesting experience continuously going through the market with some of the things. So those are things I’d like to explore today. So john, why don’t we start with going on the high level through your career path? Sure. How do you how do you become a product manager? How did you become a product manager? What brought you here and what are you doing?

John Janego 1:33
Yeah, okay.

Well, you know, like everybody, or most people in the field, I don’t think that there’s a real front door. And so it’s not a unusual for me to say that I feel like I came in the side door also. So I’ve worked at this company called Vera code for almost seven years now. Prior to that, I worked in consulting role in a couple in a boutique firm. And before that I worked in a large company. So throughout my career I’ve I’ve done a lot of interesting technical work and gradually started moving towards, you know, more consultative services and joined Vera code in a in a consultative services based role. And after about two years of doing that, there was an opening on the product management team that I pursued. And I feel like still I feel like I talked my way into it. And the rest is history, I suppose. I’ve been doing it for close to four and a half years now, and has been an interesting, an interesting ride with some acquisitions and, you know, seeing our company transition from later Stage startup to for to a more middle aged startup, I suppose is kind of the state that we’re in at this point. And a couple spots in the middle.

Vlad G 3:13
You mentioned john, sorry to interrupt. I just want to make sure I get this out of you. During this episode. You mentioned that your startup was acquired. Yeah, a here and there. Yeah. And I see this as a really interesting challenge for a product manager given that you stayed given that you’re, well first of all, you kept your job and second, the companies that we’re acquiring you your your startup kept the roll. Yeah. Although I’m my understanding is with the acquisition. First thing that is to go first needs to change is the business direction or business goals. So I would love for you to uncover that because I’ve never I’ve never been that situation. Yeah. If you can could just walk maybe walk me through maybe just overview of the challenges that you as a product manager experienced throughout the, through these acquisitions through this process?

John Janego 4:12
Sure. Well, so I think, you know, something that everybody in product management, at least should be keeping in mind as sort of the overall business strategy that your company is in, in terms of, you know, what’s the CEO thinking about? And, you know, how are your decisions as a PM, you know, whether it’s for a whole product portfolio or managing individual features within a product, you know, going to contribute to that business strategy. And, you know, when I started Vera code in 2013, and when I started on the pm team in 2015. You know, we’re very much in the late stage of of a startup You know, we we’re funded by, by venture capital. And, you know, we’re on the track of the typical VC startup, you know, trajectory where, you know, the number one metric the VCs are paying attention to is growth, not not terribly focused on profitability, not terribly focused on anything other than than growth as an overall metric. And you know, that that had been informing the company’s decisions as it informs many startup companies decisions, in terms of the product priorities that they take. For various reasons. The startup stage of our of our company effectively ended by being acquired by a very large software company which which was a an unexpected, but not not actually that that terrible of an outcome to be honest with you.

Vlad G 6:05
The not great, not terrible?

John Janego 6:07
Well, you know, I mean, everything. Everything’s got pros and cons, I suppose. But I mean, I think as a whole, it turned out to be pretty, pretty benign, I’ll say. But I think one thing that was good about it, and, you know, our, the company that acquired us, a company called ca technologies, it’s one of the largest software companies in the world has been around for decades. And, you know, they, they were very progressive in terms of their approach to acquisitions. They were they were acquiring high growth businesses and basically, you know, funding them to continue on that on that growth trajectory and to, you know, gradually start nudging them towards the path of, you know, becoming a more stable part of the overall company portfolio. And so, you know, in many respects, you know, Our our point of view as a as a startup didn’t change all that much when when we were acquired because our acquirer basically said, you know, here, you know, we’re, you know, we’re invested in your success, you know, we just, we just paid, you know, a decent amount of money for you and, you know, you guys keep doing you, we’re not gonna mess with you, as long as you keep up your, you know, your growth plans and, you know, figure out a way to, you know, make this a mutually beneficial type of thing. You know, what we didn’t see coming was that about a year and a half after we were acquired, a CA itself was acquired by a much larger company called Broadcom, which, you know, made the news as being one of the largest all cash acquisitions in history. also made The news because Broadcom is traditionally a hardware company acquiring a an all software company. So, it was very odd I suppose for lack of a better term. And it was especially odd for us in the, in the part of in the part of CA who had been on the acquire e side, you know, you know because we’d been kind of doing our thing and, you know, unexpectedly had this large hardware company as our potential new owner and long story short, they decided they didn’t want anything to do with any of the you know, these startup parts of the CA business and and spun us and and several other parts of the business out and we ended up being sold By Broadcom to a private equity firm who is now our financial backing. So for those of you who have may have heard about private equity and, you know kind of has a bit of a, sometimes a negative connotation to it. Our experience with private equity in our firm is Thoma Bravo. It’s one of the largest PE firms in the world. And they’ve been excellent. They’ve been very positive and, and only only good things from my perspective, but one thing that all PE firms share in common is, is you know, they’re focused on uh, on, on profitability and focused on getting a solid return in a relatively narrow time frame compared to those in the venture capital world. You know, we had typical VC investors going on timescale of, you know, eight plus years. Whereas private equity looks at things generally from the four to five year lens. And after what after those four to five years, they may continue on with the investment. But you know, they want to see, they want to see progress in four to five years. So this is a long, a long meandering way to talk about, you know, orienting the company from from focused on growth as the prime metric driver to focus on both growth and profitability.

Vlad G 10:33
And if you have to pick between the two, you try and balance them, but you know, profitability is still a really important thing to take into consideration. So, no, it made sense. Sorry, it made sense for me that he went all this way to explain kind of like tell a bit of a history, because it makes sense in terms of your responsibilities as a product manager, to kind of address like, Hey, this is why we stop caring about AIG and started caring about a and b.

John Janego 11:03
Yeah, exactly. And, you know, I think, I think for me personally, it hasn’t, it hasn’t had to, as it made a lot of changes in the, in the way that I’ve addressed, looked at the product roadmap and sort of think about the strategy for for our portfolio, but I think it’s important to keep that in mind at the business scale level in terms of, you know, think about, you know, if I have to pick one or the other, you know, what do I optimize for and also thinking about that timeframe, you know, because, you know, in, in the VC world, or in the startup world, you know, it’s it’s sometimes a little ambiguous, you know, what you’re, you know, what your goals are, you know, what the timeframe in terms of like an exit horizon is, at least it’s, it can be ambiguous and like, You’re part of the executive team. And even within that executive team, it’s sometimes you know, it, you know, it’s, it’s not always completely everyone’s not always completely aligned on it sometimes and within, within this situation, you know, everyone kind of is on the same page of, you know, we’re trying to, you know, we’re trying to make the company as as grow as much as we can and, and keep it as profitable as we can. Which I think is I mean, overall is it’s a it’s, it’s, it’s nice to have that clarifying, you know, you know, where you stand type of thing.

Vlad G 12:42
Oh, definitely. Yeah. I agree with you. It’s always it’s always nice to know where you’re going with this and especially when you’re talking to C level execs and a try to understand. So what is it that we’re doing again?

John Janego 12:56
Yeah, exactly. And you know, and I’d worked at a start up Before that, like was acquired, like, a couple months after I left and it, it had never been spoken of frankly, like, what the what the plan was, you know, from a liquidity perspective, you know, and so I think, you know, I was probably a little naive about it, you know, but at the same time, it’s it’s like, once you’ve been through an acquisition or, you know, a couple acquisitions, you know, you start learning a little bit about, like, the business aspect of it in terms of, you know, this is, you know, this is this is how these things go. And this is what you need to be thinking about in terms of, you know, in terms of, if you have to optimize for one or two things, here’s what you should really be focusing on because that’s what the people who are paying, you are really focused on, you know, funding you rather, you know.

Vlad G 14:01
Well paying for you or funding you. Yeah, yeah. Same thing.

John Janego 14:06
Well, one’s pay for us. But you know, the the equity firm is the one funding us. Yeah.

Vlad G 14:14
Oh, I see. Yes. The Thank you. Yeah, that’s, it’s an important distinction. And I agree with you. Makes makes sense. Alright, so having said all that, and you mentioned that at some point, after the first acquisition, your job or your focus didn’t change much. But then at the second one, it did. Yeah.

John Janego 14:39
It did a little bit. You know, I think I have a pretty luxurious position of having one of us of having a pretty strong performing product. Um, I’ll say some of the things that I that I observed just through the course of acquisition was there’s um, You know, there’s changes in the company and you know, just from a personnel perspective and you also learn about your coworkers on things that you may not have otherwise learned about them, I guess also is one way to say you know, just in terms of, you know, people’s attitude towards work, people’s tolerance for risk, that kind of thing, you know, you know, and some people are more comfortable working in startups and you know, after an acquisition, just want to leave, you know, they consider the exit to be, you know, the prize and then go to the next, you know, the next opportunity to do it. Some people are more interested in building the long term, you know, thing or, and, you know, and then other other people are justifiably a little I’m sure about what to do you know what this all means for them on a personal level to, you know, when when you see these type of things happening. So it’s a, you know, it’s, it’s, it’s interesting, it’s an interesting shared experience that I think I learned a lot about myself and a lot about just business, I guess, just having been a part of and paying attention to it, you know, and as a, you know, I want to be realistic, you know, a product manager isn’t is not like a critical person at the negotiating table when, you know, these things are happening. But you know, I think in a way, won’t be I wasn’t negotiating our exit or anything like that. But you know, like working and working in this part of the business, you have a little bit of a wider angle perspective on things so you get to empathize with those how those negotiations happen and can think about is sort of like the Yeah, maybe this actually is the best thing. For us, or, or whatever, you know, which you know, other parts of the business, you may not think about those or if you think about them, it’s it’s not as part of your life. It’s Sorry, it’s not as part of your job, you know, it’s like, it’s my responsibility to think about these as part of my job. So it’s been an interesting learning experience that I’m glad to have been a part of for sure.

Vlad G 17:21
I can somewhat relate to that. And again, somewhat being the key word here I was I was present in some of the negotiations, not at the exit, but that the Partnership negotiations, I was somewhat present again, not on all of them. And and it was definitely an eye opening experience. So for the folks who don’t have the folks who don’t participate, the folks that are not present, everything seemed to be very shady. And what I’ve learned is my personal my personal view on that, nothing is what it seems absolutely nothing. thing you think that’s going on there is going on there looks like a completely different picture, whatever you’ve imagined yourself. It’s not what you think. Yeah. This is one of those cases when it’s totally not what you think there are other things that are happening, but not what you think.

John Janego 18:16
Yeah, I think the other thing that I would add on to that, you know, if anybody is in a startup, that and seeing this kind of thing happen, or you know, experiencing it themselves to be like, don’t take anything personally, that that goes on, whether it’s at the large scale level, you know, like, if your company is acquired, great, you know, or, or if it’s at the personal level, like, you know, your boss can’t tell you about what they’re doing on their last business trip, you know, like, that’s just how it is, you know, there, especially when it comes to, you know, high stakes financial transactions. There are there are laws and there rules and there’s a lot of people’s livelihoods tied up in it. So people are often operating under some pretty strict things that they can cannot say just so they don’t screw it up. You know, like, you’re talking about hundreds of millions of other people hundreds of millions of dollars of other people’s monies. Yep. You know, so one, one tweet by a pm who heard about it could screw up the whole deal, you know? So it’s, it’s businessman, you know?

Vlad G 19:37
Yep. Yeah. No, I agree with you completely that on that on that front. And a lot of things can go wrong. And they sometimes they do if you’re not careful who you say, yeah. who you talk to what you say hi, even how you say it. Yeah. Okay, cool. Thank you. That was that was really interesting. And thank you for sharing that. That It’s not every day you get as a pm you get to sit at the higher table. Yeah. And and and I appreciate that we appreciate somebody sharing, what was their experience? Because I didn’t have that bash of mine. I mean, not all the time. So I think this is a good time to switch gears a little bit and talk about the actual products that you’ve worked on. Because I think there’s a there’s a story there about what went well, what went wrong, how can we you know, leverage that knowledge for you know, for the good of mankind as the whole end product manager is particularly

John Janego 20:41
Yeah. well, so I’ll I can talk a little bit about the product I work on and our company as a whole. First off so Vera code is an application security testing company. We sell software to other people. companies to help them test for security issues. We have about well, we have six parts of the product portfolio. And I manage one of those products. Our, which is our oldest one, it’s called static, static analysis. And it’s without sounding like tooting our horn about it, I’ll say, you know, we’re one of the we’re one of the most successful companies in the space. By, you know, a number of metrics, we have pretty solid growth, we have a large number of users a good opinions of us with the analyst community and you know, happy customers and stuff. So, uh, you know, it’s a it’s an interesting business to be in Just adding information security as a whole is an interesting business. And application security in particular is an interesting business.

Vlad G 22:09
Let’s let’s focus more on products and on your experience working within the portfolio. So as a person who did all of the above, I’m really curious to see, like to hear how you guys manage your portfolio being the individual product, product managers on individual products, how do you synchronize your efforts, if there is synchronization required, or the you guys run in each, each of you running in your own separate directions? with the hope that was, you know, 20 years later, we’ll get to meet somewhere down the line.

John Janego 22:49
I’ll say that, you know, we have a we have a high level, you know, product portfolio level, you know, vision that we that we’re still executing, but The individual product teams within them is sort of a parallel play type of thing. If you’ve got toddlers, you know the phrase in that, you know, you’re kind of doing your own thing in the same sandbox. And sometimes, yeah, you know, you hand a toy off between the two of you. But oftentimes, it’s kind of you’re, you’re moving in the same general direction, but doing your own thing. And, and a lot of that is, is just from a, frankly a technology perspective, you know, we have, you know, the, our market is segmented fairly into a few different product categories, which which have some degree of technology overlap, but but not often, a lot of that. So, as an individual product team, there’s, there’s frankly, not a lot of overlap between something Like the core, you know, team of one product versus another is working on. Instead, what are our one of our core, you know portfolio differentiators is is that we have a single platform through which you can access or interface with all of your all of the different testing solutions that we offer. So that sort of serves as that connective tissue between the various parts of the product portfolio. About the teams that I’m working with, you know, is is primarily, you know, focused on, you know, doing, doing our core static analysis business, and doing a deep and not overlapping with a lot of our colleagues doing their own businesses also makes

Vlad G 24:49
sense. It makes it makes sense in the larger grand scheme of things. Yeah, it makes sense because that’s what I’ve seen in most of the As I’m thinking about different industries, not just your particular niche, or my particular niche, it just seemed to be the smart thing to do when you’re not kind of cannibalizing your own your own customers, your own audience. So if you’re at all only seen one company, that kind of stuff, stood up to product teams, and had one company support legacy product. And then another company, literally was off to the races to cannibalize the existing market, from the legacy product in in a, I don’t want to call it a hostile matter, but it was kind of because of the personalities that were behind those products. It was more or less hostile matter. It was kind of like, no, we’re going to take over your customers, we don’t care about you. And and and the team was really, you know, and weirded out if that’s the if that’s the word.

John Janego 25:55
That doesn’t sound like it’d be a fun place to work, that’s for sure.

Vlad G 25:58
Yeah, it doesn’t. If it was not meeting with teammates, you know, or other teams, so it was not and it kind of, you know, it was kind of unintentional, but I have a feeling was not entirely unintentional, it was kind of like, you know, C level execs gathered in the quarter office, you know, had a bottle of wine or something, we’re like conversating. And one of them said, Hey, listen, would that be nice if we kind of like, tried to introduce both products and see which one is better? And the other one would say, Yeah, sure. That sounds like a great idea. And they did exactly like they. They said,

Unknown Speaker 26:39
I mean, there could be a certain amount of, you know, brutal efficiency to that, but it may not be the most pleasant place to be working, I’ll say, you know, so it takes a certain type of people, certain type of person to want to do that, you know, yeah. Yeah. Well, yeah. What I’ll say is, you know, Want to add on to some of, you know, what you’re asking, you know, like us as an organization, you know, you know, in spite of our teams working, you know, fairly independent of each other. We’ve been trying to align more closely, at least institution or I guess in a process wise, because because of having this shared portfolio or the shared platform layer, and a fairly complicated product, technologically speaking, there are often you know, cross team dependencies and things that, you know, kind of can come up and, you know, let one team block the other inadvertently. So, you know, our one of the things that we’ve been trying to do in the last couple years has been to, to be a bit more Organized at a, as an engineering and product team level in terms of identifying those and playing together in a in a big group, you know, because we have a pretty large engineering team we have about 200 250 engineers or so and a relatively small product management team about you know, 10 people but you know, still it’s a it’s a fairly big organization to manage across and so one of the things our team has been rolling out well has rolled out is is the scaled agile framework, which is which is which has been interesting, you know, I think I know it has its, it has its fans and it has its detractors. I’m, I’m kind of in them, ambivalent about it, honestly. I mean, I think you know, my, my observation has been that it’s It’s been, it’s been good to have a neutral mediators in the form of essentially like the product project managers, you know, trying to look at the team at what we call the portfolio level and identify cross team dependencies. And, you know, and and helping us roll out processes, you know, because if it comes you know, because if it’s being rolled up by one team onto another team, you know, that sometimes there’s you know, some feeling of, you know, who are you to tell me what I’m supposed to do type of thing and, you know, having having a neutral, a neutral person or neutral team in the middle, helping roll it out, you know, for the benefit of the organization has been very helpful. And then, you know, other sort of maybe, what might seem like mundane things, but important things like having having Are sprint calendars be consistent between different Scrum teams and consistent, you know, naming practices and amongst our releases and, you know, things like that, you know, that, you know, you can overcome those on it, you know, on a case by case basis. But, you know, as the organization scales and you know, as, you know, people, people move in and people move out of the company and stuff, it’s very, it’s helpful to have, you know, some consistent processes that are not just like institutional knowledge but in our instead is something that’s defined and you know, everybody’s kind of following you know,

Vlad G 30:44
Yeah, I agree with you and I’m not you know, I’m not a fan. Okay, that comes out as wrong. I’m not not a proponent of scaled agile, I’m more of a I like instruments. I like to apply instruments where they belong. So for company if a team is large enough, and I think you mentioned 250 engineers, scaled agile makes sense. Mm hmm. We’ve tried, we actually implemented a scaled down version of scaled agile, and I know that sounds. For us, we’re a smaller team who the purpose of scaling up. Yeah. So it was it was done with purpose. So that exactly like you said, so it’s not an institutional knowledge. It’s a standard processes. Yeah, they’re implemented within the delivery or engineering organization, with the purpose of you no matter how many people they are, you know, 510, two teams, five teams, 2030 teams, it would be still the same process and scaled agile is really good to them because it works on a small level, and it scales up really well, which is why they call it that. Yeah, I

John Janego 31:46
Yeah, I was gonna say and, you know, like, I think one the other big advantage I’ve observed with it, as has been that, you know, for better or worse, it’s documented, you know, it’s um, you know, you know, There’s not a single, there’s not like a template of how you roll out agile. But if you’re looking for a template, you know scaled agile is one of the few that actually has processes around it, you know. So if you know that you need to do better at agile, you could either you could go a couple directions, you find a freelance agile coach, you find, you know, a couple solid Scrum masters who’ve done it before. Or you could find you know, some coaches who have rolled out a process like scaled agile and then use that as a guiding principle and flex it to whatever matches your organization. You know, and I’m not an agile coach or I frankly don’t care I care most about getting products out consistently and, you know, predictably, but, uh, you know, it’s a, it’s not a religion. It’s just something that you got to use to make to serve the ultimate goal. So it’s not wrong or right. It’s just you got to be organized. And it’s hard to be organized if you don’t have a template to go from sometimes.

Vlad G 33:08
Yeah, I think I think what you’re trying to say at the beginning is that the the good part about the scaled agile is not that it’s documented is that it’s documented outside of your organization. So you can always refer to you refer to it, when a new person comes in. You can say, Hey, this is what we’re doing. You don’t have it’s not like we’re doing our own thing. Yeah, it’s just, you know, one of the public things that we’re, we’re doing and it makes sense.

John Janego 33:35
Yeah. And, and then, like, you know, if there’s ever, you know, one thing I’ve read about in other organizations is, you know, like, title inflation or ambiguity is about what titles mean or, you know, that kind of thing and, you know, at least again, like, as long as you’re not too zealous about it, you can say, Well, generally, if you’re looking at what does this Scrum Master do versus what as a product owner do versus what a product manager do. You know, the scaled agile definitions are a good starting point for that, you know, versus like, but sometimes, you know, you can be interviewing candidates and, you know, their definition of what they did have may have very small relation with what your definition of what you expect that, you know, role to be, you know, so yep. I,

Vlad G 34:29
this is this is the part where I agree with my guests. Not Not Not what I intended, but it’s good to agree on certain things sometimes. Yeah. So let me ask you this. Let me get back to you. Let me get back at you. Since you’ve mentioned product owners and product managers. Yeah. And I’m always curious to see how your organization defines those roles. Like, is there a difference? Is there a similarity? Is there an overlap? Tell me.

John Janego 34:56
It’s complicated. I think we’ve had on the even rollout of product ownership roles, I’ll say, and I think part of it is, you know, our company has a very, very strong engineering culture, you know, it was it’s a very technical product and it’s got a really very strong core engineering team and, and a lot of desire to keep a sort of a light touch between the product management organization and the engineering organization in terms of like, defining what, you know, how the, how the pieces get built, so to speak, you know, so, the product owner roles, you know, we don’t have POS for every single team and the POS that we have, you know, are kind of play closer to a junior pm role to some degrees in in terms of defining the overall You know, making those day to day prioritization decisions about stories, but but it really just depends on the team. You know, I don’t it’s hard to generalize just because, you know, I’m talking about like eight or 10 different Scrum teams and, you know, so it’s a little bit different from every team. So you would say in every team, you have a slightly different scope for the product owner role is that what it is, so the product owner roles that we have align more with the overall product portfolio and, and are working with multiple Scrum teams in the same way that a pm is working with multiple Scrum teams. The difference from a day to day basis is that ultimately the pm gets the final decision in terms of the priorities. If, or you know can cast the tie breaking vote pios do more day to day stories. Writing and, you know, Sprint working with a scrum masters for sprint planning purposes. And but we, we try and stay pretty tightly aligned, it’s a little bit more like co pm to some degree, but I’ll just say like the, our organization is relatively small, the pm organization is relatively small, especially compared to the size of the overall engineering organization that we’re working with. So, we we kind of just, we try and work as make make product owner role be as more as much of a force multiplier as it can be, you know, because, you know, like, for example, like, you know, I work with five separate Scrum teams And sometimes they have stand up at the same time, you know, so, you know, the PEO will beat it will be at some of them and I’ll be at some of the other ones or that kind of thing. You know,

Vlad G 38:10
you mean the PO would have to be on the same five teams

Unknown Speaker 38:13
sometimes but you know, we’ll split between different teams doing different you know, depending on what what needs to be done for that given

Vlad G 38:21
that given sprint or the stage of the product release that we’re at or or make make sense so so you’ll be always not really team member he the your your appeal is more like a junior product manager who’s, who cares. Let’s maybe dramatize this a little. He cares your pod cares more about the product than he does 14.

Unknown Speaker 38:40
Yeah, I mean, the product owner works on the product management team, not okay engineering team. So okay, good. Ultimately, ultimately, they’re representing the product management teams point of view, rather than the engineering team’s point of view. Gotta go. Okay. That’s not the position as adversarial but you know, it’s like that that that tension that’s always yeah prize I agree and hopefully healthy between engineering and strategy teams.

Vlad G 39:12
Right and then that’s, that’s the good part because if you know and we’ve seen this in let’s say, I seen this in my engagements when it becomes order takers, they have no say in what to do and they end up not having a say in how to do things. Yeah. So they can be actually end up facing technological challenges that are impossible to resolve. Yeah, build this with this, you know, build me, you know, make me a shovel out of sand and you looking at this like what? Yes. And this is interesting because in my head, in my experience in my product mindset that I kind of advocate, a product owner role on his on the scrum team, so product owner isn’t interface of the team between the team and the product management team or between the team and the business, but they’re part of the team, they’re not a part of the product management, they have, you know, intimate knowledge of a product, they have intimate knowledge of technology. But in my, in my product mindset, my, my meaning what I advocate for, yeah, there on the on the other side of the fence, and it’s interesting to see how you guys are still successful, which basically tells me, you know, it’s not the part where I agree with you, it’s not a religion, they just, you know, whatever the right tool is for the job. And if it works for you, then, you know, there’s, there’s truth to it.

Unknown Speaker 40:39
Well, you know, it’s, it’s, you know, it’s not like it’s all roses, and I don’t I’m definitely not saying all the all the great things that we’re doing and all of that, but it’s, you know, it’s it seems, you know, it’s it’s worked okay. I mean, I think there was it was a subject of much debate when we even because Product Owner is a relatively new role within our organization. You know, Once upon a time, it was only PMS. And we worked with engineering managers as the day to day product owners, effectively. So when we decided to start hiring product owners, it was a subject of much debate between the product, the product management, leadership team and the Engineering Leadership Team about where those individuals would be reporting into. And, as it turned out, in this case, you know, the the POS ended up reporting into the pm organization, but I’ll say you know, like, the PEO role is, you know, still needs to be a close ally and partner of the engineering team probably closer than the pm needs to be, you know, because, you know, where there’s been challenges with with various product owner roles I’ve seen has been when there were You know, there were mismatches and expectations or conflicts between the product owner and the engineering team. Whereas you know, the pm it’s it’s great if everybody’s always agreeing as a pm and engineering team, but sometimes it’s okay to disagree. And that’s normal. But the P o and the engineering team have to be mostly agreeing at the same all the time, otherwise it can go downhill pretty quickly, I think. Uh, I

Vlad G 42:39
don’t know. I don’t know. I mean, again, I’m not saying this is this is the truth. I’m just kind of voicing my opinion. What I’ve noticed from my experience, working with engineering teams, working with product owners, and working with business analysts who are Kind of proxy product owners, but not really. What I noticed is that team needs a representative. Yeah. And that’s, again, that’s just my experience. I’m not saying this is a universal truth. But you need a representative who understands the product and understands the business, but is still a part of the team. And this being part of the team is really important. Yeah, it’s kind of like a group spokesperson. Right? They can be assured. And that is done through that reporting mechanism or being a part of the team mechanism. However, the organization is structured. Yeah, they can. They need to be assured that that person has their best interests in mind. When I was product owner. I was the part of the team. I we had a massive reporting like I was a consultant, as a matter of fact, from a vendor, who was a managing team of people from the company and people from other vendors. So reporting went out of the window. Yeah, completely. All that mattered was that I As a part of the team, I am part of the engineering team, I understand the technology, I understand the pains that they go through. Yeah, but I also understand the product, I also understand the business. And when I went to bat for them, and it really was I went to bat for them against the business owner. We didn’t have a product organization, we had product owners, and we had business owners was very, very vanilla, a scaled agile framework, circa 2014. I think, if I’m not mistaken. And what happened was, team knew that they can rely on me defending their interest. So if example if a business owner says we need to do X, and Tim says, You know what, he’s crazy. And this can only be done in this way, and then it’s going to take five years to build something reasonable. Yeah, I would go back to that to that person, to the business owner. And I would say what you’re asking for is impossible, not because we don’t want to do it, but because There are real reasons for that. And I was able to communicate with them on their life in their language on their level. Yeah. But from the technology from the technology team standpoint, yeah. And that was kind of a very valuable part. And to me was the lesson, why it should be a part of the team because I wanted to be in the business side, I wanted to become a product manager. But it was very important to me as a product owner to be on the side of the team, because then team trusts me to go and do things for them.

John Janego 45:28
Yeah, I think that that’s a good point of view about it. I mean, I am, I may have missed this phrase or misspoken a little bit and saying they always need to agree, which is might be, but I think what I meant more is that they need to be on the same page, you know, in terms of trust, trustworthiness and right. But you know, I think, you know, it’s, uh, you know, I’ll say, you know, our experience has not been a smooth one with product owners and that’s not for a lack of having great people doing damages, we organizationally haven’t figured out the best way to incorporate that role into the, into the team. And, you know, the one that you described, you know, sounds like a, you know, a good model to it that we might that might be worth thinking about, you know, so I’ll say one thing that is interesting, perhaps about or maybe interesting, but you know, like our, we tend to have pretty technical product managers on our team, or in our company as a whole. We have a very technical buyer and a very technical product. So, so our PMS are often serving as proxy customers. Which is true across the entire industry, but because we have such a technical product, we have two fairly technical product managers as well say you know, so um, The EPA empathy sizing aspect and with the engineering team comes naturally to some degree. You know, I feel like I understand how our software is built pretty well. And don’t just come and say, like, do it engineers, you know, like, I don’t care, you know. But that might not always be the case in every industry, you know?

Vlad G 47:22
Yes, I agree. It’s not it’s it. I know for a fact that it’s not and I’ve seen it. I hate to generalize. But in many cases, let’s put it this way. In many cases, it It derives it from where that product manager came from. Yeah. As an example, in, in my organization, I have a lot of product managers who came through as business analysts. Yeah. So they’re very focused on the processes. They’re very focused on using the right frameworks. They’re very focused on doing things by the book, because that’s how they were brought up. I came from I was a software developer for, I don’t know, 1215 years, then I was managing software development team. And then I kind of sort of should have been a project manager by title. But I ended up doing product management work, product ownership, Product Management work. So what happened was, I came from a technology side and I could always, just like you said, empathize with the technology team, because I was a technical person, same way you guys are, yeah. And what I noticed was, I’m way more let’s say I’m really more focused on experimentation. Let’s try this. Let’s try this. Let’s try this. Whereas people from a project management background are really risk averse. They don’t like experimentation because oh my god, we’re gonna fail. Yes, you know, they’re scared of failure because my project fail. So it’s really, you know, a double edged sword. Yeah, kind of like depends on what kind of culture you have, what kind of people you have and where they came from. Yeah, so it’s a good thing. You guys are old technically. You guys are already on the same page, I think that’s a very positive thing. You just kind of, you know, need to warm up, again, from my point of view, not that you’ve asked, but I’m gonna throw that out. Or you should definitely think about how you guys build that trust relationship. And because the whole point of all these processes, the whole point of all these mechanisms, being in place, is not to is not to really, you know, resolve things when everything’s going smoothly, but to resolve conflicts whenever there is one, and there was always going to be conflicts. Yeah. And what I think was actually my next question for you would be, how do you define the product manager role and in that in the formal way, here’s what I mean. Here’s what I mean. I always say I usually say, a product manager is the one who’s not afraid to say no to anybody.

John Janego 49:56
You might be afraid, but you have to do it. Well.

Vlad G 50:00
It means it means you still saying it, right? That’s what matters. And I’ve got I’ve got some slack when you go into consulting assignment, you can really say no, yeah, to the customer, you have to say, well, there’s another point of view on that. But as a product manager, you end up saying no, because that’s the right answer. That’s how you not waste anything that time the resources. You know, you’re not you know, pretty things up note, we’re not doing this, because that’s focused on something else. And that was really interesting. Guests experience or or approach? Yeah. So what are your thoughts on what is what is it the product manager that how do

John Janego 50:37
you define the product? Yeah, yeah, I mean, I think it’s, uh, you know, when come into product management, you know, fairly late in my career, not late, but, you know, in the mid stage of my career, you know, I’ve been working for 13 or 14 years before I started in PM, you know, and, you know, I think the what really hit me when I started Uh, you know, was, you know, you’re the decider, and I know that sounds like a stupid thing to say, you know, but like, ultimately, you’re the person who’s gotta define what the, you know what the next thing is. And, and that, you know, that can vary from, you know, the, the, the aperture on that can vary depending on on on the stage of the product or the or the, you know, the the month or whatever, you know, but, you know, it’s like, what I really am trying to do is, is making the team, the engineering team, continue to have a successful product that they’re working on and make the product that that team is building continue to derive success for the business, you know, so, you know, like you I came from a pretty technical background, and so I empathize a lot with our engineering teams. You know, when I was working in software development, you know, the thing I hated the most was not knowing, you know, what my user what my users were doing or what my users thought of the product, and, you know, what impact software I was working on had on the business and, you know, so I always think about serving both sides of it in terms of I want, I want my, I want the business to do as successful as possible, obviously, and, you know, that comes with all of delighting users and, you know, elbowing out the competition and all of those aspects of business, you know, but also, you know, keep the engineering team have having success, you know, like, I read this book a couple years ago, and I’ve read it again, called the hard thing about hard things, which is one of the best business books I’ve ever read by Ben Horowitz. And you know, One of the things, one of the things I took away, one of the main things I took away from that that book is, is just thinking about your role as the leader in the product organization as as trying to be, have a sense of responsibility to your colleagues, in addition to the responsibility that you have for your users. Because, you know, if you serve your users, well, you are also serving your colleagues well. And I try and keep that in mind. In every day, you know, sometimes it’s hard when you’re in the weeds disagreeing about things, but I try and keep that perspective in mind because you know, for right or wrong, you know, you’re the person making the decisions. That’s just that’s literally the job that you’ve been given. So I try and take those decisions seriously, so that we don’t waste time on things that aren’t important. And and, yeah, they have a Good implant to all parties involved, you know,

Vlad G 54:03
makes sense. Makes perfect sense. Thank you. Alright, so let’s move on. And I, I’m looking at my little thing here that we have a list of things to talk about. You mentioned something about challenges where you ran beta programs. Yeah. And when you when you were deploying new capabilities of existing product, and that’s kind of a very dear topic to me because I, I live this every day. Yeah, with my current product portfolio. And I live this in my previous life where I had to run after people who had no idea what I’m offering and why am I even talking to them? asking them to be my beta customers, but I had no idea what the hell being a customer is. Yeah. So tell me, what is it? What is it that you do it? Yeah, in that sense.

Unknown Speaker 54:51
Um, so Vera code is a has a pretty complicated product offering where you We we offer are one of the product that I work with is effectively taking apart our customers applications written in a wide variety of software development languages and telling them potential risks that might be involved in that so or in the in the software that they’re writing. So the process of adding support for a new language is a complicated one. It’s requires a really specialized set of skills from just a straight up r&d perspective. And the other thing is that you don’t really your research can only go so far. When you’re thinking about, you know, when you’re when you’re when you’re trying to determine what risk looks like. Until you see real world applications because, you know, our researchers are excellent at something Some of the smartest people I’ve ever met, but at the same time, you know, it’s only a couple people’s perspective on what bad things that could be going on in user software could be. So we have often run what we call Early Access Programs when we build support for a new language. And one of the ways that we license our software is, is basically it’s a it’s a subscription model, and we’re releasing features every every month. And included in those features that we’re releasing as support for new languages. It’s not we don’t license by language or you know, only let you do a certain number of things. You know, that based on when you build by your license, you know, if you buy it in January, you get all the features that we released between January and you know, the next 12 months on your contract, you know, whether it’s nothing or Whether it’s, you know, support for five new languages, and the reason I bring this up is because, you know, a core part of making a language of making support for a language successful is to see real world customer applications being tested so we can understand what those applications look like and make sure that we’re reporting risk properly. And that and seeing that benefits all of our users this has been a phrase on used recently and in different circumstances, but it’s kind of like a herd immunity type of thing. You know, we’re a SaaS offering. We test thousands of applications. And it always we see patterns of risk in some applications. If we can use those patterns to inform the what we’ll see in other users applications as well. And So the tricky part though, is starting from scratch from zero, you know, if we don’t support a new language, you know, such as Python, or something, how do you go from support it or not supporting it to supporting it, while also retaining the level of quality that your users are expecting from you. And the way that we’ve we’ve been doing this as has been through these early adopter programs where we basically, you know, gauge the interest from customers who know we know that are using these languages and you know, get them to participate in by by letting us test our stuff before we roll it out to the broader audience. And I’ll say you know, it’s a it’s definitely a mixed bag a success. And partly is just a nature of the product, you know, it’s it requires engaging a lot of different members on the customer side, and Sometimes it’s difficult to find new users if they for, you know, like, our user base is often software developers. So if we’ve, we’ve sold into a company that has, say, a Java dev team, a Python dev team and Dot Net dev team, we’re engaged great with the Java and the dotnet dev team, but we don’t know any of the people on the Python dev team, because we didn’t, you know, we didn’t support we didn’t have anything to offer them, basically, you know, and then suddenly, we have something off for them. We’ve got to build those relationships within the customer to even find the people to work with and then, you know, build trust with them in order to get get them to understand that we’re offering them something new as an experiment. And you know, men, you know, meter their expectations. So, it can it can be complicated to to find that level of proper engagement. And

Vlad G 59:58
let me let me ask you a quick question. Well, can you guys have a Salesforce that goes out and builds those relationships bizdev or sells more than one of those two?

John Janego 1:00:08
Yeah. So you know, Vera code is a b2b product. And, you know, we have what I’d call us a fairly traditional enterprise sales engagement team, you know, we have,

Vlad G 1:00:19
alright, so when do you guys start? So I’m just trying to understand the process when you guys start sharing your roadmap with them. Okay. Sure. Yeah. So.

John Janego 1:00:29
So one thing that’s might be unique about Vera code is that we have a very large services organization, you know, I mentioned that came up through that organization. Yep. Initially, we have about 100 people on the services organization. So all of our customers have a customer success manager or a program manager depending on you know, the level of engagement that we that the customer needs, who’s our day to day point of contact with with that user and, you know, we publish a roadmap. That’s About nine to 12 months forward looking and we don’t publish it on public internet, but you know, we’ll, we’ll share it with customers and I’m in pretty regularly. Basically, whenever a customer wants to talk about the roadmap, I will gladly talk with them about the roadmap. And, and that’s been one of the ways that we’ve actually engaged customers, or throughout the course of learning about doing these programs has been to engage customers in them in these beta programs or early access programs, you know, via the roadmap where we say, you know, like, this is something we’re planning to do, you know, release a couple of quarters from now. So start thinking about this is something that you’re interested in and you know, start building out that that engagement early and you know, it’s it’s a it’s it’s established a pattern where we’ve, we’ve we have our users or our program managers, you know, as proxy for our users often anticipating that We’ll be in Early Access Program, whether we do one or not, you know, which I guess, is a good thing, because it’s at least the users are coming to you rather than you having to beg them salutely. But at the same time, they’ll all say, like, it’s one thing to say you want to participate, and it’s another thing to actually participate, you know,

Vlad G 1:02:19
right, how much how much of an effort is, is how much of an effort does it requires from your customer to participate in this early access program? I mean, it depends, you know, it

John Janego 1:02:31
depends on the product, you know, I mean, sorry, guys, it depends on the specifics of the given feature, but you try and incorporate them as seamlessly into the existing product as we can. Um, but it kind of will depend, you know, we just released a new feature that required a little bit more engagement because it was something that was a new thing that we that we were doing that that wasn’t related to, or that was not as incorporated with the rest of our product suite. So that required a lot more deeper engagement with the users than otherwise. Other other programs would have been but I mean, realistically, the level of engagement tends to be a lot on behalf of the product manager, you know, because, you know, as the person defining where we’re, you know, where the product is going and you know, most involved in the day to day with it, you know, you’re the person who can speak about it most intelligently, you know, not not a salesperson, not the sales engineer or not the program manager, it’s, it’s you and it’s, it’s, it’s sometimes it’s you versus you know, 50 customers type of thing. So, you know, I used to have weeks where, in the in the heat of an Early Access Program, I’ve got literally like, you know, 12 customer calls a week, you know, type of thing. Plus doing all my normal day to day job, right? You know, we’re in the backlog and stuff and

Vlad G 1:04:05
Yep, that’s, that’s normal. That’s normal. So let me ask you had there ever been the case or however you wanted to find this? When you rolled out something in your early access program, and then based on the user feedback, you had to scrap it? I said, Nope. Not gonna do it. How did you handle that?

Unknown Speaker 1:04:24
Well, we’ve definitely slowed the release of things before, um, because based on user feedback, because, like I was alluding to, you know, our research process can only go so far, you know, until you actually hit customers, you know. I’m always reminded of a quote, I heard, I think, from Mike Tyson, saying, you know, everyone’s got a plan until they get punched in the face. You know, it’s like you’re Your product may look great until you actually have a user use it and they tell you not graded. It’s you know, so

Vlad G 1:05:08
Oh yeah, I’ve definitely been there.

Unknown Speaker 1:05:10
We so we’ve definitely slowed the release of things. My predecessor in this role I know definitely canceled a planned release based on some early access. It’s hard to say whether that was the right or the wrong decision. I’m not gonna hindsight check somebody you know, who did the job before I had it but um, it’s but it’s definitely happened to all say, you know, but the converse has happened is also where we ended up just starting an EA program. And after getting some low ish level of participation, decided to just release it, rather than keep dragging out the beta program because you know, at a certain point, you can say, the real way to test the customer. response to this is just to stop letting a trickle in and just let people into it themselves. And that doesn’t come with its I mean, that that comes with its own set of problems, because, you know, you may have released it too early and had some, you know, deficiencies and stuff but, you know, at a certain point is kind of like you gotta, you know, I really try and avoid the sunk cost fallacy, you know, but you know, a certain point, you got to say, well, are we going to do this thing? Are we not going to do it? You know, so after you’ve been working on something for, you know, six months or something I’d read I’m always biased or just like releasing it. And, you know,

Vlad G 1:06:37
you see what happens? Yeah, I know. I agree. And I’ve been, I’ve been the same, I’ve been the same of the same mind. Most of the time. It’s just sometimes certain things. You’re just can’t can’t release them because although you get the data that you want, and I keep repeating this. I’ve heard it somewhere that there are no failed experience. They’re experiments reaching data. Yeah, and I keep I keep saying like, but we’re gonna learn so much even if we fail. There’s there’s a the other side of the coin here and that is damage to the brand. If you fail too much in the eyes of a public of the of your audience, they will start thinking you’re, you know, you’re just lower cool your product, you’re always releasing something. And I’ve seen this happen to do a number of brands where they you as a product manager, I can clearly see that they’re experimenting as a user as a you know, like, I don’t really care what drill I buy as long as the next holes in the wall. Yeah, I don’t really care. I I’m I’m upset that I can’t get things done or things that are getting done are buggy, or I had some issues.

Unknown Speaker 1:07:50
Yeah. I mean, Google is a classic example of that. Right, you know, like, they know,

Vlad G 1:07:56
they’re kind of like in the league of their own.

John Janego 1:07:59
Well, yeah, but anyway, Like, they get held up so much as like this, you know, super innovative tech company in which they absolutely are. But at the same time, they will, they will release half baked experiments, get users to use them and then kill them. You know, like, yeah, that’s with apparently no, apparently No, Ill effect the brand because those users are not actually the users that they care about. But that’s,

Vlad G 1:08:25
I, again, I’m not sure I would completely agree with that. And here’s why. And, again, this may be an edge case, but just just hear me out back in the day when Google introduced the Google Docs and the whole online editing experience. And I don’t know if you remember it, there was this thing called Google Wave.

John Janego 1:08:47
Oh, of course. Oh, yeah. It was insanely amazing.

Vlad G 1:08:51
And everybody, everybody was going crazy. Like literally people were dropping Microsoft Office. like crazy. People were like, Leaving Microsoft Office Oh, finally we can do stuff online. And for free, we get things for free and align and feel free. I mean, it doesn’t get any better. Right? It was it was a pretty big movement, and I haven’t checked the financials. But yeah, I think Microsoft noticed. Let’s put it this way. I don’t know how much there was financially, but they’ve noticed. And I think that I think that oh 365 is doing better than G Suite these days, though.

Oh, you just spoiled my story.

John Janego 1:09:33
So Microsoft noticed and they and they and they told me one Yes,

Vlad G 1:09:37
absolutely. Yes, you’re right. You’re right. That’s that’s what I was getting at. Not only they noticed that they took, they took care of that they created their own offering. Yeah, but the actual the actual trick here is that because Google has this notion of which is going to cancel it. Oh, you know what you have, you know, six months to get out of it and we just gonna cancel it. was something reasonably comparable was available for Microsoft, everybody ran back. And that that is a kind of a yes, Google is most innovative company, they probably spent on innovation more than next five companies in the row. But you can’t you just can’t trust your business continuity on the company that just cancels things out just because yeah. And that’s why Google Docs is Google Docs. And oh, 365 is basically every business I know is is either using a Microsoft Office, or office 365. or something of that nature. Yeah. Almost no one uses Google Docs. And it’s more or less a stagnant product at the moment. I I’m using I’m using Google docs for some of my personal stuff. I barely see any improvement from what was there three, five years ago.

John Janego 1:10:50
Yeah. Yeah. But it’s probably not core to the business though. I mean, if you look at their This isn’t a Google podcast, but you know, it’s They’re the it’s not the core of their growth. You know, it’s not a core core, right.

Vlad G 1:11:04
But it’s a product and it’s a product. It’s a very interesting product, given that, you know, Microsoft has been around for ages since last century, right? They came in, they disrupted the disrupted the industry, they’ve offered something completely new and different. And they failed to capture the momentum because their product division was off. And Microsoft kind of did the right thing. They let Google innovate they picked up the right pieces, what what worked, what didn’t work, and kaboom. Now you’re, you’re rolling

John Janego 1:11:38
well, and it shows the advantage of having a already having the large install base. I mean, the key Oh def feature of elbow 365 is how easy it is to switch from an existing implementation that’s not using 365 to two using it, you know, it’s a pretty easy migration, you know that versus ripping everything out and turning it into a new A new stack, you know, it’s a, it shows how if you’ve got a core business, it’s easier to noodle and innovate on top of it then be the disrupter, you know?

Vlad G 1:12:12
Yep. Yep. I completely agree. So, any other challenges, except, except for? Well, you’ve mentioned any other challenges with your beta customers, beta programs. Any other lessons learned? Yeah, you can share.

John Janego 1:12:26
Um, I think just, I mean, I think the the core thing that I’ve learned through the course of doing it and I’ve been doing them for several years now, you know, probably one or two a year regularly has has been that you got to decide how long you’re going to let something run and have a launch plan. Even if that launch plan is just like, simple of it’s like, you know, this is gonna run for three months and at the end of three months is GA, you know, where I’ve personally gotten tripped up. And where I’ve seen other, you know, other products gotten tripped up, you know, like, you know, whether it’s other products that we’ve experimented with in our company or other companies as has been just a poorly defined exit criteria, you know, and you got to stick to your guns on it. And I, you know, I think, you know, going back to your, we’re going back to what I was saying earlier about thinking about the relationship with the engineering team and, and, you know, doing doing good by them. I think, I think that, you know, for the, um, you know, for the good of that relationship, you need to, you know, have a plan to release it, you know, or have a clear plan while you’re in or have a clear explanation while you’re not going to release it and go in with the expectation that that might be an experiment. But you know, if you go down the path of saying, you know, here’s a new feature we’re building, and we’re going to roll it out for a quarter, and then it will be released for everybody at the end of that quarter. You know, you should execute on that, if you say, you’re gonna execute on that, and then you don’t, you know, that’ll lose confidence and, you know, that will, that will lose confidence of many of your colleagues potentially, you know, as well as your customers just because, you know, they, they, they may not know if they can trust that it’s going to be released. You know, I, I had a, you know, in the most recent Early Access Program, I was just running, you know, one of the customers I was speaking with, and it was, it was really, it was a really eye opening thing that he said to me, which was, you know, he was like, well, I you know, I didn’t want to start using it and get and like it if if I wasn’t sure that you guys were going to release it or not. And, you know, like, I was like, Yeah, man, of course, we’re releasing it. I got like, 12 month backlog plan for this thing. And he’s like, oh, That’s all I needed to hear. And then like, you know, became one of our biggest users of it, you know, but like, you know, in his mind, he wasn’t really sure. I mean, I didn’t, you know, so having just being clear with your users and being clear with your colleagues that that’s something, you know, that, that that’s what that’s what the plan is, and do your best to execute on that plan is pretty important.

Vlad G 1:15:23
Yeah, makes sense. I mean, I’ve lived through a couple of products that had that notion of promising and not releasing. Yeah. And I’ve lived through a product that was kind of the opposite. We didn’t say anything. And then, oh, here’s something Oh, here’s something else. Here’s something else. I was like, Oh, my God, how you guys doing? So yeah, I know exactly what what you mean, this that’s, that’s useful, interesting and useful. Yeah. So let me ask you this, because we are coming up on time, pretty soon. And this kind of ties in what with what you’ve been saying? How do you measure whether your initiative, your capability feature, or the whole product is successful. How do you know? What is your measure of success?

John Janego 1:16:11
So this is a actually a question that when I was interviewing for the job as a product manager with Zero Product Management experiment, I was asked in the interview, and by by the VP of the product team, and, you know, like, I gave sort of my naive answer of, you know, like, are users happy with you or, you know, kind of kind of thing and, you know, she was like, Nah, it’s, you know, the, the ultimate arbiter of success is, is how’s the revenue looking from? You know, and, you know, I think, you know, that’s, that’s true. In many cases, it’s not true in 100% of the cases. It really depends on the scope of the product that you’re in. Responsible for, but I would say it’s a pretty good, it’s a pretty good it’s a pretty good proxy of a lot of things I’ll say. And I would say, you know, to the people I know who are in product management roles that may not have, you know, direct bookings coming into their product or, or, or whatever, you know, try and understand the impact that what you’re working on has to those bookings. And if you can see that those bookings moving in the right direction, you know, you know, trying to try and determine if you’re contributing to that, you know, in whatever is the appropriate metric for that particular product. I mean, I have perhaps the quote unquote easy perspective on this, of that, you know, my product is sold by licenses. So I have a pretty easy metric of success. You know, if we sell more licenses this year than last year. It’s doing okay, you know. But then, you know, you can look at it, you know, at the other at the sort of the smaller scale level of, you know, you know, we have a lot of different individual capabilities within that product. And I would say, you know, when we’ve released support for certain features within that To try and define, you know, where you’d like to see the adoption of it within a couple of quarters, or, you know, by the end of the next fiscal year, or whatever the timeframe that you decide, is and, you know, do what you can to, you know, to try and achieve that. But, I’d say unless you’re working in a nonprofit or something, you know, actually no revenue.

Vlad G 1:18:50
I see where you’re going with this. Just just again, I think I just made that example, last episode. I don’t remember the product. So we had a core product offering and I was developing a bunch of products around it kind of complimentary slash, doing something, something in parallel to the core product. And one of the things that we were thinking would be a commercial product was some some sort of an integration. I’m trying not to go too deep into details, but it’s a retail point of sale. So it needs it needs inventory to function. And we’ve built an integration to pull in that inventory without manual input. Pretty obvious solution, pretty obvious everything yet, no one in the market has done that before. So we were the first and we give that away. We didn’t charge a single cent for it, because it did not. It did not make any sense to charge for it. Although we were a for profit company. It didn’t make any sense to charge for it. But it gave us a huge competitive advantage. Yeah. And it resulted in a huge impact. retention. Yeah. So I always say, and I was actually if I put since they’re viewing you, I would actually like your first answer better. I understand you’re unable to expand on it and and actually say things like, hey, it’s a good proxy measure for ROI or customer retention or whatnot. But ultimately, you’re right. If your customers are happy, then they keep paying you people as your product keep paying you for the product. And everything goes well, if they’re not happy, and you’re not the only game in town, which is what usually happens then they take their business every somewhere else and your ROI doesn’t look that pretty anymore.

John Janego 1:20:42
Yeah, and I mean, I think it’s uh yeah, I mean, we’re in this we’re actually just have released literally last week, which is why I have time for a podcast now. To do a to do a release a new product, which we’re not trying For it’s a, it’s included with our existing licenses. So our measure of success for that is going to be user adoption. Right? You know, but that’s, again, it’s kind of like the feature that you just described, where it’s a, it’s a delight or you know, and it will help, you know, retain customers, it could be a differentiator when looking at competition and, you know, ultimately will still have an, a maybe more difficult to directly measure but still a hopefully attributable effect on the on the, you know, the success of the product as a whole financially and otherwise, you know, you know, and you know, I think it’s you know, and I you know, I love company, culture and, you know, fun organizations as much as everybody else and you know this alright, that sounds like a stuffed shirt thing to say which I’m totally not but you know, it’s like i think it’s it’s, it’s always important to keep in mind And I think this is something that became even more clear to me just through the, the learning process of being involved in acquisitions is like, you got to keep a, you got to keep your numbers in mind, you know, because, you know, whether you’re, whether you’re thinking about it actively or not, someone’s thinking about your numbers. So it’s, it’s best to not be surprised about it, you know, whether you’re a public company, and it’s your shareholders or whether you’re a early stage startup, and it’s, you know, you’re one investor, you know, somebody’s thinking about, about how your performance is because, you know, part of part of working in in in high tech or a part of working in a capitalistic global economy is you know, the the desire to have a return on investment and that’s, that’s why we have a pretty privileged position of getting, you know, a pretty high paid job working in a pretty interesting industry. You know, we’re all working from home still working in the tech business. And, you know, that doesn’t come by default. You know, it comes by, you know, being successful financially as in this part. Yeah. Working on

Vlad G 1:23:18
Ah, that part I can totally agree with. So as we coming up with the almost at the end of the show, I’m trying to wrap up. Yeah. Other any questions for me? And that’s, that’s my regular thing. I always ask this question at the end. If you have any questions for me. Hopefully, I will be able to answer it in a couple of minutes. So no questions about how to solve the world hunger or something, something. Let’s do something simple about product management.

John Janego 1:23:47
Yeah. Well, I mean, I think just from what I’ve read, and you know, I’ve shared my point of view on this and you know, a lot of questions I see people asking on the blog. Through the twittersphere, whatever you want to call it, as you know, how do people get into product management? And you know, which I think is an interesting question to ask, but I think we know maybe a question that might be more interesting to interesting to ask would be like, what do you think people should expect to get out of going into product management?

Vlad G 1:24:22
Oh, that’s a good one. Because, first of all, no one ever asked me these before. And the second I didn’t think of that. Yeah, you know, how come I didn’t think of that? I would imagine to I mean, I can only speak for myself, I would imagine there would be some people that would agree with this. I would, but why personal like is solving problems. And back in the day, when I had my own business, I never said, Oh, we are in web design, or we were in web development. I was always saying something very weird. Like, oh, we’re solving business problems with technology. Yeah. And I think that’s that’s why I ended Doing being the product manager. Yeah. So for me, it’s this this challenge of finding another problem to solve and figuring out how to solve it. And then actually carrying that solution from an idea where I used to be stuck at to the actual solution to the execution of that solution to taking something to the market so that people can use it. Like in one of my previous jobs, I built the product. It was actually a prototype, I built the prototype, together with another developer, we kind of put it together using rapid application development, technology, not even, you know, true coding, to speak of, we kind of put it all together, we’ll show to a director of the Department that that thing was supposed to be working at. And she liked it. She was like, Yeah, okay. And two weeks later, we found out that the big since it was a prototype wasn’t an alpha or beta. There’s no security just you know, a demo of functions that was there. So she quietly introduced it to the whole department, but a couple of hundred people, and about 50 of them started using it. So I login into my own system, and I found 50 users actively throwing production data into them. And it was well first of all, it was mind boggling. Second, I got a got a lot of flack for it. Hmm. At the end of the day, that was like, I came home when I learned that I understood that you know, everything everybody’s gonna start demanding my head because it was highly regulated environment. But I came home and I for some reason I felt better than ever before it was felt very proud of myself. And the reason why I felt very proud of myself was because I found I found this solution so good. Well, not me personally, we as a team, but as a product manager found the solution to a problem. The solution was so good that people were okay with using Barely functioning prototype, right? Right, instead of whatever the hell they used before. So it was it was that that feeling of, you know, fulfillment, that feeling of happiness is what I keep feeling as I work on other products and then finding these other solutions. That’s that’s kind of a, that’s kind of a thing for me. So that’s why I keep doing product management because I’ve switched a few careers in my life. Yeah, I love this one, probably more than anything I’ve done before.

Unknown Speaker 1:27:29
Yeah, I’m right. Where with you and me, I think I found the most enthusiast enthusiastic PMs to be those who came to it after having been in other roles because you, you’ve seen multiple sides of the fence, you know? versus like, you know, if you’re 23 or you know, you don’t, you may not have a lot of perspective on on what I think This is something that you really, you know, or I hate this word and are passionate about or, or you know, that you’re, you know that that is going to be satisfying for you personally, you know, like, you know, it’s an interesting job, and a lot of people want to do it. And I think it’s a great opportunity to learn about business and technology at the same time. But that may not be what you want to do with your life. And, you know, when you when you start a job when you’re 35 or 40 you know, it’s, you know, like, I started when I was, yeah, 35 as a product manager, and I’m like, Yes, yeah, I I’m glad that I’m doing this now versus when I was when I was younger, you know, because I have a I feel like I’ve got a little bit I, you know, you can inherit the, the perspective of having done a lot of different other things and and see the value of having, being able to take things across the finish line and satisfy your users and stuff.

Vlad G 1:29:00
Yeah, that’s great. I mean, I completely agree. And I keep seeing someone this on social media, mainly on Quora, I do a lot of Quora whoring and one of the things I see there is like, well, I’m going to get an MBA and become a product manager. Is that a good enough path? Or should they just go straight to product management without the MBA? I’m looking at these guys. And I understand they’re college kids. They’re trying to map out the path for their career, but product management is not what you want. Do something else. I don’t know. Open your own shoe store. Or go hike a mountain.

John Janego 1:29:35
And I would say to anybody who’s listening, and I’m tempted to respond to the people who asked these questions on on Reddit or core or whatever, you know, I’ll say like, as the person on the other side of the table who’d be interviewing you for that role? You know, I’m more interested in like, you know, the, the perspective you’re bringing, not necessarily if you’ve got an MBA or Yeah, or whatever it’s like, it’s like, how is your perspective going to help the success of our business? And and are you going to be able to empathize with our users and help them, you know, help them succeed by building something that they that they need, and therefore helping us to succeed? You know, you’re Yeah, you know, MBAs are great. And one of my, you know, best mentors in my career, you know, came to pm after having been gone through his MBA program, but you know, like, it’s a it’s just a tool in the toolbox. You know, it helps you learn that business aspect that, but it’s not it’s not the end all be all. That’s for sure. Yeah, I completely agree.

Vlad G 1:30:42
All right. We’re, we’re at the end. Thank you so much for being on this episode. I really appreciate it.

John Janego 1:30:49
Yeah. Thank you. It’s been a very good conversation. I appreciate you having me.

Vlad G 1:30:54
Absolutely. It’s been a pleasure. Thank you, john. And till the next time You’ve been listening to the real world project management and I’ll be your host Vlad Grubman. Until next time

Vlad Grubman – Product Leadership

All posts by Vlad Grubman

Real World Product Management – Episode 17

Real World Product Management – Episode 16

Real World Product Management – Episode 15

Real World Product Management – Episode 14

Real World Product Management – Episode 13

Real World Product Management – Episode 12

Real World Product Management – Episode 11

Real World Product Management – Episode 10

Real World Product Management – Episode 09

Real World Product Management – Episode 08

Product Manager / Project Manager