In this episode I am talking to Ksenia, a data analytics and BI specialist. We had a chance to discuss a data-driven approach to building products, the impact of AI in the near future, and what product management would look like when AI comes a-calling.
Transcript (courtesy of Otter.AI)
Please note that the transcription below was generated automatically and may contain misspellings and errors. If you want to help with cleaning the transcript – please get in touch!
Vlad G 0:07
This is real world product management.
Alright, Hello, everybody. This is another episode of the real world product management and I have senior Kalashnikov on the line today. I think Irina will join later if she can. She has other arrangements at the moment. So Excuse me, can you please introduce yourself and tell us a bit more about who you are? What are you working on? What is your role?
Kseniya K 0:38
Okay, sure. I’m I’m not sure whether I’m supposed to say hi, everybody, but Hi, everybody. Currently, I’m working in EPM as senior business intelligence analyst, that actually implies that I’m working with both business departments and the architects who are building the solution for the enterprise. So it’s More on the edge of business and technical perspectives. And that would be nice to share some experience in how that is actually working for businesses and technicians and people who are interested in both of those.
Vlad G 1:16
And that’s why we have you off today. Yes, yeah. So So I love the words business intelligence. I actually I like intelligence as a whole and business intelligence specifically. So talk to me about the data. Why is it important to have business intelligence? Why is it important to have data collected and processed? And we know it’s kind of expensive and gets expensive over time. So why should businesses invest in that?
Kseniya K 1:49
First of all, I feel the urge to mention that business intelligence is not a tool, and it’s not a thing. It’s more of a set of approaches technology. GE architectures, whatever it takes just to get the data. So when we say that we need business intelligence, that’s just like the very, very beginning of the very long road. That means that we are just getting started. And it’s I’m saying that not to just scare away, it’s just to say that there are a lot of ways how to do that. So basically, when companies are introducing business intelligence, they think that it is something extremely huge, very complicated, very expensive, and probably we don’t need that. Why overcomplicate stuff, what the heck. But the key thing is that business intelligence actually gives you more insights out of the data, which you either already have or just planning to ever gather. You can pick just a few very starting small approaches and small techniques, and they will already give you much more insights and ideas about what to do and what to start with. So I’d say that business intelligence is great to start with. Because it helps you both work with the data that you have. gather more data even out of the sorry places you’ve never thought you’re going to get data. And also, because it can help you build out the bright future for both your company and the environment that you’re working in, so kind of works for it. After all, even being these complicated as it sounds.
Vlad G 3:30
I like I like the bright future reference things.
Kseniya K 3:34
We like future references.
Vlad G 3:39
Oh, yes, definitely. Yes, indeed. So question, um, we kinda know and I’m with you on this one, as much as I like to disagree with my guests. One of the things that I noticed since we’re talking about a company’s building, bright future and all that will usually happens in at companies and I’ve been I don’t want to say I’ve been a victim of that, but I happen. Companies have want to use advanced analytics, bi, maybe even machine learning artificial intelligence on the data that they have or they are able to collect. they assess the costs or the assess the expenditures, the resources that they need to process the data. They think it’s expensive, they end it is just hiring data engineer, or data scientist and somebody who can collect process and understand the data. They look at all this and they say, you know what, that still expensive I we can’t afford this. Now we’ll probably do this, you know, q3 next year, which is basically never and they never do it and they obviously they’re leaving money on the table. So what is your given your background? What would be your kind of idea or recommendation approach to how should companies really approach this? Since since we both understand that we both want to build a bright future for all of us? How would you approach that problem?
Kseniya K 5:13
Well, referring to the thing that you said in the very, very beginning, I’m not sure whether it’s the right expression to us right here. But it’s like, a lot of the company’s approach business intelligence and advanced analytics, as you’ve mentioned, them is like, you know, they’re trying to cuca hair before they actually catch him. They just entered the market of business intelligence, they see how cool it is, and what a great experience other companies had with working with advanced analytics, and they’re like, Yay, let’s do that. But the key thing is that advanced analytics is gonna work out the way it’s supposed to only when you have like the data infrastructure already prepared, and you cannot jump into that just clicking your fingers, you should prepare First of all, and in a lot of cases This preparation takes way more effort, much more money than expected. And it’s way more complicated. But the key thing if you do that, right, advanced analytics is just going to smoothly slide in. So I’d say that to start implementing that in your company and just to go data driven, if you are planning to, at first you have to fix up the mess that you already have right now in your company and fix that at first before you start talking about business intelligence, advanced analytics, machine learning and all this kind of stuff.
Vlad G 6:38
I’ve I’ve had a few projects, kind of even before they became products, I’ve had a few of those kind of shut down because of because of this always expensive rationale. My personal take on this is yes, it’s expensive. Yes, it’s complicated. Yes, you need to clean up the mess but you have to start somewhere you have to stop At some point in in present and keep pushing towards the future and kind of way I, the way I see this was that, hey, we don’t have to buy the whole machine learning thing from day one. We don’t have to invest in the full blown artificial intelligence approach from day one. What we need to do is we need to understand what we have today and see how we can use it. We need to move on slowly to regular just look at the statistics, look at the regular analytics, move on to advanced analytics, move on to the AI and then eventually we’ll get to the machine learning because by that time, we’ll be prepared we’ll be ready to have the data will understand the data gaps. So, you know, data that we’re not collecting today, but we will be collecting based on what we see instead of just stats and analytics down the road. So that was kind of kind of the approach I was thinking. Fortunately didn’t take so but that that did workout in less less place to try it. I think whether sent that data driven approach is important. So is it just a buzzword a trend? Or is it the real deal? Talk to me about that?
Kseniya K 8:12
Well, I just wanted to make small comments regarding the project that you just mentioned when it did not kick up. As much as I hate to admit that and I hate to say that even out loud, I hope people aren’t gonna hate me for that. But um, a lot of the companies are working pretty much fine using just XML files and like simple reporting tools, not something very complicated, even without, you know, and they are able to perform the analysis they need and to gather all the information that can help them out. So the key thing is that Yeah, you’re absolutely right that we have to start with a small steps and small steps can be as simple as it is just make clear business rules in your Excel spreadsheets. Make obvious restrictions onto the reporting dashboards that you’re using. And that would be your first step to seeing the gaps then to introduce a new approaches then to apply more complicated more advanced analysis onto that and then moving further to where you would want to so totally agreed on your point to that. Simple steps are just okay here.
Vlad G 9:23
Yeah, just just one note, one note in what you’re saying I it’s not that I don’t like Excel love Excel is just that after in my career after seeing two gigabyte files that run whole hedge funds, I would say find something.
Kseniya K 9:40
Yeah, I hear you.
Vlad G 9:43
Don’t abuse the thing.
Kseniya K 9:46
And one of the most recent projects I have been involved in there was a very complicated constructional work done when the company has been trying different combinations of various equipment and There were more than 100 combinations within one stack. And it all was handled in the XML file. So when I saw that file, it was like really? Do you do you really use this? Is that okay for you? So I’m not judging them is just the, you know, the depth perception. So, maybe there’s something better for you to do that. Kind of maybe. Yep.
Vlad G 10:24
Yeah, yep. I mean, I’ve seen I’ve seen people using Microsoft Access. And that’s kind of the whole the whole story about the road it people who can get things done by their own IT departments so they start learning Microsoft will they already have Microsoft Office they already know Excel. They maybe learn Microsoft Access because it’s slightly better in its actual database, even though it’s you know, reduced significantly from sequel but still basic database and that’s how you get the whole departments riding on rope databases because Yeah, things things are not things are not there. So, back to the original question. So data driven approach is a thing. We want to use it as much as we can, regardless of whether it’s a simple Excel file. If it’s something more complicated or if it’s an advanced analytics suite, we’re still we still want to use it. For what, what do we what are we going to do with the data? How are we going to how we’re going to use that data? I mean, do you have any examples of using the collected data for real world usage? I don’t know. Building Products, changing products, anything of that sort?
Kseniya K 11:42
Yeah, sure. Well, first of all, I would say that your question is still connected to the story about the raw data and people trying to use axes themselves. That the key thing is that in the majority of cases, people are doing that out of despair because they as you said, they cannot reach out to IT specialists and They have to solve the problem themselves. So they just invent new ways. But the key thing is the business intelligence kind of gives you the information, it makes it more democratic sort of thing for you. So you don’t have to invent your new wheel, you just use the already simplified and comprehensive information for you. Once were going data driven, and we are willing to do that to the BI, I would say that, in a lot of ways, it would make a lot of sense to just have the hypothesis and you know, like the first statement we’re starting with, but in a lot of cases, people do not have that on the project. Just to give an example, prior to working in EPM, I’ve been involved in the banking products, and we’ve been implementing a new functionality, which actually was not articulated properly, it was just aimed to do the job better. So just do that or give us Your best shot sort of thing is like a classic example of not articulated. task which has to be solved. And there’s no particular request, we have to deal with that and come up with some adjustments based on the data data is like the first thing we can use to actually do that first step. So we were working with a user friendly, user user friendly application, as we thought, and we were testing it trying to make it better and more, more mobile, more easy to understand for whoever is going to use that. And also there was the increasing number of information people were uploading into their application, so we had to handle that as well. So I’d say that given that we were not given a specific goal or specific restrictions on how to use the data we were given. We just started collecting Seeing the data which was most to the eye. So the most logical thing would be just together everything, you know, like get all the data you can get. But we were more aimed into getting the information which is more obvious. So let’s say for us, it was the quantity of transactions people were doing in various periods of the month. So obviously, when you have to pay bills for your apartment, that’s like one date and the month, when you have salaries, that’s another important date in the month and so on and so forth. So we were gathering a now analytical information data from this specific dates. We’re trying to get, how many people are online, what steps they’re taking to get to this information, what kind of transactions they’re doing? Are they getting any bills? Do they print out some information? Do they probably decent many populations inside of their accounts, etc, etc. So As a first step, it would be good to actually have the goal and the purpose articulated from the business department. But as long as we did not have any, we just went with the most obvious thing that we thought was is just to get the information which was mostly used and on the surface. So I would say that if you are like, in the middle of a huge ocean and you don’t know where to swim in the land of data, this is the third thing that you can do. Just grab the information which is on the surface and makes more sense to you than anything else. And that can be a good start.
Vlad G 15:38
Okay, so please go on. So what, what happens then next, you take the data, you apply it, you make changes. How do you assess whether you’re successful or not? I’d say that we’re actually missing one more step. before we’re assessing whether we succeeded or failed. We So how we’re applying this information, so it’s not enough to just gather information, you know, like, okay, hundred of people were using the application at BBM yesterday, like perfect. So what like, how am I going to use this kind of information? What does it tell us? Exactly?
Kseniya K 16:15
Yeah, like, what is it about what can I do to the So basically, once we gathered this most obvious as we are going to call it for now information, we have to do something to it. And current situation in the data driven approaches and in the BI trends, particularly, a lot of analytical thinking, and understanding is still done by the humans. We’re going to get to that, but let’s assume that it’s still humans job. And using the information that we’ve already gathered, we have to at first, understand where it can be used. So let’s say I have a number of people who were using my application yesterday. How can I use it like to wear clothes Basically, I cannot use that to the currency rates because it makes no sense. But I can use that to see how stable my application was. So what was what was going to happen that if the 200 and first person is going to enter the application, and everything is going to crash, so this is my point where I can use this number. So once we gathered information we have to think through and in a lot of ways from the business perspective, the part where this information can be used, and in the future applied to make some conclusions and build up some models even on the hypothesis level. So let’s call it this way. So as you said that we need to implement that like optimize it somehow to later figure out whether it’s a success or failure. So let’s say that we figured out that okay, that 200 and first person to answer the application is going to crash it Perfect. This is a valid information, we can use it. So let’s use it, how we can use that. So according to this, like the most obvious thing we’re going to do is just work with our technical support part and make the application more stable. So we can handle this, this number of people using it, maybe we can increase it even more if we know that the functionality is going to be expanding or becoming more popular and the sort of thing. Also, we can come up with alternative ways how people can enter the application. So let’s say a lot of people are using the four digit numbers, combinations to enter the application like kind of code lock. So what if we’re going to introduce the fingerprint scan, that’s going to increase the speed of login in and that would just reload the system a little bit because this an alternative way how to do that. So once we’ve found out the ways we We can use and apply information we gathered, we can actually come up with the solutions, how this information can help us. And what can be increased, because we’re basically working with how to make things better.After we’ve implemented this kind of things, simple, simple, old fashioned testing, so you always have to test out on people. It can be you yourself, like on the first party, the development team, they might be a little bit subjective. But anyway, that’s good to test with them in the first place. Then the beta testing, the alpha testing, all this kind of stuff that we know. So this is the optimization and implementation part that you’ve been mentioning. And only after that, we can actually figure out that Okay, so the, the application did not crash. And people are satisfied with the alternative way of logging into application because well, aside from state Typical ways, there’s always the satisfaction factor, which we cannot ignore a no matter what our business is connected to. So if we say that Okay, so this kind of information helped us to improve our application, people are happy statistics are happy system is okay, then we can consider that to be a success. He, for example, people rejected the fingerprint scan, they say like, it’s too complicated, like my fingers are frozen, it’s not working, it’s just given to me bugs led me into the app. So it can be considered a failure, because it did not give us the optimization that we’ve been intending. And it actually gave us the decline in the satisfaction rates, so people got frustrated with us, so it can be considered a failure. So I’d say that before we actually label it onto white or black, it would be good to also have those two additional steps prior to any like finalization of the data usage at this point. Okay, that
Vlad G 21:01
that is that is really interesting. Thank you so much. Yeah, well, I get that. You need to you need to process the data. And obviously, you can’t just throw data at the wall and see what sticks. I think what you’ve described is improving usability and performance optimization using using the data. I was wondering if there’s anything else any other issues that you can stumble, when you are looking at the data, like, for example, in your in, in the story that you told about optimizing the performance and potential use improving user experience, instead of needing to type in the password on a small phone or tablet keyboard, maybe it’s just easier to use a fingerprint, how accurate that data was, or rather, let me rephrase that. How sure are you that The data is actually telling you the truth. And what do you usually do? How do you make sure it tells you the truth? And what do you do if it doesn’t? And I think you mentioned, you mentioned something, your story about data being collected by human by humans. One of the things that I know from from my past experience is, even if the data is not collected by humans, it’s still you know, in in real world, it’s still being generated one way or another by humans. So you can’t always rely on that data being true. Because there’s always you know, room for room for error on the data side. So talk me through how you guys deal with this.
Kseniya K 22:45
Well, I would agree on to that part, because human factor is always there so people can be tired while aggregating information they can be distracted so they can make him stay or use the outdated position just because they forgot to To update them or something. So, it’s always understandable why data can be false because a was processed by the human after it was aggregated by the machine. So it’s more than obvious. I do agree with that. I would probably say the very obvious thing and not, you know, like, not as sophisticated to you, as you were probably expected me to say. But um, I would say that the first thing you need to do to figure out whether information is lying to you is just to use common sense, you know, because when you see the information, which obviously, like confuses you or makes you question stuff additionally and more than it’s supposed to, there might be something wrong with that. Good example of that also from my banking experience, so we were preparing the it’s like it a little bit different, but still from there. So we were preparing reports to present to high authority Once a month, and the person who was in charge of generating those reports was using various coefficients of currency rates in Belarus, where I am from and where the bank was working the currency rates has two approaches of how how to calculate them, like statistically, a algebraic clique, let’s call it this way. So various ways how to calculate them. They are approximately the same, but still different, basically, because the approach is different. So the key thing is that during preparing this report, the responsible person used the coefficient which was calculated using the different approach than it was used in all the previous month. Obviously, that created a difference like the huge delta between the past months values in the current month values so the regulator actually called and said, Hey guys, what Are you doing there? Why the Where’s the money? Why everything changes so much? So basically the same that could have been done in here is one more additional checkpoint link, which would figure out you know, like this edge cases, the Delta like okay, that differs way too much than he was in one month before two months before it cetera, or where did you take the coefficient from please show me that and Oh, the value does not match. So just setting some common business rules and restrictions for validation would help you to cut to catch up the mistakes that were done by humans, would you do not come into the common sense? So like the most obvious things, you can do that manually with your own eyes, the ones which are not as obvious but still questionable can be caught in business. drills and how to Elsa secret whether data is lying to you. Um, I’d say that he would always say you probably will always have the way to compare that. So if we work in with statistics with analysis and different levels, like if it’s just the reporting or the dashboarding, or the cloud analysis already, there’s always the way to figure out the trend. And you would see that some things are changing, not the way they’re supposed to not according to the modelling that you’ve been building previously. So you can just find them yourself through common sense. So I say that to find out the obviously false data use common sense, aka human brain. business rules, aka validation and restrictions, and three trends and proceeding business models. So like this three steps can actually give you the insights whether the data is wrong or not really.
Vlad G 27:05
Wow, that that is that is interesting. I had and I’ve talked about this product in one of my first episodes, I’ve had a product that I was building that relied on data collected from points of sale. So they itself was not generated manually, but the process was, and the fun part. Again, maybe I’m I misspell or miss pronounce the steps that you’ve mentioned. The funny part was that business rules would be okay. And the salesperson in the store that was using the boiler sale was actually following the business rules. But they would find ways to game the system and we’ve expected about You know certain certain percentage of transactions running through the system and in depending on the store, it could be from one or 2% to as much as 15 20% of transactions to be fraudulent, meaning they would adhere to business processes, they would adhere to business rules, otherwise the system will not process the transaction at all. But we will do something else about that transaction that is fraudulent that is wrong. And if you just look at the transaction it like you said, the human use common sense and human eye, it would never you would never see that it’s a it’s a wrong or fraudulent transaction. And you we actually developed a number of separate rules, if you will, or additional algorithms that would look through the data, match it with the business logic and then match it against a number of additional rules. To see if they just had transaction was fraudulent to give you a a removed example not not from that industry but something that everybody understands. So this is one of the examples that was given to me and what is what is a fraud at the point of sale system. For example, you want to sell someone a bottle of champagne in a bar, right and this is ideal bar right? You just you just work there so you don’t really care. And so use your friend you can charge them in the club and the bar is really expensive bottle that usually costs $20 make us 300 so he’s your friend, you don’t really want to charge him $300 but you want to give them that bottle of champagne at a lower cost. So what you would do is you would take a chocolate chocolate or you know, something less expensive. you swipe it on the barcode scanner and you put a bottle and thought Have it so you don’t use the camera that tracks you. And there are cameras that track the cash registers, it will see that you’re scanning a bottle of champagne. But the barcode that is being scanned would actually be from a chocolate so you pay, you know, a couple of dollars for a bottle champagne that you’re supposed to be paying $300 this way, transaction is correct. You know, business business rules are are observed. You just scan to a piece of chocolate and you know, you’re paying whatever 510 dollars or should I get worried, you know, this methodology way too good. Yeah, because, because I as I said I was building product, it was actually analyzing the data and highlighting the fraudulent cases. So it was one of the uses for the product was to catch the fraud. We’ve one of the pitches we did was that if you I’m just saying To remember the exact phrase. So we’ll we’ll show you how much money we’re leaving on the table by analyzing just the fraud. And
it was it was something to the extent that please, we’re not going to charge you any fixed fee where they charge you a percentage of the recovered money based on how much further we recovered based on how much in efficiencies were recovered. And people usually in the couple of months after, you know, two or three months then once they’ve seen that the amounts that we’ve recovered and potential they ask we’re asking to switch to a flat fee no matter how big it is, because you know, any reasonable flat fee would be still will still be less than, you know, the fraud and the amount of money that that we were recovering. It was really really huge. The the ways people were gaming the system, the ways and this was the telecom industry. So think selling iPhones at $1,000. Price mark. So if you can, you know, game the system and sell two or three iPhones at thousand dollars a year commitment and gain commissions from it, it’s a pretty significant amount of money. It’s pretty big deal.
Kseniya K 32:12
Yeah. So I would say that this is the beauty of this kind of approach that no matter how intelligent your system is, and how perfectly verify your business rules are, it still can give you the wrong data. And no matter how qualified and experienced your people are, they still can make a mistake. So always combine those two. And they can give you at least a little bit better result that each by themselves. So that pretty much is what would work here that business rules in the camera and people’s attention probably could have given you the way how to catch the person who’s actually scanning the chocolate bar, but it did not. So there’s a lot of gaps in their overlay.
Vlad G 32:57
Yep, yeah. I mean it It was funny that even you know, the store owner would come in and say, or a bar owner or business owner would say, I know they’re stealing from me, I just can’t understand how they’re doing it. And these are the people that have been that have been in the industry for many, many years. So if anybody knows how to steal from their own cash register would be them. But yeah, that was, it was interesting how data can solve problems cheaper, as a matter of fact, then installing very expensive, very expensive security systems, closed circuit cameras and all that, like I really don’t need the camera, I really don’t need anything. I just need to see the data and I can tell you the outliers and I can tell you that this transaction has something something’s off or you know, the number of transactions does not match the number of items in your inventory, which usually different things in many systems for whatever reason. All right, this is this is interesting this is really great, thank you. So what, how else? In the real world again? How else can we use the data? Assuming we can figure out we can clean it up and figure out that the data is correct. What else can we do? Can we use the data for what else can we do with the data? Can we use it to plan the products? Can we, I don’t know, create new products based solely on data? I mean, I would love to have certain things right? Look at the data and say, Hey, you know what this feels like we need to build a product and that something similar did happen that exactly that. And that’s what I’m saying. I would love. I would love for this specific thing to happen.
Kseniya K 34:45
If I may, that reminds me on an off topic of how I met, your mother said come it was like we were told we should totally open a bar. Yeah, we should open a bar. So it’s here we have the data. We should totally start up a new product. We should do that.
Vlad G 35:01
That’s good. That’s good. That’s a very good reference. Thank you.
Kseniya K 35:07
So the key thing in here, probably the not obvious thing, like when we have the data, and we have the urge to make the earth a better place, you know, like to create a better economy and this kind of stuff. That only the obvious way we can go. But I’d like to give an insight on the nuts so much of an obvious way how we can actually use the data when we have it. It’s incorrect, and we want to use it somehow. I would say that one of the ways we can use data is to start developing the data culture among employees. What do I mean by that? It’s not like data driven culture, as you’ve probably heard the term but like the data itself, culture is that the employees who would learn how to kind of be hasty with the data and treat either Right wait. So in I’m pretty sure in a lot of your projects you as well. So how people are just dumping the information onto some weird spreadsheets, do not save them properly, do not update them once it should be updated, turned into garbage data pretty much. So the way that we actually were able to figure out the data out of this dump and see the value in it can be one of the stimulating ways for the employees to actually start treating data right. So let’s say you come to your employees and say that Okay, guys, so we had to analyze a lot of information through like these means and these means and whatsoever. And we were able to figure out that let’s say the, the number of purchases and the quantity of the box in the storage is Ashley. God basically from this Publication not from the others. And in the future reporting, we’re going to use information used only from this application as our main source. So that would actually give people the understanding. Okay, so if the main source of information is now this application, there’s no need to support, let’s say, this other folder of data, because it’s going to be obsolete, not used, not even for historical purposes, is just going to be useless and overwhelming. So they would create a habit of following the data and the trend in the data in the organization and the company, which is actually give them the profit, they would give them. The initial, you know, like, discipline sort of thing. They would be able to figure out like, what data to store and which one is not. And another way is to like, when they’re going to be understanding like what data is used, that would make them more aware of probably the mistakes and not how to say that probably not accurate usage and storage of the data. So they would say like in the particular example that I’ve been given earlier, so the person was preparing the reports once a month. So it’s not as often and as urgent as you would think. But in this particular way, it is extremely important. So you have to be like super concentrated, super focused and very, very much into the stuff that you’re doing. So it will be probably easier and more understandable for the employees how to separate and delegate different tasks of a very complicated reporting or a dashboard or whatever to different employees. So the so that the cross check would give them the most correct information, and each of the employee would developed their own responsibility for the piece of information they’re preparing. So as one of the non obvious ways of using the information gathered is sort of giving your employees the inner data discipline and data culture in treating numbers and various information correctly. And the way it can bring value to the company through themselves already, not after some smart machine or some smart algorithm is going to transport a some way and give us information. So like you are responsible for the data you’re given. So be nice to it.
Vlad G 39:40
Funny, it’s nice to look at it like that. I like the term that you’ve used the data culture of more than once I’ve seen people not really eager to collect to help companies collect the data because there’s this stigma Hey, this data will be used against me and they’re not wrong. That’s let me just say that there’s definitely a way how the company can use this data to. I don’t want to say abused, but take unfair advantage of the workers. And one of the examples that come to mind and if you if you feel you can comment on that was the recent discovery, I think would be the proper term that the algorithm was optimizing the roots of Amazon delivery and Amazon warehouse workers. But what was missing from that algorithm was the account that these are not robots These are real people. They need to you know, sometimes have a sanitation break. Oh, that’s just it was and it was there was a lot of there was a lot of complaints. There was a lot of materials published online, all over the place. In regards to Amazon, not taking into account the actually not Amazon, but the algorithms that they were employing, not taking into advantage. The fact that these are real human beings So for example, if you’re supposed to work an eight hour shift, it would plan literally all eight hours down to minutes and seconds. Make sure you’re the most efficient there’s was no account for you know, bathroom breaks, no accounting for just stopping and catching a break, stopping and taking a breath. Same thing if you’re a delivery. You’re driving the mouse on track to deliver goes to individual customers or from warehouse to whoever ordered it. You they will optimize the route without accounting for traffic and it’s fine in rural areas. But as you go into larger cities like Los Angeles, San Francisco, New York, Boston, Chicago, the large lommers it becomes a really big problem. And Amazon did not account for that, or whoever built the algorithm did not account for that. And as much as we love, you know, being efficient as much as we love making sure that everything is working as smoothly as possible. There was no way for people to meet the quotas or there was no way for people in those large cities to meet the expectations set up by the algorithm. Yeah, right. So in your, in your term data culture, I love it. I am making a note I’m probably going to use it. It’s some some points in my work. Thank you for that. We’re gonna make sure that we trademark please. Absolutely. So what I wanted to say was the the culture is is it the way street Yes, we want to encourage workers employees to save Data collected data preserved the data in the right way. But we also want to make sure that it’s being used responsible. That’s where I was getting it. By the way, since you’re an expert, I have I have a rookie question. Question. Is there reasonable? Can I reasonably expect the algorithm to learn? And I’m deliberately using artificial intelligence, machine learning terminology, because I don’t want I don’t want to say, you know, what is their statistical approaches or analytical approach? I want to use as generic as you know, as Internet’s terms as possible. Is there a way for the algorithms or whoever’s working on them to learn from real humans so that they, their their inefficiencies are built in you can’t optimize to remove those inefficiencies. They’ll be there just because we’re, you know, living breathing organisms and not not machines? Is there a way to account for that? On the algorithm or on the data level? Not you know, when I’m planning things, I kind of feel that you’re expecting the yes or no question from me, then, no, actually, actually, no, please don’t please elaborate as much as possible given, you know, given what we know now.
Kseniya K 44:24
So, if I may, I’m like a fan of weird references. I’m going to give another one recently been reading The Hitchhiker’s Guide to the Galaxy by Douglas Adams, and you probably familiar with the concept of the deep sauce computer, like the smartest computer in the world ever created and all the universes so humans gave the computer the task to figure out the answer, the ultimate answer to the lies of the universe and everything, and the computer was versus in disinformation. Through all the statistics through all the assumptions, risks, complications has been processing for 7 million years. And then the answer was 42. So I would like answering your particular question with this reference, I would say that the machine is able to like if we’re using the buzzword is able to learn as much new answers as possible. And use as many details and assumptions that take risks into considerations to the most possible way to participate in preparing predictions or forecasts or some analysis, that I still believe that the decision maker is anyway the human. Aside from all the science fiction and all the fantasy world stories where the intelligence is too intelligent to overcome the human And I strongly believe that the final word is upon the human and as advanced as the technology in machine learning and artificial intelligence and all the processing of the advanced analysis, and deeper bi in specifics is getting with time, that is still going to be more like this invest in that. So what next? So the person is going to be the decision maker, I’d say that the machine is able to learn a lot and to act and answer a lot of very complicated questions. But it would not be completely correct to leave the machine decide. So the decision making is still left to the human brain and to the subject matter experts who are humans and people and who are actually like people with the business. insights, deeper understanding, empathy, if you may. So all of the emotions and emotional intelligence, which are machines that are lacking and hopefully will keep lacking.
Vlad G 47:13
Thank you for the reference, by the way, I absolutely love the books. And both, I think both movies Well, one of one is the BBC production one is the actual movie. So funny that you said that will leave decision making to humans because as of last year, if I’m not mistaken as of last year. And I’m just going to generalize it because I don’t remember the specifics. So don’t quote me specifically on that. There was an experiment where scientists built AI, predicting or analyzing the symptoms and predicting the diagnosis in patients. With a very specific niche of the medicine, I can’t remember for the life of me, and I don’t have time to look it up right now, the specific niche, but it was one of the general things like seasonal flu or, or allergies or something like that. So it’s a very common case. So there’s a lot of data. And they did the analysis they, they ran, they created the algorithm, they’ve built the AI around it. And To their surprise, I predicted the correct diagnosis. Better than the human doctors, it was not marginally better. It was actually significantly better. So this was not like a marginal error. It was, hey, if this was the real deal, we should use computers rather doctors because computers are distinctly better.
Kseniya K 48:56
Yeah, that kind of reminds me the way that kind of Same project project I heard about when they were given the LRS algorithms, a lot of X ray files to analyze and to figure out whether it’s a tumor, or some kind of oncology coming up in human. And that gave actually the exact same result. So the doctors were missing out information and making less correct diagnosis then rather than the machine dead. So that sounds kind of the same to the situation that you gave.
Vlad G 49:30
Yeah, that sounds similar indeed. So that’s my point. And and we’re, let’s be reasonable. We are very early in stages of building algorithms and building machine learning and artificial intelligence capabilities. Because we’re not as as humanity. We’re not cultured, we’re not collecting enough data, we only now started to realize all the benefits of collecting the data. So we’re probably going to build additional you know, we’re going to need additional several years to build up the enough data for some serious algorithm usage. And that sense, we probably going to see even more even larger gap between what the doctors can do. And since we’re using this, this case, be what the doctors can do and what the computers can do. And in that sense, it feels like at some point, we’re just going to give up it’s really easy way I see. It’s really easy to give up to say, Hey, we don’t need the doctor to make the diagnosis. We just need the doctor to validate the diagnosis. So it makes to basically confirm that the computer was correct. And that’s it. We’re basically as far as I like to call it another called shoulder reference I I for one, praise the coming of our future robot overlords.
Kseniya K 50:58
So
Vlad G 51:01
Possibly thinking I’m getting there I, for once I like to think that computers will help us will definitely lift some of the load off our shoulders in terms of processing, analyzing data and giving us some kind of a, some kind of an outcomes, some kind of food for thought. At the same time, I feel feels like it’s too easy to say, hey, computer knows better, let’s let it decide.
Unknown Speaker 51:30
Well, you know, that kind of already happened to us remember when the first computers were so advanced in all the calculations and how fast we were doing some mathematical actions. So at some point, humans just had to admit that okay, we cannot do math as quickly and as awesomely as computers do. Well, it’s just that okay. And do you see people actually started stopped making two plus two now people are still Using that they just admit that okay, that computers can be better in this case. So I would, I think that it can be the same way with a further development of intelligence, self computers. So isn’t the example as you said, okay, the computers are given the better and the more accurate diagnosis to the patient’s into complicated cases, okay? So it’s just the way we can learn from them. So we can just analyze the information that they given us and try to find out like, what did we miss out? How can we can become a more advanced specialist, an expert ourselves and just keep treating people and just becoming a better version of ourselves. It’s more of the human way how to deal with that. So the computers just can get more advanced in the actions that are performing. And the humans can just become better in all the other aspects which are close to four machines and four algorithms. So Do not give up on humankind so easily, we can still use computers and even if they’re smarter than us, we can still use them and grow more from them and get more information from them and become a better versions of ourselves that positive vibes. Okay, so your
Vlad G 53:17
your outlook is is strongly positive. optimistic we were going to get better at this,
Kseniya K 53:24
you know, as much as we love Terminator, honestly, like every part of it as much as I love it. I still think that humankind is having something that machines are never going to have. And it might be a little bit, you know, like philosophical question, and which is not a part of this particular discussion. But in the particular case that you given. I think that even though machines are getting smarter than humans in some of the ways, attention in some of the ways not in all the ways it’s not the end of humankind era, and it’s not the limit for humankind capabilities, okay.
Vlad G 54:05
That’s, that’s really optimistic. Thank you. I appreciate that. All right. So we’re getting closer to the time. And I have to ask those two questions that I promised you before we started the recording that I’m going to ask. So one is, how does it feel? Or what do you think? or What is your opinion on working from home? More or less permanently? And as a data expert, I want to throw in another curveball, asking, how do you think, since it provides more data to be collected, like how long you’ve been online? How long have you been coding how long you’ve been, you know, sitting behind the computer? Would that affect our understanding of workplace as a whole would that be would there be any kind of consequences from that on on the data so the data collection Analysis side.
Kseniya K 55:01
Okay, so answering the first part of the question about remote remote working for most of the time, I’d say that, thanks to again, the growing intelligence of computers and the way they’re proceeding the information that we’re given, like this specific case, like we’re exchanging our voices for the ocean, over across the ocean, and we’re still doing that and all the data is going clear. And okay, so I’d say that remote work in 21st century is becoming less of a problem in its core, then the human work itself. What do I mean by that is that we do have all the means to do that like intranet, notebooks, remote workplaces, etc, etc, etc. But it’s more up to humans to adjust themselves like their natures upon this particular environment. So for them, to be more data driven to be more you know, like Simple example more attached to the schedule. So let’s say this particular hours, you have to be online, with your team having meetings, no matter what happens. And if your cat wants to have some extra food, you just have to be online and get more into the technology. You’re doing more data that you’re getting like those texting codes, and different comments in the GitHub, like different pushes and rolls and stuff and stuff and stuff. It’s more of the human kind way to adjust to this way. From my perspective as one standalone person, I’d say that it’s a in the technical and in data driven perspective, it’s not complicated for me, because I’m used to working with remote teams, and just being very, you know, like responsible onto the way I’m doing, but from the human part of myself and from the emotional intelligence And feels kind of lonely because you’re facing the machines only not the real humans you do not hear them laugh You do not see the faces so it’s a little bit complicated into the emotional part.
Vlad G 57:12
And as the data specialist I can you please remind me what was the core of the question like what data how the data we eat okay. So it was a data we collected from the remote from people working remotely How would that affect us? In the long run it would we collect something useful we collect something dangerous to us, those types of things.
Kseniya K 57:37
Okay, again, tricky thing I’d say that if you’re using like this monitor things like lot of the programming stocks are using like they’re keeping the movement on your on your computer that you’re actually like moving and typing something in, that would make the relationship between employer and the employees more mechanic and more techniques. So it’s my ruin. In the relationship and like the core of the work, not to ruin but just like, bring some damage onto it in the long run, because both employers and I’m going to be trusting enough, like, what are you doing there, like why your mouse Mouse is not moving, I see that you’re not coding there. And people are not going to be trusting of that, okay, I wanted to go drink some water, but I have to be here, right into code, otherwise, otherwise, the data would be collected wrongly, and I wouldn’t be paid properly. So it might bring some distancing, uh, you know, like mistrust in between people in some ways. But among the positive consequences, I would say that it would just show in funny thing, unexpectedly more managerial questions and issues like you can get the data in numbers like analysis and statistics about how your team actually was performing, then you would see that okay, I’m actually having the specialist who is not Okay with the tasks I’m given, or I should delegate one task to two people instead of one, as I do have it now. So probably the data that we’re going to have from working remotely, might give us insights onto how to optimize our managerial part of the work. So that’s from my perspective. And I think Interesting, interesting, now that you mentioned that the manager role part and the mistrust.
Vlad G 59:29
As, as the person who used to be a developer in the previous lives, I actually think it was create way more mistrust than I mean, my my overall outlook is that is going to do more harm than good because and that’s the one of the things that I read online, is, there’s a couple of systems out there, that track freelancers work, and what happens is they take several pictures, randomly You know, in a specific, a specific unit of time, let’s say five minutes, 10 minutes, it takes several pictures and counts the number of keystrokes. Then take screenshots. So not only it takes the picture through the camera to see if you’re in front of a screen, also take screenshots of what you’re working on, looks at the applications that are open currently on the computer, and so on and so on. Wow. And violation alert. Well, apparently, you know, you have to consent to that before it starts. So you can if you can just start doing randomly, if you take on the project, then you consent to this information being being taken from you. And I think up work is one of them. There are a couple of others. And I know that because we’ve I’ve used the artwork in the previous we hired people off the artwork in the previous job that I had. Yeah, well, I work as well. Mm hmm. Yep. So let’s One of the things that can happen here is that, you know, it’s it’s almost like spying on the person. And even then you’re you have the best intentions, even if you’re like, Okay, I really like this project, I want to work on it. And I think, you know, judging by the result is fair, but if if this is what’s happening, and the system automatically says, Oh, you did, I didn’t get enough screenshots of you working in this 10 minute period. So you’re not going to be paid for that for that 10 minute period if you’re being paid by the hour. So what happens with people actually end up working way over to make up for those missed 10 minute periods when they were not accounted for. So instead of an eight hour work day, people ended up working than 12 hours to make up for those little bits and pieces that were missing. And, again, as a former developer is a person they used to do that. I think it’s a pretty dangerous path to go on. They’d rather keep track of a result rather than the process itself. And then I mean, ultimately, I, I keep saying this in multiple situations, people don’t buy a drill the buy holes in the wall. So if you deliver the result, I don’t really care if you worked in 10 minute increments, or if you just, you know, Sprint through the whole thing in four hours and then spent another four hours trying to relax and bring your mind back. Yeah. There’s, there’s, there’s that idea that, you know, it may create additional mistrust and hostility. And a last question that I have for today for this episode is if you have any questions for me, and again, let’s keep it up to me being able to answer the question
Kseniya K 1:02:46
Don’t worry, I’m not gonna challenge you with anything like to artificial intelligence Lee because I don’t know that either. And so the key thing I actually like generated this question throughout our discussion. So if you are like you’re As I see that most of your positions are more of the product manager side. So you are more from the business perspective from the overall situation tendencies and like this part of the project. Thank you. Yeah, bigger picture of the project. Exactly. So how do you think he is there might be maybe, to some extent, in the future, these threats to you to what you do to what you’re analyzing from the intelligence and from the business intelligence from the data driven approaches, and from the algorithms that are as we figure it out, advancing way too quickly. Well,
Vlad G 1:03:44
that’s a great question. Thank you. I I was the right word to them. We just said I didn’t think of this. But now that you’ve asked this, I think it’s a beautiful question. Thank you. No, I don’t think there’s a thing I think we are actually under less threat of from AI than musicians are. And I’m sure you’ve seen cases when computers write music that is more pleasant to the, to the year than humans. Yeah. And I, I’ve got I kind of I was subscribed at some point to a source of computer generated music, it just was not generated. That particular person was not generating it in the style that I liked at that time. So I lost that the thread I should probably go find it. But since computers write, you know, compose music, they write lyrics, they write, create paintings, it’s natural that they will one day start doing product management work. And my rationale is product management is as much art as it is science and computers can probably do both at some point, but in the same way They probably going to be more supporting rather than replacing role. I think we get to have a lot more insight into why people do things, and how people do things, which is how we create, and how we enhance our products, and the whole data driven approach to building your product. And that you remember you remember, I alluded to that as is there? Is there a way to look at the day they say, hey, this looks like a new product to me. Definitely, I definitely want that to happen. Because this is one of those cases when we can we can create products that create products, kind of like we’ve we had plants that manufacturer tools, and the way that those tools were used to build other tools, and then we were building add products. So it’s the same way though you can you had robotic mechanize them their robotic tools so that you just tell the tool what you want or like a 3d printer at the end of the day. So think about it this way instead of giving the 3d printer a specific dimensions or specific things, specific instructions how to print a certain item, you just tell them hey, I need something to open this cat. And then artificial intelligence would analyze that hey, this is a you know, aluminum cans, so you probably need a knife. So you probably need a knife with certain level of sharpness and you know, being sturdy and from specific type of metal so you can cut the aluminum and it would make all these decisions by itself and then print the 3d print that knife. So that would be Oh, by the way, there’s a great literature reference if you know it. I think it’s rubber Shaklee The product was called confabulate er, so you would tell him what you want. They were 3d printed or or created from something. I heard of that. Yeah, it’s look it up by I think Robert Shockley had a series of stories. This is one of the short stories. And one of the problems that because yobbo confabulate er had was he had he had a consciousness so it never made anything twice. Mm hmm. It never made any any of the thing fly. So if you ever asked it to build something from steel, it would never make anything made from steel like ever again. And that was kind of just kind of like a premise of the story. That’s why how people got into the trouble. So if you want rid of that, and you know that that’s kind of a that’s kind of a way I see this happening, so we’re definitely gonna get there eventually. I’ll probably retire by them. But I still want to see this because it makes your life a lot easier. You can stop thinking about little details and you actually can start thinking about bigger pictures and you pictures just gonna get bigger because you Now you can, instead of thinking, Oh, hey, we’re going to put you know, Part A, insert Part A into detail B or into item B, you can start thinking, Okay, so I want, I want a bicycle, right? You know, it has to be this tall or this thing. So you can, instead of writing requirements for every individual piece, every individual tiny little thing that comprises your product, you can start recording or writing requirements for the whole product. And then whatever that is to fill, show intelligence, we’ll figure out how to build that product. So that would be really cool. Yeah,
Kseniya K 1:08:35
well, honestly, I wasn’t expecting such an awesome answer. So thank you very much. That’s actually very cool to hear, like how you see the, you know, like artificial intelligence and like the data driven future from from your early business perspective is really very awesome. Thank you.
Vlad G 1:08:54
That by all means, yeah, I read a lot of science fiction. I still do. A lot of crazy
Kseniya K 1:09:00
That’s what I was implying but never mind.
Vlad G 1:09:06
All right, this has been great. Thank you so much, Sonia. I appreciate your being on this episode of the real world product management. Thanks for having me. Absolutely. It’s been a pleasure. And I’m hoping we’ll have you again, I think you have a lot more to contribute to our knowledge and understanding of advanced AI and machine learning and artificial intelligence. So I’m looking forward to talking to you again,
Kseniya K 1:09:31
Thank you for having me, thank you for today.
Vlad G 1:09:37
You’ve been listening to the real world project management and I’ll be your host, Vlad Grubman. Until next time