[00:00:00] Matt Paige: Today we’re chatting with Arda Bull, co-founder and CTO of Hockey Stack and Hockey Stack’s, a SaaS analytics and attribution platform, the unit’s website, CRM ad data so that marketing and growth teams can actually. Measure marketing’s roi. Nowhere to invest more and see account based 10 signals. And y’all been experiencing substantial growth as of late, attracting notable customers like Airme, lavender, cosm, to name a few, and I’m pumped to get into this story.

[00:00:38] Of Hockey Stack today. So it’s a story of multiple pivots on their journey to product market fit the holy grail of product market fit. So many great learnings for product and engineering leaders in this episode, including some insights towards the end. You’re not gonna wanna miss with Arda and what he’s learned on his journey of building hockey stack.

[00:00:59] But welcome to the show, Arda. 

[00:01:02] Arda Bulut: How are you? I’m excited Doing good 

[00:01:04] Matt Paige: here as well. Yeah. Excited to get into it. Hockey stack’s doing some awesome stuff right now. And hockey stack, I’ve been following y’all as of late and it’s such a great example of a product that’s built right and the way we think about that, you gotta.

[00:01:20] You gotta build the right thing, right? That’s valuable for your end user, viable for the business, feasible from a technological perspective. And then you gotta build it the right way, which is a lot into your wheelhouse on the CTO side, in terms of being maintainable, scalable, secure, and usable. And the problem that you’re solving is a big one, especially now in, recession.

[00:01:40] Hyper attention on budget. I know, I’m feeling that. And you’re going after a. Problem in the market. But to start though, I want you to take us back to the beginning. When you started, it was the height of the pandemic. You had an initial vision of what you wanted to build, which is actually different than where hockey stack is today.

[00:02:00] But take us through that first part of your journey. 

[00:02:03] Arda Bulut: You had a great description there, but you know what I say, it hasn’t always been like this the first year, especially like it wasn’t easy as you said there, there has been a, and the first product, even though it was like always an tics product, wasn’t anything like this.

[00:02:19] When we started it was like at the height of the pandemic and like we were trying to do other projects and one of the key things that we noticed there, We couldn’t really like measure product usage and like we tried to mix panel lamp to those kind of classic product A tools and maybe it was our fault, but we couldn’t really get time to work.

[00:02:40] We couldn’t really set set them up easily. So the initial, like the very, very first idea that we had was actually building a product analytics. That focused on ease of use, that focused on actually like giving insights automatically so that you won’t have to look at anything yourself. We want to use artificial intelligence, which was like, it’s weird.

[00:03:01] It was always like at the height of AI as well. Then it’s also it’s also trend right now as 

[00:03:06] Matt Paige: well. That’s the weird journey, A new height, that you’re we’re going into with generative ai, right? Yeah. But it’s interesting. Going back to that point, you mentioned you built an analytics. With the focus on ease of use.

[00:03:18] Yeah. I think this gets into part of the learning it like nowhere in that statement did I hear like the target customer or the problem you were going after? Maybe go deeper there on that initial kind of thing you were building and where you hit some roadblocks. I. 

[00:03:32] Arda Bulut: Yeah, I guess like you also had a great printer from the beginning.

[00:03:36] One of the key things that we want to do was build a product that was easy, that was like, from the setup perspective, from the usability, it had to be like intuitive for whoever we were selling to. The AI was just a way to for us to just say that there’s gonna be some magic there that’s gonna give you the numbers easily so that you won’t even have to like, analyze the data yourself.

[00:03:57] But like the. The actual first product that I mentioned now, we tried working on it for about five to six months. We were talking with people like the usual talk with your customers, talk with people, potential buyers, et cetera. We thought we were talking with them. We were getting all these like great feedback or that’s a cool product, that’s a cool idea, you should do that or something.

[00:04:21] But as we’re. One thing we noticed was no one really wanted to put the script on their website to actually track the data. No one wanted to share their like current data stack with us. So even though they were saying like cool product, et cetera, it didn’t really mean much. Then you had to talk business with them.

[00:04:38] No one gave any money to this product. Yeah, 

[00:04:41] Matt Paige: that, that’s a key piece too, right? Is that this concept of, you can get customer feedback and they may say how awesome it. But when push comes to shove, when it comes, like you mentioned, putting the, with your tool, it’s putting a script on their website, we’re actually paying for the solution.

[00:04:58] If you’re not getting those positive signals it may not be actually good enough to replace status quo of how they do it today. Yeah, 

[00:05:05] Arda Bulut: exactly. The actual validation comes v p people use the product. Not when they say they can use it or that it’s so interesting or something. That was the first key learning.

[00:05:17] We tried to get that work. As I said, we were like five to six months or something, but at the end, like we realized it, it wasn’t going anywhere plus us three. Like we didn’t have any LinkedIn press or something done, so it was like three unknown people coming from Turkey. How are you gonna trust that basically?

[00:05:35] So after that, like we realized we had to change something about the product, like we. At the same time 

[00:05:42] Matt Paige: let me pause there actually. So you’re in Turkey. It’s you and your other two founders. And you’re at this inflection point, right? And so many folks, when they get to this point, they scrap it and go find a day job.

[00:05:53] You know what, where you’re what, maybe early 2021 at this point, and you’re at this inflection point of, do we keep going? Yeah. Yeah. And what was the. What was the trigger for y’all to keep going? Was there was it somebody in the founding team that’s like, all right, we’re gonna keep doing this.

[00:06:11] Did you have an insight that kind of led you to go down another angle? What? What pushed you to keep building? 

[00:06:18] Arda Bulut: Yeah. I think it was just blind faith, yeah, 

[00:06:23] Matt Paige: sometimes you need that, right? Like 

[00:06:25] Arda Bulut: sometimes, yeah. We weren’t sure if it was gonna work. We weren’t sure, if we were actually tackling the right problem, the right audience, whatever.

[00:06:32] But we just want to build something and we like working together. So it was just like a matter of, okay, what are we gonna do? What are we gonna build, actually? So it never even crossed our minds to, at that stage, especially Find another job. It was more about what are we gonna do? I remember we had some motion docs where we were doing like pros and cons list of each idea that we have, like what’s working here, what doesn’t work there.

[00:06:58] And we had some very terrible arguments around that time on everyone wants to go in some different direction. But during that stage, one of the ideas that we had was a webinar text tool. Like instead of focusing on product antics and saying that we use AI or something, we realized that no one really cared about like the technology that you’re in, as long as we are providing some value to them.

[00:07:21] So around that time we tried like focusing on a webinar tool that’s like a competitor to Google Analytics. You can think of this as the second version of the product. Okay. The idea there was basically like tracking the same way that we were tracking like the product analytics part, but for web analytics and all actually like showing people the journeys of all the visitors that they had, giving them like easier to understand dashboards rather than going to like Google Analytics and like going through all their like complex data visualization methods.

[00:07:54] That was the second idea Around that time, there were of simple webinar tool, like privacy friendly tools that were coming out as well. So like we wrote their wave along with them at that point. Yeah. And one of the key things that we did around that time was actually applying to a website called AppSumo.

[00:08:14] It’s like a lifetime deal platform. Have you heard. Yeah, I’ve heard of it. Yep. Yeah. Yeah. We applied there and it was, I think er actually applied there, but he didn’t really think much of it. He just filled out an application and forget about it. And he just left the product there to chill on its own for a while.

[00:08:32] And then after a month or so, as we were still like deciding on what we were gonna do next, we realized that there were like a little traction there. That may be like a means something. Oh wow. Yeah. Like we came back. So 

[00:08:45] Matt Paige: You didn’t even realize it was getting traction. It just, it was something you had did.

[00:08:50] That’s what I love about so many journeys and stories. It’s these random serendipitous moments that happened. So you started to get traction which, gave you another kind of nugget of insight of, okay, there may be something here. 

[00:09:05] Arda Bulut: Yeah. Yeah, exactly. I think around that time, like it’s made about one one k or $1,000.

[00:09:12] We taught us the, again, thing there. We saw that, like we said, okay, maybe there’s something here. So like we decided to invest more in depth, small channel. There are like some Facebook groups or other like committees that they have for the buyers there. So basically we just. Try to be more active talk with the customers around there.

[00:09:31] And as people trusted us more, like we try to actually gain some traction from the part which is like easy to use website analytics tool. This is before like any attribution, before B2B assess any of the current things that we are working 

[00:09:44] Matt Paige: on right now. So who is your target customer at this point? Yeah.

[00:09:48] Or did you really have a target you were going after? 

[00:09:51] Arda Bulut: At that point, we didn’t even choose a target audience. It was whatever customer basically showed us, mostly agencies and e-commerce people though their audiences Usually those people, like the Facebook groups are full of them.

[00:10:04] But yeah, like we started gaining some traction there. Like people really like the product. I think one of the key things there was playing the underdog against a big tool like Atlantic, because like then you become that big. There are gonna be out of people that don’t like it. There are gonna be out of people that like really hate it.

[00:10:24] That’s also like one of the things, all of those simple Retics tools used. And we tried to use it as a, like the alternative tics, like the analytics that you’ll actually want to use. That was the messaging around that time. People got behind that, like they were sick of Google, Netflix.

[00:10:41] We also tried to fight with session recording and heat map tools a little bit as well. That was like a time where we were trying to position us based on like other tools. That kind of works for a while, like if you’re going for that kind of a why, but I think in the long run it wasn’t gonna really work out because.

[00:11:01] At some point you have to change your messaging so that your product is at the focus of it instead of some other product. Yeah. If you have, that’s interesting. Yeah. I think if you have like another tool in your header in your website Yeah. Like in your main page. Yeah. I think that’s gonna be a problem later on that you should probably think about.

[00:11:20] Yeah. But it’s worked for a while. The money from the money we made from AB. Probably the pree round that we did there, like just from the buyers, just from the cost that they bought there. That really helped us going for at least like another year or also 

[00:11:35] Matt Paige: that money. So at this point you’re you’re at this next inflection point, you’re starting to get some positive signals.

[00:11:40] You actually have got some kind of revenue coming in. You mentioned like these agencies are interested, but you still haven’t gotten to click this is it. We’ve got product market. What’s that next inflection point that got you? I think this is where you actually start to get to what hockey stack is today.

[00:11:56] Yeah. What was that next inflection point? 

[00:11:59] Arda Bulut: Yeah. Basically like we got the money from smo. We were doing good, but the problem this time was like the customers and the features that they were requesting wasn’t really aligning with the vision that we. For the website analytics tool leader, they were asking for like white labeling features.

[00:12:18] They want to basically show use our product, show it as their own to their own customers, especially like the agencies. And we didn’t wanna go down that road, you, your product then wouldn’t have any, Brand or something we, yeah, as a SaaS. Our SaaS, like we felt closer to other SaaS businesses, but we didn’t really have a way to validate the idea to actually focus on that.

[00:12:40] So around that time, like the second pivot that we were about to make was more about an audience problem. Rather than like the actual product, because we were like happy with the product. It was usable, like people were getting value out of it. So we didn’t really think about the product aspect that much around the pivot.

[00:12:57] So what we did there to actually decide on what we were gonna do next, how we were gonna execute that is talk again with a lot of. But it isn’t just like talking about some abstract concept or like a problem that they might be having that they just stay to like in a cold. They actually had something to show to them, like the actual product.

[00:13:17] And we could ask them like, this is the product, this is how can use it. Would you use this? Like, how does, how do you think this works? Fits in your workflow? That was like the big question that we were asking around that time. Unlike. We tried that with e-commerce people. We tried that with agencies and we also tried that with SaaS people.

[00:13:36] And what we realized that SaaS people were generally a lot more responsive to our messages. Yeah. They were like, they really wanted to like help us out as well. And they were also interested in the product. But the key thing is most of the people that we talked to weren’t interested in 90% of the.

[00:13:56] Wow. I remember one person just said I’m not gonna use this. I’m not gonna use that feature. I’m not gonna use this page. All I want is this specific thing. And that specific thing that they wanted to see was actually attributing revenue back to block posts. It was like insight that we had. Yeah.

[00:14:15] I love 

[00:14:15] Matt Paige: that too. Cuz some people may hear that, that, oh, I’m not gonna use 90% of your. And they walk away with their tail between their legs. But what that customer just gave you is like the biggest inside of all. Yeah. Like here’s the gold, this 10% right here is what I care about. And not only what I care about, it’s what I would pay for.

[00:14:35] Yeah. And what’s shout like props to y’all for actually now focusing in on that area. So now you’ve started to understand. Who’s that core customer that kind of B2B SaaS marketing person looking for attribution, and you’re getting to what is their job to be done, which is beautiful. And now you’re starting to you’ve gotten to, what’s that core problem that needs to be solved, right?

[00:14:59] Yeah. At this point, 

[00:15:00] Arda Bulut: yeah. At that point, like we started the messaging with the same thing that they told us, like at between revenue back to block post. And the fun thing is around that time we didn’t even. That much about attribution, like it was just a, like a funny word that we heard about. We didn’t even have that functional thing, the product, it was just like a precursor to that.

[00:15:22] So just that weekend, like Pura just hacked away, built like that, the very, very first attribution feature onto the product and try showing. To be SaaS businesses and like with that, with actually like being able to show that they were a lot more like open to their problems. We could really talk about like the core problems that you mentioned that they were having and we realized that it isn’t just about block post or like revenue there, it’s about actually unifying data.

[00:15:52] That they were getting with the rest of the tax stack that the company is using. Because without that, like the, I think someone else just said that like they were, every month they were praying that the blog post, that they were like publishing will have some kind of traffic Samsung kind of visitors because otherwise, like they had nothing to show to, like execs to C suit, whatever.

[00:16:13] They were just like praying to get that success and they had no way to measure it. They had no way to optimize it. 

[00:16:20] Matt Paige: Yeah. When the alternative is praying for success, then yeah. Yeah. You got a good if you can solve against that, then you got an opportunity space there.

[00:16:28] That’s such an awesome story. I’m curious though you’re the CTO of this product. Yeah. I’m assuming it’s built on a really modern stack, being a new solution. But I’m seeing like every week, like new features, new functionality. Being built. What do you attribute that to? Y’all’s ability to quickly iterate and build and deliver.

[00:16:48] Not only build the new features, but actually deliver and put them into production in, a safe and secure way. 

[00:16:54] Arda Bulut: Yeah, I think it’s about like the mindset, because even two years ago when he first started, deciding on tech, like the tech, actual tech stack was just about, okay, which technologies do we know?

[00:17:08] Yeah. Which are technologies that we can actually like push some code, some production server, and what’s the fastest way to build the mvp? Basically, that’s what we like Bora and first thought. And that’s how we like built the frustration of the product that like some parts of that code is still being used in production right now.

[00:17:26] But like from then on it was about always choosing the simplest text that you can have for that stage of the company so that you can quickly find something to show off to people. And like even now, while we are like pushing features, it’s about making, like finding the simplest way to actually build that.

[00:17:45] And I’m pushed that and iterate over it over time to actually make it like the complex thing that it is now. I think like Base Camp had a great example about this, like while building their calendar feature, they didn’t just go out and build like this complex calendar, but instead they tried to understand the core problem that people are having.

[00:18:05] Yeah. And then build feature around that core problem instead of just saying okay, we should build a calendar or something. In our case as a we don’t just go out. Build the most complex thing and see if that works for the people. We try to like iterate over the process to make sure, like the first version works, second version, not that well, maybe we improve it at the third version.

[00:18:26] So that way like we’ll have something to show the people every week. Every week there’s something like new happening in the platform. Yeah. And right now especially, 

[00:18:36] Matt Paige: No, I was just gonna say, you’re speaking my language. Basecamp is such a good example and use case Yeah. Of how to do this. And like you said it’s quickly iterating and not being afraid to put something out there so people can react to and you can continue to iterate on, right?

[00:18:50] Arda Bulut: Yeah, exactly. Like we have some features that not also people use. Sometimes we remove features from the product that we know that no one’s using. So that’s, that also happens, but you should invest like from the start, try to invest less than you would normally do. So then you have to actually build, remove it from the product.

[00:19:10] It’ll be such a big close at the end there. Yeah. That’s like a big thing there. 

[00:19:15] Matt Paige: And that’s such a good nugget too. It’s the everybody’s always in the mindset of build more features, put ’em out there. But what you just mentioned was critical. It was. If a feature’s not being used, if it’s not adding value, remove it.

[00:19:29] Because at the end of the day, like you’re only creating more complexity in the solution for your user. I like to think of it as you’re forcing your users to burn more like brain calories, right? With the more stuff you have out there. I love that approach. Even early on, y’all are taking stuff out.

[00:19:45] If it’s not adding value to keep it lean and very focused on the problem it solves. 

[00:19:51] Arda Bulut: Exactly. The simplest example for that is like in the sidebar, for example. You think about how many things you have in the sidebar and like how much page that you have. Yeah. Just the other day, like we had to remove one feature, like the complete feature from the sidebar because like we knew no one’s using it right now.

[00:20:09] It isn’t like the key thing in the product right now. So you have to like sometimes do those kind of sacrifices to actually make the product. More intuitive, like it comes back to the ease of use as well. If it’s less complex, then people are like more likely to use it more. 

[00:20:24] Matt Paige: Yeah. That’s a great segue.

[00:20:25] And that’s a big piece of what we think about as built right? Is the product usable? Yeah. And it’s more than just the ui, it’s the actual user experience. And one thing I love about Hockey Stack is y’all don’t just think about it in the span. I’m a customer. I’m in the solution. You take it further than that, and I see this with the interactive demo that people can use Yeah.

[00:20:46] Online. And I know that’s an engineering effort to do that. There’s all kinds of, how quickly you can get it set up. I think y’all, you’ll talk about, you can get set up in two minutes. Talk about that and how you think about the importance of ease of use Yeah. In the product. 

[00:21:01] Arda Bulut: The thing there is like, there’s always gonna be some effort to actually set this and use these tools, but it depends on whether you’re putting the effort on the customer side or the developer side.

[00:21:11] And like as much as possible, we try to put it on our side, put the weight on our shoulders. So that for the customer, everything looks automated. Everything looks like very easy set up. That means that we have to do out of con like configuration out of like generalization on our side because we integrated a lot of tools.

[00:21:28] We get out of data from from these customers and they have different configurations of these tools. We don’t ask them to actually provide us. All these information about their configuration, like they don’t have to fill all these forms to actually integrate at all. For them, it’s just one click.

[00:21:45] But for us it’s actually making sure in the background that everything works according to the generalized model that we have for our data. So it’s It’s about who is it gonna be hard for either you or the customer. And I would always prefer for it to be hard for myself other than the 

[00:22:01] Matt Paige: customer.

[00:22:01] Yeah. I’m stealing that. I love that concept of putting the weight on your shoulders and not your customers. That’s such a great way to think about it because it really, you have that trade off, right? It can be on your customer’s shoulders or it can be on yours. And one thing I heard you mention, like a big part.

[00:22:18] Product or solution are the integrations and making that easy. Yeah. How do you go about prioritizing and determining which integrations to add to the platform? Do you have any kind of criteria you go through when you’re saying, let’s prioritize this integration first over this integration within the solution?

[00:22:37] Arda Bulut: Yeah, that’s a good question. And like it has a very simple answer. Whatever video. It’s things that people like, it’s things that our customers ask us. It’s things that we In the download. Yeah. So like the simple metric that we use is okay, who is asking for this? How many people are asking for this feature?

[00:22:56] If we have just one person asking for integration, we still put that in the roadmap, but like a little. Lower than the other things. And if we get like a lot more people think we use this tool as well, if they like mentioned that thing, that test gets prioritized more and more until it’s like in the cycle for this week, in the cycle for next week.

[00:23:17] So it’s like very simple. But it works like for the last couple months at least, we aren’t building anything that our customers aren’t asking. Yeah, it’s just local. So many good. 

[00:23:29] Matt Paige: So many good insights here. And that’s the thing. Like so many people, I think overcomplicate this, what you just said is, does the customer need it? Have we heard them ask for it? And the big piece there is you’re actually continuing talking to customers. Listening to customers, which a lot of people overlook. A lot of the times people will talk to users and customers at the. But don’t do it throughout. Yeah. And you make it easy in that way cuz it’s not some complex formula or prioritization framework.

[00:23:56] It’s no customers have told us they want this, they need this. So we prioritize it in that way. Exactly. Yeah. To wrap it up. I got one more question for you. CTO Hockey Stack, y’all are growing. Y’all are, we’re just chatting before this. You’re living it up in San Francisco now mingling with all the folks there, but what’s.

[00:24:15] The biggest thing you’ve learned, like if you could go back to your former self at the beginning of this, what’s that one piece of advice that you would give your former self or another young, CTO or engineering leader about building a solution that can scale and grow? 

[00:24:33] Arda Bulut: Good question.

[00:24:34] I will. The biggest thing while building a product with limited resources is like the last thing we talked about, actually, prioritization. So like you have, it’s like a very simple problem. You have about, I don’t know, eight to 10 hours that like you can potentially work in a day and like most of the day, like even though you think that you are prioritizing the same things, if you actually.

[00:25:01] Drill, drill down and actually see what you’re doing at hr. Usually there are gonna be things that don’t really matter that much, but you still do because you think that they matter because you didn’t like criti. Critically think about those stuff, but if you can actually prioritize your day as well as prioritize the futures, you can actually have a lot more impact.

[00:25:20] I’m like a big. In the 80, 80 20 rule or yeah. Yep. Most of the things is gonna come from that 20% of the work that you’re doing. The other 80% is just manual things that we can probably like automate or just don’t do at all. So if you can actually correct the code, if you can actually always try to optimize the process that way you’ll be able to move a lot faster than a lot of people are think like, how are they actually building all this so fast?

[00:25:47] But in fact you are just focusing on that like key part, key 20% that’s giving, like creating delusion of that. You’re doing like everything at the same 

[00:25:56] Matt Paige: time. Yeah. Yeah. Such a foundational lesson there. And it’s broader than just. For engineering a solution that applies to life and everything.

[00:26:06] And going back to that customer you talked to, sometimes it’s that 10% of the solution they care about. Yeah. But that’s awesome. But Arda I appreciate the chat. It’s been great having you on. Built right. Thanks for joining us today. Thank you. All right, so let me stop.

[00:03:02] Matt Paige: Jason, let’s kick it off. So welcome everyone to our first edition of Built Right Live. If you’re not familiar with the Built Right podcast, we focus on helping you build the right digital solution the right way. Check it out on all the major podcast platforms. We drop a new episode every other week. We got a good one to drop today too with Erica Chestnut, Head of Quality at Realtor.com.

[00:03:24] Matt Paige: So go check that one out. And like we said, please drop in comments as we go along. We’ll be checking the comments and that may tailor our conversation a bit as we go. But today we got a really good one for y’all. We got special guest, Jason Schlachter, founder of AI Empowerment Group and host of the We Wonder Podcast.

[00:03:44] Matt Paige: So he is got some, podcast chops as well. But Jason, give us an introduction so folks have a little context of your background, your history, and what AI Empowerment Group exists to do. 

[00:03:57] Jason Schlachter: Awesome. Thank you, Matt, for the introduction, and I’m glad to be here. This is exciting. I see a lot of comments coming through chat which is making it also great to see the, participation already.

[00:04:07] Jason Schlachter: Yeah, so my background is in AI primarily. I spent about the last 22, 23 years in the AI industry. I went to school for a master’s in AI in 2001, back when there were basically no jobs in ai. And that led me down a path where I started off as a researcher doing a lot of work for DARPA Defense Advanced Research Project Agency, Army Research Lab, Naval Research Lab, Nasa,

[00:04:34] Jason Schlachter: Intelligence organizations, all kind of stuff that you could imagine would use AI before mainstream businesses were going crazy for it. And at some point I left that world moved into a strategy role led AI strategy at Stanley Blacken Decker for their digital accelerator. And then from there went over to Elevance Health, which owns Anthem Blue Cross/Blue Shield.

[00:04:56] Jason Schlachter: And there I focused on leading the RD portfolio and strategy mostly around AI and then as a product lead for their clinical AI work. And so, since leaving Elance, at AI and Empowerment group our focus is really on solving the people part of ai. That’s the way I like to sum it up really nicely cuz what I’ve seen, and I think a lot of research supports this, is most efforts to deploy ai.

[00:05:19] Jason Schlachter: Do not return the business value that people expect it to return. About 90% of AI initiatives fail to deliver on the business value that’s promised. I’ve seen many organizations where it’s a hundred percent. It’s almost never a technical reason. It’s almost always something at the organizational level.

[00:05:39] Jason Schlachter: So there was a maybe a misunderstanding of what was expected for the project. There wasn’t a deeper, nice deep, enough vetting of the use case. There’s maybe misunderstandings by the sales and marketing team, so they weren’t able to sell it. The project was canceled at the last minute because of legal concerns, data concerns, contract concerns.

[00:06:00] Jason Schlachter: So AI Empowerment Group really addresses all those non-technical. Challenges by upskilling the workforce, getting them AI ready so they can make the right decisions by holding workshops to help figure out which use cases are worth pursuing, building out the strategies to support that and much more.

[00:06:18] Jason Schlachter: But that’s, a highlight. 

[00:06:19] Matt Paige: Nice. Yeah. Yeah. Awesome. Jason. So everybody listening, I wasn’t lie when we said it. We had an AI expert. He’s been in this game for a while, that the hype around generative ai he’s, been at it much longer than that, but those who don’t know HatchWorks we we’re your trusted digital acceleration partner delivering unique solutions to achieve your desired outcomes faster, and really on a mission to leverage AI and automation.

[00:06:45] Matt Paige: Paired with the affordability and scale of nearshore to accelerate your outcomes. But Jason I’m pumped about this conversation today. We’re giving people a, sneak peek into our generative AI playbook, but hitting on one of the most foundational concepts with, which is how you actually identify and then vet some of these use cases.

[00:07:05] Matt Paige: But let’s, start at the foundation in order to start defining use cases. Let’s ground people in what generative AI is and what it isn’t to set the stage there. Awesome. Thank you, Matt. 

[00:07:19] Jason Schlachter: Yeah. So let’s talk a little bit about generative ai. Generative AI is a subset of the field of ai.

[00:07:25] Jason Schlachter: And the field of AI has been around for a long time, like thousands of years. And I know this is sounding crazy when I say it like that, but I’m gonna, I’m gonna back it up for a minute. So even going back to the biblical tests of the Old Testament there are like parts that talk about ai, they talk about people creating autonomous machines and systems that can do tasks that can operate autonomously to take away the, menial work that people don’t want to do.

[00:07:53] Jason Schlachter: And they talk about these, systems as created things that just don’t have souls, don’t have consciousness. And I think philosophically they were already addressing a lot of the use cases that we could even think about today. So thinking about the use cases for ai, for auto automation, for robotics, It’s been happening for thousands of years, which I felt was shocking when I figured that out.

[00:08:17] Jason Schlachter: And, so moving forward to today the, modern field of AI emerged in the 1950s and in the sixties and seventies. It was research in the eighties and nineties. It was commercialized. It was already a multi-billion dollar industry in the eighties and nineties. I think a lot of people don’t fully realize that.

[00:08:34] Jason Schlachter: And then of course, in the last 10 years or so, it’s really gone completely exponential. There’s been big data, deep learning, generative ai, adversarial networks. It’s just a full breadth of everything. And I think most recently we like to see things through our human eye like lens.

[00:08:50] Jason Schlachter: We anthropomorphize everything. So for the first time, like in a long time, it’s not some system in some enterprise that’s making some pricing decision. It’s this thing you can talk to and it talks back to you. And that’s scary and exciting and interesting. And I think that’s what’s driving a lot of the hype.

[00:09:08] Jason Schlachter: And it’s generating things. So for a long time, we’ve often said that creativity when, and creativity is hard to define, but like creating things is the human quality that machines will never have. And now they’re doing it. And so there’s questions like, what is art? What does it mean to compose something?

[00:09:24] Jason Schlachter: Who can win an Emmy? Who can win a Grammy? And so this is like really what’s, causing the hype? So, generative AI is artificial intelligence that generates content. And the kind of content it can generate in today’s world is text like, documents, words, phrases code, because code is text.

[00:09:46] Jason Schlachter: So it’s just a, certain type of text. It can generate images videos, 3D content, like for games it can generate music. You guys might have seen there was a Drake song that came out that was supposedly like, pretty popular, actually. Sounded good. 

[00:10:02] Matt Paige: Matt, did you, I’ve not seen that yet. Was it produced?

[00:10:05] Matt Paige: They did some generative AI to produce it. 

[00:10:08] Jason Schlachter: Drake didn’t produce it. Somebody else produced it, but it was Drake singing it. Oh yeah, Drake, he found out about it after it started becoming popular and it was like his voice and his style to his music and, somebody just basically trained a model on his voice, his style, and dumped it out there.

[00:10:26] Jason Schlachter: And there’s just these questions of what does it all mean? It can generate speech and audio in that same use case. Other side of it is like very, like hard sciences. Generative AI can generate like biochemical sequences like protein molecules. So very, open. In terms of what’s possible it is probability based.

[00:10:48] Jason Schlachter: It is based on, deep learning architectures. Which means that it’s, probabilistic. And I won’t go into the technical side into exactly how it works, but it’s not thinking and reasoning in like a symbolic, causal way. It doesn’t understand that if it rains today, the ground will be wet in, in like, a a very expressive way.

[00:11:11] Jason Schlachter: The way we understand that, it just has some miracle representations that, are able to connect those concepts together. And so it might respond intelligently but, it doesn’t actually think and understand in the way that we typically would expect. It also will reflect any kind of bias or flaws that are in the training data.

[00:11:29] Jason Schlachter: So if you had healthcare training data, and in that healthcare training data, certain members of the population are not getting the care they need for like societal reasons, not clinical reasons. And then you trained an AI system to make decisions about what care should they get, when they should care, when should they get that care for the best outcome that bias would pull forward into the model.

[00:11:52] Jason Schlachter: There’s ways to mitigate the bias and but, generally this is a challenge. If you have bias in the data, you have to account for it the best you can and the bias will show up in the end. And so with generative models, it’s the same. If we write with prejudice or bias or hate speech, like it shows up in the generative models as well.

[00:12:12] Jason Schlachter: It also pulls us into the post content scarcity world. Like up until this moment we basically lived in a world where there was a limited amount of content. At some point it was hundreds of books in the world and millions of books in the world. Now there’s no number of books in the world.

[00:12:30] Jason Schlachter: There’s an infinite number of books in the world that can be generated on demand. And so that really changes the whole world in which we operate. 

[00:12:40] Matt Paige: Yeah. No, that, that’s awesome context setting there. But what was really cool was the, history dating back to biblical times. I was not aware of that.

[00:12:49] Matt Paige: That, that’s super interesting. But you, like the Drake example. You mentioned you can think of whole business models changing here. That’s a big piece of this. You also think of the accuracy of the data, and we’re gonna get into that in a minute when you’re talking about vetting some of the, viability of these use cases.

[00:13:08] Matt Paige: But I think one big piece of it is with a hive cycle, you sell this in the dotcom boom, a lot of this, there’s a lot of people. With a hammer in search of a nail, right? Yeah. The hammer being generative ai. Yeah. Let me go find a nail, lemme go find something I can do with this. Yep. And back to basics. It’s important to flip that and focus on the outcomes and relative use cases first, but maybe take us through like how, to think through some of the like, higher level business outcomes to start to bucketize where you can focus some of these generative AI use cases.

[00:13:42] Jason Schlachter: Yeah, absolutely. And, maybe I can start to Matt, with a, bit of the, why we’re going through this and what it means to find these use cases and, I’ll segue into, some of those. Okay. In this talk finding the use cases, validating the use cases I wanna talk about a couple like preamble type things.

[00:14:04] Jason Schlachter: So first, if you’re out there with customers, if you’re out there. Trying to solve problems, trying to figure out how to make your, product better, trying to reduce your claims processing costs, like you are the expert and you are the person that knows the, opportunities and the needs that, you could address.

[00:14:23] Jason Schlachter: And so in that sense, like you’re the perfect person to find the use cases for AI and generative ai, and it really is on, on, on your shoulders to elevate those opportunities and bring in the rest of the stakeholders. And so I think to do that, it’s really critical that you understand at a high level, at a non-technical level, like what is ai, what can it do?

[00:14:44] Jason Schlachter: What’s hype, what’s not hype? What are the opportunities and risks in, in pursuing this approach? How would I frame out and scope and describe this use case in a way that I could bring in the other partners? To be a part of it. And so there is this, ability for you to do that with a fairly basic understanding of how to think about these things.

[00:15:05] Jason Schlachter: And that’s our goal here today, to get that, basic understanding. And then if you think about finding the use cases, making the plans there’s a need to make a plan, there’s a need to find the use cases We don’t plan to have a plan. We plan to get good at planning.

[00:15:22] Jason Schlachter: And the reason why is because your plan doesn’t survive first contact with the customer or because of where I spent most of my career, first contact with the enemy. And so understand, right? I, had to adapt as I shifted from the, defense world to the right consumer world. I had to change a lot of my phrases and sayings.

[00:15:42] Jason Schlachter: And this is one of them. First contact with the enemy to first contact with the, customer market. And I, and we live in this dynamic world. So in finding these use cases, like it’s not that there’s gonna be the perfect use case. Like the goal here is to get good at finding use cases, to get fast at validating them.

[00:16:00] Jason Schlachter: And, trying them and learning. Because the faster you can do that the, better you’ll get to keep up with this exponential curve that’s ahead of us. And then the last thing I wanna say is we are here to talk about generative AI because it is exciting and there’s lots of things you can do, but for most businesses, Most of the use cases for AI are not gonna be generative ai.

[00:16:22] Jason Schlachter: Like most of the business value is gonna come from the stuff that is not taking up all the headlines right now in the media. It’s gonna be pricing your products dynamically or better. It’s gonna be automating some of your internal customer service or claims processing. It’s gonna be facial recognition on your I don’t know, like your product, that makes something a little bit easier for your, consumer to, to log in.

[00:16:48] Jason Schlachter: So even though we’re here talking about gender and AI and it’s very exciting, I just wanna put that in perspective because you don’t wanna be looking with this hammer for all the nails in your organization. This is just like one tool and it’s a very, powerful tool and that’s why we’re talking about it.

[00:17:04] Matt Paige: Yeah. And I like the way, I like the way you framed it. It’s like building the muscle. That’s the essence. Building the muscle of how do you go through this process. To get to the end outcome that you want to get to. So that’s a, foundational piece of what we’re trying to do. Think of this as like a workout.

[00:17:20] Matt Paige: Y’all this is the, intro. We’re, the trainer. This is the beginning of the workout. Yeah. So and I think there’s different areas you can find opportunity, right? There’s internal areas, there’s external areas. It can be revenue generating co so there’s different focuses where you can start to think through where do you wanna focus some of these efforts.

[00:17:40] Matt Paige: But any thoughts on that? 

[00:17:42] Jason Schlachter: Yeah, exactly. There’s a great quote by Douglas Adams, which says that technology is oh God, I’m forgetting the exact verbatim, oh, technology is a word for something that doesn’t work yet. And I think it’s a great phrase because if we’re talking about ai, it means we’re not talking about a solution.

[00:18:05] Jason Schlachter: It’s a technology, it’s not a solution. And so we want to pivot to what solutions could be, right? So optimizing your internal company operations could be improving a product or service for a customer, could be optimizing like your defenses, your cyber your, it could be improving your documentation.

[00:18:27] Jason Schlachter: So there’s all these different kind of use cases that are either optimizing your business or innovating your business, helping your customer in some specific way. And I think if you look at it at like the industry level we can dive deep into some more like industry level type stuff. There’s a lot of specific use cases at the industry level.

[00:18:45] Jason Schlachter: So like on the financial side these kind of models can be used for customer segmentation. You could custom, you could segment out customers by needs and interests. Targeted market campaigns. You can do risk assessment, fraud detection in healthcare. You can do drug discovery, personalized medicine, medical imaging.

[00:19:05] Jason Schlachter: On the manufacturing side, there’s product design, there’s manufacturing planning and quality control. There’s on the technology side, there’s more efficient coding, software development and processes, cybersecurity, automating data science. I’m just running no one’s. You don’t. Do you guys want to remember all these?

[00:19:22] Jason Schlachter: I’m just trying to give you like the shotgun view of oh my God, this is a lot because this is only a small bit of it.

[00:19:26] Matt Paige: There’s something you said just leading up to this, we chatted about, and there’s this sense where people can stay at the surface level of what AI, generative AI can do, but where you get the gold is where you focus into a specific domain discipline, where your area of expertise is.

[00:19:46] Matt Paige: That’s where you find something unique. So it is important to think about within your industry, within your business, within. The, problems that your customers have. Yeah. That’s a key element to where you’re thinking how you can apply these things. And another, I heard someone talking the other day when you’re thinking about what you wanna roll out in use case and all that.

[00:20:07] Matt Paige: Take the word AI out of it, and does it still have value? Yeah. Does it pass that smelt test? Like you reference like the, Google and Apple events recently, apple didn’t mention AI really at all, but it was foundationally in just about everything. 

[00:20:22] Jason Schlachter: Yeah. That’s a really stark, that’s a stark example of that.

[00:20:25] Jason Schlachter: Google talked about AI a lot. Apple didn’t talk about AI at all. And I think Google positions themselves to be a company that delivers AI as a tool, right? Like they’re selling ai as a solution. Apple doesn’t really try to sell you ai. Apple tries to sell you a good experience, a seamless experience.

[00:20:46] Jason Schlachter: So there’s not a strong need to talk about AI specifically. They might talk about like intelligent typing or smart notifications or something like that. And that makes a lot more sense. Matt, I think maybe if you want we could jump into some of these sort of questions that help. Yeah.

[00:21:03] Matt Paige: So, just to set this up, this is one of my favorite areas. So many folks I think get stuck early on thinking in an incremental nature versus kind of a stepwise trans transformational nature. So j Jason, take us through these questions. Great place to start. If you’re talking with folks in your business trying to facilitate and exercise around this, take us through some of these questions and how to think through ‘

[00:21:28] Jason Schlachter: em.

[00:21:29] Jason Schlachter: Okay. Awesome. Yeah, so these questions are, very simple. They don’t even say anything about AI specifically, but they’re gonna help you get to the core of the use cases where you could deploy generative ai. And in a bit we’ll talk about and how you validate and assess those opportunities.

[00:21:44] Jason Schlachter: All right, so this is a question that I, I heard from some buddies of mine at pro Lego. It’s a AI consulting company. When we were talking about use cases if I had an unlimited, a number of u of interns how would I deploy them to maximize business value? So that’s a question to ask yourself.

[00:22:01] Jason Schlachter: You have an unlimited set of interns, you’re in charge of them all. Where do you put them? I think like some people might be like, I don’t really know. Other people might be like, oh my God, yes. Like they need to go do this one thing for me. Because that will save my life, right? They need to go and sit in our call center because that’s where our customers suffer the worst.

[00:22:23] Jason Schlachter: Or you need to go and review all these claims because we’re six months behind on processing claims. If you can do that, then you can talk, you can find friction point or an opportunity that would benefit your company or yourself. And there’s some different variations of this question that I would ask too.

[00:22:44] Jason Schlachter: Matt and I were going over these earlier and just spinning up different versions that hit at like different slices. So another one would be like, in addition to unlimited interns, what if you had unlimited staff? So you manage a team of infinite you go from team of however many you have now, 5, 10, 50 people to unlimited people.

[00:23:04] Jason Schlachter: What would you have ’em do for you? 

[00:23:07] Matt Paige: Yep. Another way you framed it too, I think you said, what if I had a small country working towards a problem I had, just to put it like in context. But what that’s doing is that starts to sound kinda honest. It Yeah. That it does. You’re right. A lot they’re. There’s all kinds of dystopian stuff we could get into as well.

[00:23:25] Matt Paige: Yeah. But it’s it’s that reframing though, cuz it’s not so much about the people element of the resources. And that’s the beauty of starting to trigger some of these questions when you are de dealing with technology like ai, it takes some of those constraints out of the equation or it flips the script a bit.

[00:23:44] Matt Paige: So that’s the idea behind some of these. 

[00:23:46] Jason Schlachter: Yeah. And so I’m gonna continue this, Matt, with a few additional questions. Yeah. Keep going. Gone from interns, so not super skilled, but maybe very eager and capable to staff, know what they’re doing. Next one I wanna ask is, what if you had unlimited experts, you could bring experts from all fields to your team to help you, what would you have them do?

[00:24:09] Jason Schlachter: That takes it up a level now, because one of the things that generative AI can do is it can empower people to do things that they’re not experts in, but they can do with generative ai. So I’m not an expert painter, but I love art. I have a lot of ideas. I’ve seen art. If I can describe verbally my perfect vision for a painting, then I can use generative AI to create that painting.

[00:24:31] Jason Schlachter: And, it’s gonna look really good. It’s gonna look like a professional work of art if I do it right. So I’ve become like an expert in the sense that I’m now an artist. There’s probably a lot of philosophical arguments about did I create it really? And can I view myself as an artist?

[00:24:47] Jason Schlachter: But, practically speaking it will be difficult for people to differentiate between that AI created painting and someone creating a painting. So if you had experts, how would you use them? Okay, we’re gonna keep going. So 

[00:25:02] Matt Paige: we got some good questions popping in the chat. Not that we have to hit ’em right now, but there’s some Keep, keep ’em coming, y’all.

[00:25:07] Matt Paige: We’ll, try to weave some of these in a minute. Yeah. Keep hit, keep hitting the questions. 

[00:25:11] Jason Schlachter: Okay. Okay, here we go. All right, so this is one of my favorite ones. Up until now we’ve been focused a little bit on that internal optimization of my business, right? So how can you optimize your business internally?

[00:25:23] Jason Schlachter: Yeah, you could have used those interns to follow your customers around and give them an amazing experience, but it’s been a lot of like internal locus of, you. Yeah. Now we’re gonna shift it to external. So if you could give every one of your customers a personalized team of as many people as needed, five people, 10 people, a hundred people, and their sole job is to give your customer an amazing experience, what would that team be doing for your customer?

[00:25:49] Jason Schlachter: I think that is one of the most like, Powerful things to think about.

[00:25:53] Matt Paige: That’s powerful. Yeah. And it, that is taking it from incremental to potentially business model changing disruptive use cases. And that’s the idea of this exercise, right? It’s starting to get into more of that blue ocean starting to just generate the ideas, get them out there with a reframing and hell are some of them in chat G P t and give them some context and ask ’em there you can have them play a role in your in your facilitated exercise as well.

[00:26:23] Jason Schlachter: And so this is not to imply that you generative ai currently can fill the issues. It’s not, we’re not meaning to imply that if you’ve created this, imaginary team that you’ve given it to your customer and is doing everything to make your customer have an amazing experience, that then generative a can meet those needs.

[00:26:43] Jason Schlachter: Most likely it can’t. The point is though, is that you’re starting to get to the core of thinking with a different framing of I could write unlimited articles on behalf of my customer. I could book I could book everything they need for the entire week on their behalf. I could go clothes shopping for them.

[00:27:02] Jason Schlachter: There’s a lot of things that you could do with an AI model that can generate things and also summarize and explain things and, represent design and stuff like that. So that’s the gist of this. And then there’s one more question, the kind of fear mongering question here.

[00:27:20] Jason Schlachter: And this is if your customer if your, sorry, your competitors, if your competitors could do the same, your competitors had unlimited staff, they probably would use that staff to make great customer experiences as well. But I’m gonna frame it in an adversarial way. If they use that staff to put you outta business, what are they gonna do?

[00:27:42] Jason Schlachter: So now you have to think about this because. Most of your competitors won’t do that, but the best of your competitors will be doing that. They’ll be thinking through these potential use cases and when the technology is ready or when it makes sense from a value perspective to apply the technology in that way, they’ll be ready and waiting to do that.

[00:28:02] Matt Paige: Yeah, and this is the one, it when we were talking about these, it hits a different area of your brain when you frame it from oh shit, the competitor’s trying to put me out of business. What are they gonna do? And it, does, it gets you to think about it from a different lens in a lot of ways.

[00:28:19] Matt Paige: So those are the framing questions in essence. So this is all about idea generation, reframing how you think about things. The last point you made was interesting too. Even if it’s not perfect right now, you can still begin testing and playing around with this cuz things are progressing at a rather Alarming, crazy, whatever adjective you want to add rate right now.

[00:28:42] Matt Paige: So what may not be possible today could be possible in three months, six months, a year, five years. So it’s core that you start thinking about this now and how it’ll impact your business model, how you operate, 

[00:28:57] Jason Schlachter: right? Yeah. Cuz it’s not the technology that fails to deliver value in almost all cases.

[00:29:01] Jason Schlachter: It’s, yeah it’s the, system point of view. It’s the organizational failure. You, your organization and your team should be able to, frame these opportunities in the right way and, be data and AI driven in their thinking process so they can act fast.

[00:29:20] Jason Schlachter: Because when that new capability emerges, and we saw it when chat g PT four hit the market. There were some companies that like overnight had applications. Some of them were bogus and borderline fraud, but those have fallen away. And now we see like Adobe deploying image creation models inside of their Adobe platform so that you can like completely generate a new background for your foreground.

[00:29:46] Jason Schlachter: Or you can erase an object and then ask it to generate a new object. And it will do that in, the application. So those are starting to become more more, mainstream for sure. 

[00:29:59] Matt Paige: Yeah. That’s one we’re playing around with it HatchWorks right now. I think it’s Firefly is the name of the Adobe product, similar to a mid journey or something like that, but it’s within the Adobe ecosystem.

[00:30:10] Matt Paige: I think this is where we’ll start to transition into some of you have ideas, you have a list of ideas you’ve generated, but how do you begin to test vet the viability of. We should do these over these that’s the one of the most important things is how do you start to prioritize some of these use cases And there’s a bit of a call it a rubric or Yeah.

[00:30:35] Matt Paige: Analysis. You, take it through. So Jason start to take us down this path of how you begin to wait and prioritize some of these ideas. 

[00:30:44] Jason Schlachter: Absolutely. And Matt, let’s stick, let’s throw up our our, use case that we’re gonna use to, to illustrate some. Yeah, let’s do it. Go through it. 

[00:30:52] Matt Paige: Okay. So yeah you, set it up you got the, real story.

[00:30:56] Matt Paige: And I’d say too, we got a couple, there’s one related to the stock market. There’s one related to chat with a customer, interactions. We may play around with a couple of those later, but yeah, let’s hit the, main one. Jason’s taking a big trip in about a week or so. So Jason set up the use case for us.

[00:31:14] Jason Schlachter: Okay. Awesome. Yeah, and Matt let’s, make sure we get those questions in too. I, so the use case I am, I’m most focused on right now is travel. So I’m, heading out to Japan in a bit with the family and trying to book our, travels and I, want to be on the edge of the like touristy kind of stuff.

[00:31:34] Jason Schlachter: I don’t want to be like deep in it and, so that means I’m looking for experiences that are just like a little bit off the beaten track. And so booking hotels, looking for national parks, trains, buses, do they can, it says that the hotel room sleeps for, but I only see two beds.

[00:31:53] Jason Schlachter: Are they charging us extra for kids? Like all this kinda stuff. And it’s a huge amount of time to really dig into it if you wanna make it right. And I don’t really want to hand it off to a travel agent because. I, like the idea of being in the details. I like the idea of having the controls.

[00:32:08] Jason Schlachter: But with Expedia or Priceline or TripAdvisor what I’m having to do is I’m having to like, break down the larger itinerary in my own mind. Research all these different places, of which, most of them, which I’m not gonna go, some of ’em I don’t really understand and then look at for individual things.

[00:32:25] Jason Schlachter: Can I find a train from point A to point B and what does that mean and how much does it cost? And how do we, where do we put our luggage and can I find a hotel in this city? And I don’t really know which district to stay in all this kind of stuff. So, if I had the ability to give myself a team of, staff that were gonna work on my behalf as a generative AI might, I would wanna say to the generative ai, I’d like to take a travel, I’d like to take a trip to Japan with the family.

[00:32:51] Jason Schlachter: We want to be outdoors hiking. We want to, get our hands dirty, doing archeological digs. We want to take lots of photos. We wanna be at local cultural events. We want to be at the GU Festival in Kyoto on these dates and Super Mario World super ipo. Super important to my kids and to me. So we wanna go to that as well.

[00:33:12] Jason Schlachter: Give me some itineraries and, like figure out all the connection points and show me like cost structures and explain to me which ones are better than the others and why. And from that, It’s like I as if I had my own executive team working on this for me, and then I could look at it and I could say this looks cool, but I don’t want to go there.

[00:33:33] Jason Schlachter: Or I could even query it like, Hey, why are you putting me in this city? I didn’t ask for that. And I could even respond with we found that like people like you who have gone to Japan and visited the city really enjoyed it for these reasons and it fits comfortably with your schedule here and there.

[00:33:49] Jason Schlachter: It would just be like a very easy conversation. And from a, an, let’s say like a TripAdvisor perspective, like it’s all AI driven. There’s no customer service agents, it’s scales, there’s no time. So that’s, to me like a, use case that very clearly is gonna become dominated by generative ai.

[00:34:09] Jason Schlachter: Yeah there’s one catch and we’ll get into this moment with the things away. It needs to be right. Do not wanna be stranded with, two kids at a bus station. With a hotel that only sleeps two people, even though it, booked it as four. And that’s where generative AI is not so great.

[00:34:29] Jason Schlachter: And so we’ll talk about that. 

[00:34:30] Matt Paige: Yeah. What are the stakes? And first of all, I’m jealous of the trip. That’s awesome. You’re getting to do this. But we can put ourselves in like the seat of Expedia or a company looking to disrupt Expedia. How should they be thinking about this? And, frankly yeah, Expedia should be very wary cuz this, is the type of emerging technology that literally could upend an entire business model.

[00:34:53] Matt Paige: And just as an aside we, gotta episode coming out later with Andy Sylvestri or leads up our design practice. There’s potential for this shift from a imperative to a declarative approach, A point and click approach to like declarative where I’m talking and interacting.

[00:35:12] Matt Paige: With the interface, so it changes how user interfaces are designed. So be on the lookout for that. It’ll be coming out in a few weeks. But Jason start to take us through Yeah. You just set up the context. What are the different dimensions that you can start to way a use case that’s right.

[00:35:32] Matt Paige: To determine how viable it is. 

[00:35:34] Jason Schlachter: That’s right. Okay. So we’ll start with business value, but we’ll keep it really short because business value is something that is, well studied, and so you want to be able to assess the business value to assess the business value with generative ai.

[00:35:49] Jason Schlachter: You may wanna rapidly prototype, you may wanna do wizard of Oz kind of things where maybe you give a customer a chat bot and you label it so that it’s very ethical and transparent as you’re talking to a, generative AI bot. And it’s, very expressive.

[00:36:05] Jason Schlachter: It can look through all the documentation the all the manuals. It’s not just dumping technical information to you, but it can, reformat it and answer your questions. But at the same time, you have this whole thing that it’s AI bot driven. What you could really be doing on the backend is you could be having some of your expert customer service people quickly typing stuff out.

[00:36:26] Jason Schlachter: And so you haven’t really implemented anything technologically, but you’ve started to assess the viability of a customer accepting that they’re gonna engage with an AI and understanding how they engage with an ai if they structure their queries differently if they scope their, requests differently.

[00:36:44] Jason Schlachter: So that’s an example of the business value where you could start to get to it. Next, and this is a really big one, Andy. This is, or sorry, Matt. We do have Andy on the call. He’s MCing it all in the background. Matt, a really big one is fluency versus accuracy. For these generative models, fluency means generating content and, that’s what they do.

[00:37:05] Jason Schlachter: They generate content really, well. Accuracy means that the information is factual. And so if you asked a generative AI model, text-based generative AI model to help me write a short story about, and you explained what you wanted to write, it could dump a story to you. And it’s probably gonna read really well.

[00:37:23] Jason Schlachter: It’s probably gonna be great for creators that need help structuring their content or want to add some details to their content. It just really speeds up that kind of workflow. In that case things like hallucinations, which is a term for when generative AI models say things that aren’t true.

[00:37:43] Jason Schlachter: There’s a lot of technical reasons why that happens, but they do that. Then in that case, it’s okay because like fantasy create creativity abstract thoughts, like those are all interesting aspects of a short story. But if you have an agent that’s meant to give you medical advice and you’re asking it do you go to the hospital like, what’s going on with me? You really want it to be accurate and it’s not as important that it generates creative content or that it, yeah. 

[00:38:13] Matt Paige: And this is a new kind of dimension I feel like with AI and generative ai, the importance of this one moves very high up the list of considerations where it wasn’t as nascent as a concept.

[00:38:27] Matt Paige: I think in the past you mentioned business value, that’s still critical. Always gonna be there. Yeah. This one’s interesting cuz. It, it lit things. It can go rogue, it can hallucinate like you mentioned. And what is the, risk or the outcome of if something goes wrong? Yeah, 

[00:38:45] Jason Schlachter: So you have to, think about your use case.

[00:38:48] Jason Schlachter: Is it a use case that demands fluency? In which case it’s something that we can you can address more easily with the models. And if it’s accuracy there’s, ways to mitigate this. If you do demand accuracy you’re able to train models, you’re on your own, you’re able to tune some of the existing models.

[00:39:09] Jason Schlachter: So there’s, like foundational models emerging for generative ai. These are like open AI’s chat g PT four. But also Google has Bard, etta has I think llama, so like a lot of these companies are, building their own models. These are foundational models.

[00:39:29] Jason Schlachter: They have very large representations of language and semantics. And then they layer on top of that with ability to be prompted and respond appropriately. So these are models that you could use off the shelf for some of your business use cases. And if fluency is your goal, those are probably great fits.

[00:39:48] Jason Schlachter: But if you have a, need for accuracy, you may need to tune them on your own data. And so this is where you start to, to ask yourself, do I have enough data to do that? So it wouldn’t be impossible to generate a model that answers medical questions. It’s a great use case for generative ai if it is highly accurate and probably highly regulated.

[00:40:13] Jason Schlachter: Yeah. Maybe even reviewed by a clinician in certain, in, in certain or many use cases. 

[00:40:19] Matt Paige: Or if it reaches a state of getting into the unknown, territory, can the model be geared in a sense to where it’s not. Spitting out a random response, but it is saying, I don’t know there’s that element of it as well, which how do you start to actually monitor that, may be a bigger, totally different problem.

[00:40:44] Jason Schlachter: Yeah. There’s not a lot of, self-reflection is a challenge right now for these models. They’re, they know everything, even when they don’t because what’s in their mind. Yeah, exactly. They’ve been trained on a certain set of world data and they have a partial understanding of that data.

[00:41:01] Jason Schlachter: Yeah. And, they look pretty convincing when they talk about what they know. But when they’re asked to talk about something they don’t know, they don’t say, they don’t necessarily say, I, I can’t talk about that. They, try to answer it in the context of what they do now, and because they have like partial understandings of what they do now.

[00:41:18] Jason Schlachter: There’s not like a, an explicit like expressive representation of these concepts and some kind of logical reasoning and causal kind of way. It’s all very probabilistic. You get very weird emergent phenomenon because you can find weird edge case paths through the, probabilities of these models.

[00:41:35] Jason Schlachter: So fluency and accuracy is, a cornerstone of how you should think about your use cases. The other really big one is low risk and high risk. We talked about this just a moment ago, but what’s a low and high risk? Like Expedia sending me to a foreign country with my family and telling me to go stand somewhere on a corner because there’s gonna be a bus and there isn’t, is high risk, right?

[00:42:00] Jason Schlachter: But me jumping onto like T-Mobile’s website and asking a question in natural language and getting back like a personalized explanation. It’s pretty low risk, especially, and this is interesting. So you can do retrieval, augmented training on these models where in order to sup, like in order to suppress errors and to build confidence for the user you can force it to only say things that it can back up with a document that’s retrieved.

[00:42:28] Jason Schlachter: So in the, oh, okay. It could pull up some kind of like knowledge base article that exists in, T-mobile’s data set. And it could say this is the thing I found, but I’m not gonna make you read it. Here’s like the two sentences that directly answer your question. But if you need to dig deeper this is the document that I use to generate this answer. 

[00:42:52] Matt Paige: And and this is taking it a step further than just let’s just get the op, the open AI chat, G P T A P I and just integrate right now you’re starting to weave in some of your own company’s data information to enhance.

[00:43:07] Matt Paige: The experience, the model, all of that. So that’s upleveling it a bit versus just slapping AI on your, product service or process. 

[00:43:19] Jason Schlachter: Yeah, exactly. And then that’s a fundamental question too there’s a lot of use cases you can unlock with off the shelf stuff, but there’s a lot you can do to tune these models.

[00:43:27] Jason Schlachter: And so when you tune these models, que do you have the data? Cause you, if you’re tuning, let’s talk about if you’re tuning them. So if you’re tuning a model, why would you do it? You might do it cause you need more accuracy in the kind of these case we explained. And in that case, you need to ask yourself, do I have the data to tune it?

[00:43:43] Jason Schlachter: And so what do you need to tune it? You need your own documents that represent the, knowledge sets and the way of speaking about the things you care about. So in T-Mobile’s case, it could be like their, knowledge bases, their tech technical documentation. You also, Need, you may need prompts and answers.

[00:44:05] Jason Schlachter: So one of the ways these models get built is a very labor intensive step where, people literally write out a prompt and then write out an answer, and then they show the model both. And they use those to train the model as to this is what a good answer to this prompt should be. And, some of these bigger companies like Google and Microsoft, they have like thousands, if not tens of thousands of people employed full-time, like writing prompts and answers.

[00:44:28] Jason Schlachter: It’s a very labor intensive part of the process. So that might be something you do to tune a model. The other reason you would tune a model if not for accuracy might be performance. So maybe you don’t need a huge model. Like maybe you can run with a really small model that takes less compute. You can run it on a locally, on a device or just costs less.

[00:44:50] Jason Schlachter: But you need to tune it because you’re, building an auto mechanic helper generative AI system that that helps your auto mechanic rather than reading car manuals for cars that he hasn’t worked on for a while, he just asks the question and gets the immediate answer with reference back to the model, the manual pages or something like that.

[00:45:08] Jason Schlachter: Like in those cases it could be small, it could run on device. So those are some considerations there. And then the other piece here is what’s defensible and non defensible. Is it important for you that the model that you’re using and the use case you’re building is, defensible from a business perspective?

[00:45:30] Jason Schlachter: So yeah, let’s get back to the travel example. Would it be defensible if Trip, if TripAdvisor built that capability? I’m gonna pause, I’m gonna I’ll, throw you the question. 

[00:45:44] Matt Paige: Yeah. And, folks in the audience too, if y’all want to answer. You know what’s interesting? If it’s simply if you could do the same thing referencing chat, G P T or some large language model that’s open to the public, I’d say no.

[00:46:00] Matt Paige: It changes the whole business model and defensibility of their business. Now, if it’s leveraging to your point data that an Expedia or a TripAdvisor has that they can supplement, Into the model, then I think it does begin to have an element of defensibility. But what’s, your take? I’m curious.

[00:46:22] Jason Schlachter: Yeah, it would have to leverage custom data from, TripAdvisor. They’re not gonna get, yeah. Anything that’s capable of doing that kind of use case off the bat. They’re gonna have to spend a lot of time and a lot of money leveraging their own data to tune those kind of miles. And even then, I think it’s really gonna struggle with being accurate.

[00:46:43] Jason Schlachter: Cuz there’s so many connection points, right? Transportation, hubs, hotels, sites, but if you think about what they have they, can trace member trajectories through like cities and tourist areas and restaurants. So I, do think there’s a lot to it that they probably could do. I think it’s partly defensible on the model basis.

[00:47:03] Jason Schlachter: Yeah. It’s partly defensible because Expedia might be able to do the same. Priceline might be able to do the same. Booking.com might be able to do the same. I would argue that there’s nuances that TripAdvisor has, that they capture, like extensive photos from users and very bi like multimodal, like hotels, cars hiking restaurants.

[00:47:29] Jason Schlachter: E everything is, so across the board. But I, think even if it’s not fully defensible, they still need to do it to be competitive in their industry space. So, it’s somewhere between like differentiated and, highly defensible to the competitors might ability to do the same, but maybe not quite in the same way.

[00:47:49] Jason Schlachter: But I, think ultimately what’s interesting is like non defensible doesn’t make it bad either. Like things can be very high value, but non defensible. So in this case of TripAdvisor, it might be that the model is non defensible. Like it might be that they can build this model, but so can every other travel service.

[00:48:11] Jason Schlachter: So then there’s like other levels of defensibility, right? Like, use cases and business models were defensible before AI came along. So yeah. What other ways is it defensible? Like it could be that, that their brand alone is, helping to make it defensible. Like I don’t necessarily want a startup, an AI startup, even if they’re well-funded, sending me and my family off to Japan for a while I, might not trust it.

[00:48:37] Jason Schlachter: I would much rather go with a TripAdvisor. It might be defensible in that they have partnerships and integrations in a way that this actually works, right? Because the rubber has to meet the roads still, if they’re gonna book these itineraries. So there may be other ways to make it defensible that isn’t the model.

[00:48:55] Jason Schlachter: So I think when you think about these use cases from business perspective, a defensible model is great if you can do it. But you’re not gonna get a defensible model without spending a lot of money and having a lot of data. Yeah. So it may not be critical. 

[00:49:07] Matt Paige: I think it deals with, is it connected to your inherent value prop or the customer, facing side of the business model itself.

[00:49:18] Matt Paige: Then this defensibility question becomes really important. But you mentioned brand, actually is a differentiating thing. Now I’d say most folks it’s, the level of apples and the big ones where that’s where you see the, brand defensibility truly shining through. But that’s a critical piece of this.

[00:49:38] Matt Paige: I’ve seen there’s like websites that track how many AI startups are happening being created like every day. And there’s some where they’re literally just putting a skin on top of a foundational model and there’s no inherent defensibility to it. Somebody could have spun it up over the weekend and it’s like, how do you weave through. That in es in essence, is there something, is there substance behind it that makes you unique? 

[00:50:09] Jason Schlachter: Yeah. That’s an interesting example cuz those companies were serving a market need some in some ways. Like in the very early days, the average non-technical person probably didn’t know what open AI was, didn’t they had a website, didn’t know they could go to the website and subscribe to their model, just saw it in the news. And then they get a friendly cartoonish bot popping up on their their, iPhone ads, Yeah. Saying here for access to the model. And it’s that was, a marketing niche that Open AI was neglecting.

[00:50:40] Jason Schlachter: I think they’re, picking up on that now.

[00:50:42] Matt Paige: A great example is the, Chat G P T app. They didn’t have an app for a little while and there were competitors that created an app just leveraging chat, G P T. Yeah. And they were able to get some amount of, yeah. Probably actually crazy scale, but then Chat, G P T OpenAI came out with their app.

[00:50:59] Matt Paige: And you probably just completely killed their whole business model. So that’s like the whole defensibility piece. How easily can a competitor in and just take it over? 

[00:51:08] Jason Schlachter: Exactly. Exactly. Okay, so there’s two more things I wanna touch on here. Yeah. One of them is whether it’s internal or external facing, like this kind of relates to risk, but it’s not directly related to risk. So if you think about internal versus external if you’re using it to create, and this is really where, these generative models have the most value to create content where fluency is, the highest need and, risk is low.

[00:51:34] Jason Schlachter: So this internal facing use case of help me compose emails to my colleagues faster, or help me create marketing content that I can post online faster, or like generate blog posts for me that I can just tweak and, send out. Or summarize to me this. This document that I received from one of my partners or explain to me th this chain of emails, like those kind of things can really boost productivity.

[00:52:05] Jason Schlachter: They’re fairly low risk. There’s a human in the loop. Human in the loop is, maybe the magic word here. If there’s a human in the loop and it’s just proposing information or helping to accelerate something low risk. Those are often internal facing. But when you’re customer facing there’s, higher risks.

[00:52:21] Jason Schlachter: So that’s another thing you wanna consider too. And then part of that is, is doing the AI ethics component. So in all of this there’s a need to consider the implications of the ethical implications of using AI models even in your own business, but especially if they’re affecting customers.

[00:52:41] Jason Schlachter: At Elavance, we were building AI models for healthcare and we were impacting people’s ability to get care with those models. Yep. Our intent was to improve their health outcomes and to make things better. But things can go wrong. And even when they go right there’s always risks that you have to assess.

[00:52:59] Jason Schlachter: And so we would hold these, ethics workshops and the idea here is to, dive deep into what it means to build this. And so I’ll just, I’ll spend a moment on that, Matt. Yeah. But I, think it, it this, happens like really early on it’s not like something you do at the end of your use case pitch when you’ve got your funding and you just need to move forward.

[00:53:24] Jason Schlachter: It’s really like early on in the process of the viability of the idea and the business value. And so there’s a, there’s ethic workshops you can do where you can work with a team of stakeholders and you start off really small low, overhead, an hour or two, get the basics.

[00:53:42] Jason Schlachter: And as you grow your, business case and your, plans and your, funding, then that’s when you start to land more and more layers of this. And this is actually something that, that we do for our customers. We help them to, work through these kind of ethics workshops where you, want a third party that has experience running these and understands how things go wrong to, to run this internally.

[00:54:03] Jason Schlachter: And so you look at your users, you look at your stakeholders, identify all the stakeholders you try to understand There the, values and the interests that the users and stakeholders will have. What kind of tensions might arise? Like how are you gonna test your assumptions? Do you think about the impact you could have changes in behavior that might emerge?

[00:54:27] Jason Schlachter: A great example for me is like cars, like Atlanta, where we live, was built after the invention of the car, primarily because the original Atlanta city was burned down and they rebuilt it really after cars came to be. And at the time the the mayor of Atlanta said I, dream of building a city that is a car, first city.

[00:54:49] Jason Schlachter: And it’s that seems like an anathema today for us, but that was the AI of the time. They wanted to build an ai, first city, a car, first city, right? Yeah. And now Atlanta’s like really difficult to walk in and traffic is bad, congestion’s bad, and we’re slowly peeling back the layers of that a hundred years later.

[00:55:07] Jason Schlachter: So that’s an example of like changes in behavior. If. There was an ethical review committee for the car, first City, like maybe some of those things would’ve come up. So there’s also things like the group interactions that emerge. So how it affect groups. There’s questions around data and privacy explainability.

[00:55:29] Jason Schlachter: So if a model is, impacting your life, like you should be able to understand why it’s making those decisions. We don’t want to take the, distributed bias and distributed failures of our, current sort of like business ventures and centralize them in a way that nobody can question and understand them.

[00:55:47] Jason Schlachter: There’s questions around do you have a human in the loop? How do you monitor performance? How do you mitigate things? How do you get feedback? And so all these kind of things are discussion points like what is fairness? What does it mean to be fair in this use case? This is part of the validation cycle, but you just, you start light an hour or two on the first pass and by the time you’re, funding like a big use case in a big program, like it should be very rigorous. There should be processes in place, accountable stakeholders and all that stuff. 

[00:56:17] Matt Paige: No, that’s awesome.

[00:56:18] Matt Paige: And, great example of something that can be facilitated with AI empowerment group in, Hatworks there. So we got about five minutes. I’m wondering, Jason, we could jump into some of these questions and topics in the chat if you’re up for it, unless there’s something else you wanna cover. That’s it.

[00:56:35] Matt Paige: That’s great. Yeah. Yeah, is there’s, Jacob had one. Does anyone use AI for scheduling appointments? And I don’t specifically know of a tool. I’m sure there’s several folks trying to achieve this, but this is like a, perfect example of a use case that you could disrupt a Calendly or products that exist out there.

[00:56:56] Matt Paige: How could that impact that workflow? I need to schedule appointments, plan out my day. I don’t want to be the person having to reach out to somebody and say, Hey, does this time work? Does that time work? Jason, that was an interesting one. Any thoughts on that? 

[00:57:11] Jason Schlachter: Yeah, I think there, there are use cases like Calendly that, that do that today.

[00:57:16] Jason Schlachter: And I think there’s, other AI startups out there that, that do something similar. But I, guess I would challenge the, notion of what it is that, what is the real task that, that you want done or that I want done? It’s not strictly that like I wanna schedule the meeting with Matt and so I want Calendly to go figure that out for me.

[00:57:38] Jason Schlachter: That’s still that that like process level where I have to get it done. I would love to just have a, more robust agent where, I said Hey, I want to talk to these 10 people this week. Go figure it out. And then Matt gets an email from Calendly saying, Hey Matt, Jason has identified you as somebody who’d like to speak with this week.

[00:57:56] Jason Schlachter: What is your availability? 

[00:57:59] Matt Paige: And what you just did there is you took the question from earlier if I had a team or a staff that could go and do this, how would they solve the problem versus me having to be like the main point of failure bottleneck in the process. That, that’s a great example of how to reframe how you think about a use case.

[00:58:19] Matt Paige: I, like Chris is creating movie scripts about Batman’s early days, which is it’s funny, but like it does change how that whole industry works, potentially. Yeah. From a creator perspective and all of that. We 

[00:58:36] Jason Schlachter: are so for people who are not like deep into stable diffusion or, Dolly, there are models out there right now, generative AI models creating movies.

[00:58:47] Jason Schlachter: And writing the scripts for those movies. And so it’s, emergent. Like I, I believe like in the next year we’re gonna see like TV shows where the script has been written. The Yeah. The actual animations have been completely created by the ai. They may not be successful, I don’t know, but it’s happening.

[00:59:06] Matt Paige: But this is this is one of those big transformational disruptive type of things you think back in the day from music going digital. Yeah. Same kind of thing. Yeah. And there’s gonna be the movies, the studios trying to fight this change of AI and generative AI playing a role.

[00:59:21] Matt Paige: But it has the, feeling of something similar that’s happened not too far in the past. 

[00:59:26] Jason Schlachter: Or what if it’s make, me a commercial that’s gonna cause people to hire AI empowerment group to, to help them with AI strategy create music for it. Like some kind of like amazing techie humanistic background and, write the script and then use my voice to make, to create it.

[00:59:44] Jason Schlachter: And Because it can speak like me. Cause it’s trained on my voice. Like it will just speak for me. Yeah, it’s possible. 

[00:59:51] Matt Paige: Yeah. And Clause brings up an interest. Interesting One. How can businesses leverage the potential of utilizing chat sheet PT to enhance customer interactions, streamline various business processes while ensuring data privacy and compliance?

[01:00:05] Matt Paige: Yeah. Particularly when it involves sending data via the API back to open AI cloud. I think this is an inherent like risk type of aspect. 

[01:00:15] Jason Schlachter: This is a good one. Yeah. So Klaus, you mentioned you’re with a, German company and, the EU is passing measures to, to require that any use of generative AI be approved by committee and be licensed I believe.

[01:00:32] Jason Schlachter: And I think we’re gonna continue to see pushes for that. I don’t necessarily think that we should be. Regulating generative AI or ai at that level? In the broad sense, I think there’s specific use cases that should be regulated. Just we regulate food with the F D A or drugs.

[01:00:51] Jason Schlachter: Certainly in certain domains and where there’s certain need for precision, it should be regulated. But I think for a lot of these startups with low risk, that should be able to get out there and do it. But, in Europe, you’re probably gonna be faced with that, challenge. One way to, to mitigate what you’re asking about is not to send it to OpenAI.

[01:01:08] Jason Schlachter: Run your own models, run them in your own cloud, host it in your building push it to the end user, run it on their client machine. And so in doing so, you’re not necessarily sending their data to open ai. There are open source models that are emergent in generative ai. And some of them are pretty mature.

[01:01:32] Jason Schlachter: Stable diffusion is a great example. It’s first class generative AI model that’s open source. There’s a lot of large language models and chat c b T type capabilities. On the open source side, I’m a firm believer that the open source models will overtake the closed source models given time.

[01:01:52] Jason Schlachter: So yeah, it, you may not 

[01:01:56] Matt Paige: have, there’s even like a there’s a leak document I think from Google. I, believe it was real, but they were cautioning get this exact thing internally that hey, the open the, and it’s funny, they call themselves open ai, it’s not really open per se, but you look at like meta tech taking that strategy and there’s other kind of foundational open source models out there, but they have the potential to overtake things that are being developed internally.

[01:02:23] Matt Paige: Yeah, 

[01:02:23] Jason Schlachter: Meta’s a great example. So open AI originally founded with Elon Musk and, others to, to open source these AI models so they wouldn’t be closed source then strong armed overtaken by Microsoft. Yeah. Now Microsoft owns it. They, make them closed sourced. Meta, has and, Zuckerberg has surprisingly shown up to be like the big open source creator of these models.

[01:02:48] Jason Schlachter: And I think from a business strategy it makes sense. Goo Google’s playing to win, Microsoft’s playing to win. They want to, be the winners in this generative AI race. I don’t think Meta wants to do that or necessarily needs to do that. They’re playing to not lose. If, they raise the water for everybody, then everybody, is okay and nobody loses.

[01:03:11] Jason Schlachter: And I think that’s Meadows play. And, that’s a good strategy against these two giants that are dumping all their money into it. 

[01:03:18] Matt Paige: So the network effects element there too, right? If they’re, At the foundation of it it kind ofra raises their, business. It happened 

[01:03:27] Jason Schlachter: with stability stable diffusion there’s thousands and thousands of, versions of stable diffusion being spun up because it’s open source and Dolly has its trajectory. 

[01:03:37] Matt Paige: Yeah. We are at time. We could go a little bit longer. I just to close it out though, I love the last comment there. It’s heard that these tools are an expansion to your imagination.

[01:03:53] Matt Paige: They totally agree. One of my favorite uses of chat, G B T is telling it to graphically describe any concept, great foundation for any type of media creation. But it’s an interesting concept. It’s like that co-pilot and it’s like a whole nother topic. Yep. Yep. Matt, there’s one. Yeah we can keep going.

[01:04:11] Matt Paige: Let me do just the, call out and then we can stick on for another couple of minutes. But yeah, so like, we mentioned earlier, patchworks and AI empowerment group, we are partnering together. So like these type of custom workshops is the exact type of thing we can take your organization through.

[01:04:28] Matt Paige: Jason, you mentioned the ethics based workshop this is the part where having an expert is critically important and hit up Jason or I and we can help you help get that facilitated. But any other closing thoughts? And then maybe we can jump to a few other chat items.

[01:04:45] Jason Schlachter: Yeah, Matt, totally agree. I love the ideation process, the creative problem solving piece, and I love hearing about the kind of problems that are real and concrete and those kinda opportunities would be a lot of fun and productive for, both of our organizations.

[01:05:00] Jason Schlachter: So hopefully we’ll hear from some of you. I would love to. Pick up this one question from Monica Lapera. Which is the biggest fear for some people is that AI can replace some jobs or even professionals. How do you balance the pros and cons that AI brings to the world? A great question.

[01:05:18] Jason Schlachter: We’re not gonna answer it in the last moment here, but I, think it’s a great question just to surface, because there is immense responsibility. This is really the dawning of, the an age in which how we work and how we live and how wealth gets distributed and who has what is gonna dramatically change.

[01:05:37] Jason Schlachter: And there’s a lot of hype out there. Generative AI is not everything that it’s hyped up to be. And, it’s gonna take a long time for a lot of these things to happen. But the reality is, That we, under, we over predict the short term change, but we under predict the long term change.

[01:05:54] Jason Schlachter: And so this is a, great question of service and I think we just have to really be deliberate in the ethics of all this and try to build the world that we wanna make and not, the world that we can. There’s just 

[01:06:06] Matt Paige: tools I’d say too. It’s do you have the opportunistic mindset or the negative or positive, I’m forgetting the, correct terminology here, but think about 20 years ago, majority, a large portion of jobs that exist today did not exist previously.

[01:06:26] Matt Paige: So a lot of times transformational, disruptive, things like this create new opportunities we don’t even know exist yet. So I think this is like one of those things that has the potential as well. Even though it may be replacing some jobs, I think it’s gonna create a whole host of new ones in the process.

[01:06:43] Jason Schlachter: Absolutely. And a lot of what it’s gonna do is not replace jobs, but replace tasks. So if you’re like a medical claims reviewer, like I’m just taking a wild stab in the dark here. You might not love reviewing medical claims. It might cause it well, and you have some training that makes appropriate for it or, it’s easier than being out on the ER floor all night.

[01:07:06] Jason Schlachter: But you may not love all aspects of medical claims processing. And so this is where I think AI can remove some of the, burdensome tasks that you don’t enjoy so that you can focus on the stuff you do enjoy. So what if you could focus on the really interesting clinical challenges or like the really puzzling situations and not the mundane minutiae of comparing numbers or checking dates or understanding the timelines and stuff like that.

[01:07:32] Jason Schlachter: So I think for a lot of people, for most people it’s gonna, it’s gonna remove the mundane, more automatable tasks but, not their jobs. There certainly will be people whose jobs are lost. But like you said it’s, always changing. 

[01:07:49] Matt Paige: Yeah. It’s like back to jobs to be done communication is the job that’s existed for a very long time.

[01:07:54] Matt Paige: From talking to physical mail to email, to slack and keep going. The job remained the same. It’s just how you did it changed. Yeah. And just the last one, just because Chris is hitting on it how’s it gonna impact the stock market? Anything being done to regulate that? I’d say, I don’t know.

[01:08:15] Matt Paige: I think there’s a lot of stuff already being done today leveraging ai Yeah. In terms of stock trading and that, that’s already prevalent in a lot of ways today, but I don’t know. Any thoughts there to wrap us up with the last kind of q a question? I don’t know. 

[01:08:30] Jason Schlachter: Yeah I would imagine that most stock trading right now is already done by ais.

[01:08:35] Jason Schlachter: So maybe the question is if theis get better, like what does it mean for us? Yeah. Yeah. I don’t know. The, only stage that buy I can give on that is put your money into a index fund and forget about it. You go anything else is a gamble, whether it’s AI driven or not. 

[01:08:54] Matt Paige: That’s right.

[01:08:54] Matt Paige: That’s right. Cool. That was really appreciate you being on Jason. Thank you everybody that came and participated. We will be putting this out there on the podcast and sending out the recording to everybody that joined. And we got a few I think good takeaways in terms of templates and things we can share from this talk as well.

[01:09:15] Matt Paige: But really appreciate the time, Jason and everybody. Have a good rest of your day. Thank you, Matt. 

[01:09:21] Jason Schlachter: Thank you guys for the questions. It’s great to be here. 

[01:09:24] Matt Paige: Thanks everybody. Bye.

Matt Paige: Today we’re chatting with Erica Chestnut, a true champion of quality. She’s been building and leading quality teams for around 15 years now. At places like Cabbage Turner broadcasting calendly realtor.com to name a few, and that’s an awesome list there. By the way, Erica and before that she’s led development team. She was even a developer herself, so I know you’re gonna fit right in with our built Wright community here. But welcome to the show, Erica.

Erika Chestnut: Thanks. Thanks for having me. 

Matt: Yeah, excited to get into this topic. This is one we haven’t gotten into yet on the Built Right podcast, but. Today we’re getting into the often overlooked and sometimes undervalued topic of quality in software development. And the topic of why good process and culture are really at the foundation of good quality. And PS, for everybody listening, stick around. We’re gonna get Erica’s take on her perspective of generative AI and how it’s impacting the discipline of quality. I know everybody’s talking about it, so we want to get Erica’s take on that as well. But to set up the problem, Erica, you talk about organizations who want to improve their quality are often focused on the wrong thing. So what is this wrong thing and what can they do about it? 

Erika: Yeah. Quality is always, not always that’s a poor statement. Oftentimes, when people think about quality, they think about the end state and therefore they think about the thing that happened right before the end state. Which is oftentimes testing. And so when they say our quality is not good, they say our testing is not good, or we are not investing in the right type of testing, i.. manual versus automation, or we don’t have enough coverage. We don’t have enough code coverage, we don’t have enough functional or non-functional testing. But the reality is actually that it starts much further up the stream. And you started to hear about this when we, when the industry was like shift left with testing, but then just like most buzzwords, right? Innovation, innovative, right? Like it, yeah. It’s not unpacked. And so now it’s like shift left testing. Okay what does that genuinely mean? And what is the impact of that? 

Matt: And real quick shift left testing, that’s meaning moving quality further up the value stream towards more than the beginning of the process. Is that right? For just for listeners?

Erika: Yes. But it’s not moving quality up, it’s moving. The it is moving quality up, but it’s really about moving the validation, the checks the awareness. Yeah, what is impacting our product quality? And so one thing that I always love to say is that process creates quality process, results in quality, good process, because process creates consistency and continuity, which results in quality. So when you say moving quality left or further up the chain. People are still thinking testing. Oh, we’re testing the requirements. Are you checking your process? Are you checking your communication flow? Are you checking your documentation? Does everybody have what they need? Are you checking to make sure that the quality team is not starting the new sprint at a deficit? Because the engineers didn’t start stopping before they started finishing, right? Like you, you’ve gotta. Shift the idea of what impacts quality, what creates poor quality. And it’s not just testing. 

Matt: Yeah. And you make a good point cuz quality is at the end for all instance of purposes. That’s the last thing. Let’s check everything, make sure it’s good to go. And a lot of times they can be the scapegoat when something goes wrong or doesn’t get right. Delivered. And I love how you hit on this concept, the process, but to clarify a lot of people think like process, they think like tools, but it’s not about the tools. Think tools are often, over, put on a pedestal in terms of, oh, they’ll fix everything, but it’s not about the tools. It’s the underlying pieces in the process. And I love how you talk about the culture element that comes into play. As well. Yeah, 

Erika: it’s, and there’s definitely tools and it’s testing that’s at the end. It’s the testing, it’s quality is the entire thing. So actually I have to retract my statement. It’s not that it’s not about moving quality. It is it’s. Quality doesn’t need to be moved. Quality is everywhere, right? Yeah. The opportunities to lead with quality are everywhere, so it’s not about moving it left or right or up or down it’s about acknowledging that there are opportunities to improve quality in everything. Is it improve quality in our process? Is it improve quality in the tools that we use and how we leverage them? Are they the right tools? Are they answering the right question? Are they implemented in a way that it’s cohesive, they’re not cohesive, that it integrates into our system in a meaningful and impactful way. All of that is quality. All of that produces quality at the end state. And they all come together like they’re, it’s not just testing. It all comes together to produce quality. 

Matt: Yeah. So to make this more real, I am curious cuz you’ve been in a lot of. Interesting companies from small to large, and I know you do some like side consulting stuff in the past, but what examples do you see? What are those common pitfalls that companies have, whether it be process related or just in quality in general? Is there anything that’s I see this every time, or this is like a big thing that typically happens a lot? 

Erika: One thing that I see this always, it’s a It’s, I think a pain point for not a pain point. It’s, yeah, it’s a pain point for me. Is that Or a pet peeve? That’s actually a better word. There you go. Pet peeve. People do, it’s a pet peeve of mine and I see it all the time. The structure of the business is not clear. And it’s fundamental quality opportunity that is missed. When it’s, The structure of the business is not clear to the teams or the business flow, like the whole, like what is it, what are the boxes that make up the business and how does it flow left to right? What are the, what are the little exits along the way? And what happens is not. Fully unpacked for the team. And then when we go through like hiring companies are going through this massive hiring these windows, and then we’re throwing people in and we’re saying, Hey listen, go to your team. They’ll help you. The team has blinders on, the team has blinders on, and they’re like, this is our little world, but we’re not providing this big picture view. For people to understand at the top level this is our business. This is our structure, this is how we talk about ourselves internally. And this is very clearly how it moves down into the organization. From a structure and from a business business flow. Like the actual product. And so I find that’s, those are missed opportunities oftentimes. And they don’t recognize, leadership doesn’t recognize that, that it’s impacting quality, and I’ll go into teams and I’m talking to teams and they’re like, I don’t know about this. I don’t know how this integrates with this other system. I don’t know. I had one, one manager say, my area, my enterprise, Area doesn’t integrate with this other area, these other, like this main area of our product. It does. It did. And they didn’t know it. So like we put on these blinders and you’re like, Hey, I’ve got my area and I’m good. It’s but are you thinking about how your area integrates with these other areas and what the impact is and do you understand and are you mindful of that? Yeah, that’s, so that’s the thing that I think it’s missed.

Matt: That’s interesting, especially I guess when you get into larger scaled organizations. But it gets back to we talk about a lot about at HatchWorks connecting to the outcome and understanding the outcome and knowing that in all layers of the organization. Yeah. It’s so important cuz you have to understand, what is the business outcome trying to be achieved. But I love your point around the connection between multiple teams and having that, yeah. Having that quality understanding between the different organizations. Quality and innovation. In my mind, I feel like they can sometimes be at odds. Quality is very much process driven rigids not the right term, but you do want like foundational process in how things are structured and then on in the innovation side, a lot of times, whether it’s like business model innovation or anything like that, you’re thinking of breaking process and norms. How do these two play together and how do you create balance between quality and innovation? Those two companies? Yeah. 

Erika: Most companies struggle with that as well. The, that, that balance of quality to innovation because obviously the business is running after innovation. That’s the, they wanna stay ahead in the market. They want to be first to make that next big change. They want to be the unicorn in this space. To do that. Sometimes you’re running fast and you are focused on what’s the new Wizbang feature that you have, but that can be a struggle. It can it, it can be at the expense of quality and if we’re not looking at it, If we don’t pay attention, we’re like, listen just innovate. Get these new products out, get this out. And you might have the teams, you might have the quality teams saying, there’s a problem with our quality. There’s a there’s a problem with our architecture and we’re building these new features on top of it. And so these new features are nice and shiny. But we’re putting them on top of something that doesn’t smell so great. And eventually the new shiny thing will wilt and it will also smell right, because we are not we’re not considering that we never fixed the actual problem. We never cleaned up. The smelly stuff, right? Yes. And that’s part of quality, but it’s not just bugs. It’s what might create delivery problems. What might create inefficiencies? What, what doesn’t allow us to roll back quickly if we have problems? How long does a problem linger out there? How many open issues do we have? Just. Even just meeting acceptance criteria. The turn of how frequently, how long it takes to get something delivered, and then we’re making. We’re taking shortcuts because the requirements weren’t 100% clear. And so we had to go talk to product a lot. And then we went back and forth and we made changes. And then all of a sudden, something that we developed two weeks ago that was actually pretty well baked has now been hacked at the very end, and were released out there. And then there’s an edge case that we didn’t know about, but it’s like an extreme edge case, right? So it’s that innovation like when you think, are we innovating too quickly? Over quality. What is the impact of qual impact of our innovation quality? Did we release these new features that we, this new functionality, there’s new innovation, and do we see a high level of defects? Did do, did our defect density increase? Did our CSAC scores go down because our customers are like, Ew, this is broken. You told me about this new hotness and now I’m coming here and it’s just broken. That sucks. I don’t wanna use your product anymore. I don’t wanna tell somebody else about your product. Right? We have to balance that, but it’s oftentimes a struggle. 

Matt: It’s almost like quality in a lot of ways is the enabler for innovation. If you don’t have that foundation set, yes. It makes innovation that much more difficult, to actually. Really do that. And I love the, I got a visual in my head. I have a, a one year old baby. So when you mentioned the didn’t smell so great. That’s bringing up some bad memories from last night. Things were thrown away. I don’t want to get into it. But the you hit on some other things though. In our business, and we’ve done kind of a foundational shift as of late, really focusing on what are our leading indicators. Versus our lagging indicators, like what are some good leading indicators in quality? And you mentioned, I think like some time to resolution and things like that. What are you looking at whether, lagging or leading that are indicative of either, things are going good or maybe I need to like hone in a certain area.

Erika: Yeah. That, like that defect density, right? What’s, what is our release health look like and do we see a lot of releases that are going out and we’re seeing our health dip? Are we seeing a lot of releases returned? A lot of rollbacks, reverts incidents. What’s our mean time between failures in production, right? These are all alarms. These are red flags that we can look at and say, maybe we’re innovating too quickly. Maybe. Maybe we need to slow down what’s causing this. Maybe we need to look like we’re our requirements not fully baked. Were our acceptance criteria not clear? Where was the failure? Did you know? Did we push in something really late that increased defects? Did we not? Is it, how, what type of failure is happening? Is it a backend failure? Is it a load capacity issue? These are all things that, like, when we begin to unpack that and we say, hold on, we’re seeing an increase here. Let’s look at it and understand what the problem is so that we can target it, fix it, and then go fast again. But often times they don’t. That’s such a foundational. 

Matt: Yeah. Yeah. That’s a foundational piece is knowing what those metrics are. So you have your, Dash dashboard, for lack of a better term of your indicators. Yeah. And then when something’s off, you know where to dig into. And I heard you mentioned the the defect density. Is that just like volume of defects or is it hitting on something 

Erika: more specifically? Yeah it’s volume of defects. So let’s say that, we’ve identified. 200 defects in production. And please don’t start me math because math is hard. So we’ve identified 200 productions. 

Matt: No it’s Friday for us. We’re not getting into math.

Erika: It’s Friday and it’s been a Friday. Yeah. Good. Not in a Margaritaville kind of way, although maybe it needs to be very soon. 

Matt: Yeah. That’s next. 

Erika: Yeah. But Like the number of defects. And then let’s say that we, we, we have a trend. We see that we have, maybe some spikes here and there. But we start to recognize that those spikes are happening every time we release to production. That means we’re recognizing that we are introducing in every release a spike of issues that then we’re having to work back down. How do we improve that spike? Is there a correlation? And then what is causing that? What are we missing? Do we not have enough automation regression? Are these regression issues? Do you know? Do we not understand our system well enough that we understand the impact of the changes that we’re making on downstream areas of the system? What is creating that spike, which is costly? Because especially if it’s like a critical area of the system generates an incident, you’ve got no less than 10 people jumping into that conversation. You’ve got the eye of the cto, the eye of the cmo, so you’ve got executive leaders and you’ve got senior leadership. This gets really expensive and they’re just looking at it and waiting. And they’re jumping in, they’re engaging in the conversation. And then you’ve got. The management, the middle management layer, and then you’ve got the ICS that are implementing potentially the ch, you just got a lot of people in that. It’s costly. And so it’s Hey, we’re seeing, we release and we spike, and then we spend on top of the innovation time to get those spikes back down, or we’re leaving them out there and the customers begin to deal with death by a thousand cuts because, oh, it wasn’t a big issue. But there are a thousand of them that everywhere that comes like gnats, right? And you’re like, yeah. Like walking into a 5,000 bugs. 

Matt: Yeah. I went to school down in, in south Georgia, in Georgia Southern. So I’m used to the gnats. You can probably sympathize being Atlanta East in. 

Erika: Can I get around you? 

Matt: Yes. That’s a good point though. You talk about, it’s like how do you help the business recognize the value and the impact? Of process and quality. Yeah. And it’s almost like it, it’s before it’s too late, I think is the key thing. It’s like, how do you help them recognize that value? What is, where have you found success or what are some good things to hone in on to help connect it to business value before it is too late? And then you got, yeah, all the C-suite breathing down your neck, like you mentioned as a scenario nobody wants. 

Erika: It’s telling that it’s identifying the story of quality within your organization. So hearing from, I love to, like, when I come into an organization, I really want to hear I hold what I call my what? The bug meetings, and I’m meeting with different people. I’m asking them some similar questions depending on where they are level wise. Some are a little bit, more detailed conver questions, some are more strategic but they’re still in the same vein. And then I’m looking for those categories. I’m looking for the sentiment and the conversation. I’m looking for the themes to surface to help understand where are the problems. Because the thing about telling a story is you want it to be compelling. You want it to be interesting. We’ve all picked up a book before and gotten, maybe a chapter or two in, or watched a new series and got into the second. Second half, halfway through the second episode and was like, this just isn’t my jam. The story of quality is no different. You have to tell a compelling story. You have to explain it in a way that attaches and connects to the business heart, to what the, what leadership is interested in. To the value of the business, which is the customer, which is our revenue. You’ve gotta connect it into that conversation, and that takes time. That that, yeah, that requires a lot of moving parts and pieces, but when you understand. The sentiments. When you get that feedback, you’re at least able to say, Ooh you’re worried about availability, or, ooh, you’re worried about, SEO tracking, or, you wanna understand our customer sentiment. Okay how can I get that and surface that information? How can I make that visible through the lens of quality and say, Hey, listen, we’re tracking this. And we wanna hold the teams accountable to it and start to drive that conversation. So you’re taking the heart of what the business is interested in, and you’re moving it through the lens of quality and pushing it back to the teams to say, this is something that we need to look at. How are you improve? How are you helping to improve this? 

Matt: You’re like a quality marketer. You, it’s, I promise that’s one of the reasons you’ve been so successful in your career is being able to connect that story. That’s so cool. I love that. 

Erika: It is not an easy thing to do. I don’t tell you. No, but it’s interesting. It’s not, it’s interesting, but people don’t always think, the thing, the interesting thing though is the frustrating thing is that it’s not all explaining that I have to go through that. I can’t it’s not. Common. It’s not a common expectation. And so yeah, I’m like wandering around sometimes thinking, what data do you have? What? What is the data and people like, but why? Like getting this data is hard. And I was like, I know. And I don’t really have a why for you yet. I’m actually just trying to see what you are tracking. What do you think quality is and what do you measure? Because now I wanna pull it together into a single, cohesive conversation and be like, now when we look at this across the board. Hey, we have a problem right here. Should we focus in on that?

Matt: Yeah. That’s how you connect the dot. That’s right. And one thing you mentioned earlier, you talk about acceptance criteria. I’m curious your perspective on this when should quality members on the team, whether it’s a QA engineer or whatever role it may be, when should they be engaged in understanding the user stories, requirements, or whatever it may be?

Erika: At the very beginning with everybody else. Here’s the thing. Yeah. With quality team members have the benefit of constantly exercising the entire system. If somebody knows the ins and outs of your house and you have a problem, or you want to make an addition to your house, wouldn’t you call them first? Yeah, somebody who’s constantly, I’m thinking we just had a problem with our AC and we call somebody that this, the. It was the same guy that came out and fixed the AC problem we had last time. I’m not gonna talk about the shadiness that feels like, but he clearly said, he was like, yeah, he’s talking to my husband and he’s saying, cuz I wasn’t out there, but he was talking to my husband and he was like, yeah, this is what we talked about last time. Here’s this, that, and so forth and so on. Like he knew the problem. Which made getting to the resolution or understanding the, like just that knowledge push in made it so much quicker to get to the resolution and therefore cost us less money because he is out here less time, right? Yep. That’s qa. QA is constantly exercising your system from the customer perspective is, which is who we care about. QA is connected to the heart of the customer inside the business. 

Matt: Yeah, it’s the health of it. And I want everybody to like pause for a second just so you don’t miss this point. If you’re a scrum master product person or whatever it is, bring your QA folks into these ceremonies early on. Yeah. Because to your point they can save you. A lot of times they’re gonna be thinking about something from a different angle that you may not thinking about one, and they’re gonna be given additional context when they actually are doing the testing, which is gonna make their job. A lot easier. So anybody that’s not doing that br bring your QA friends into those conversations. Yes. Earlier 

Erika: on and I will I will point out like one thing that I often have had to do when I’m going into new orgs, when I have a new team, I have to coach inside of my team because the QA folks can be wallflowers at times, some of them can be wallflowers, and so they will come into a conversation and they’re like, yep, I’m listening. I’m actively listening. And that sounds odd, but okay. They know what they’re talking about and so I’m just gonna wait for it to come to my desk. And I have the context, but the QA organization, It’s one of the things I love talking to the QA community about. It’s like we are more than just testing in that single step in the delivery life cycle. We provide that value. We need to speak up, we need to provide the, here’s a gotcha, have you considered this? Have you turned the box in this way? And when you when, when team members, when were brought into those conversations, ask for that, pull on them, get the, request that feedback like, Hey, What do you think? Like I’m, these, this is these are the boxes. I love to talk about things in, in the form of boxes. So this is the box of the flow. These are the boxes of the flow. Currently, we wanna shove one right there. What do you think about that? Like what’s gonna happen? How does that help or harm the journey that you experience and go through and think about from a customer perspective? Is that good? Is that bad? Ask those very specific questions to, specifically to the QA team, to, to draw them out and get the, that insight. 

Matt: And that’s a facilitator like tip there, right? If you’re a Scrum master product person. Yeah. Like one thing that we do in a lot of workshops is we’ll always ask around the group, Hey Lisa, do you have any clarifying questions or anything like that? Bob, do you, and you go around the full room. And it’s funny a lot of times why people say no, but, and then they’ll go into what’s on their mind. So that’s a good tactic to get those, like you mentioned, wallflowers, to speak up. Cause they, they do have an opinion and it’s a valuable one a lot of the times. All right, so the hot topic right now, everybody’s talking about it. Everybody and their mom, generative ai. Now we’re playing around with GitHub co-pilot and some other tools at Hatch Works. But I’m curious, what is your perspective, thoughts, theory, whatever it may be? The prediction on how generative AI will impact the quality assurance discipline Positively, negatively, how it evolves. What’s your hot take? 

Erika: It’s significant. It’s significant. The thing to remember with all of the technologies, these are tools in our toolbox, yeah. I’ve heard the conversations, people in and out of QAO, the end of testers, the end of all of these things. But AI’s been building in the quality space for years now, for years. ChatGPT. I loved some chatGPT, right? Just being able to ask questions. It is another way to turn the box. It’s another way to leverage a tool to help us better communicate, to help us quickly write scripts, but just in general, like this generative ai, like the conversation around it, automated routine testing. It’s like it’s just generate generative AI can create new test cases that mimic the variety of user behavior and edge cases. Let it do it right? Yeah. We still, the humans still need to be in the conversation because we still need to analyze that. We need to, AI can handle those routine tasks, but we are analyzing it as humans. But it, it changes our role. And that’s the thing, like it doesn’t go away. It changes our role so that we can, it could be more cognitive. We can literally sit with something and think about it. As opposed to this is mundane, this is redundant. You know what people have said years ago, you’re just banging on testing, is just banging on a keyboard, which is not, it has never been. It is not, yeah. But it gets us even further away from that idea because now we’re like, let the machine take the inputs and generate something, and then let us tweak it to be more informed, more intuitive, more human. Let it let us use the machine to do predictive analysis, analyzing historical data to predict potential problem areas. Let it enhance performance testing or increasing QA accuracy, unbiased unbiased testing. This is a big one, so go further into that. So the story, right when the Apple Watch came out. Eventually became, one of the stories was like women are the biggest users of it. I don’t have data points. It’s been so many years. Women are the biggest users of this, but it does not have it does not have period monitoring on it. But yet women are the biggest users. It was a miss, right? Correct. Cause women were not included in that product team. They were not included in the usability testing. Like the, this was a miss, a big miss. And when it was added, like women were like, hallelujah. Thanks. But we have these bias, especially, it’s like you talk about like people you know in the DE D E N I space, and when you think about accessibility, I have bias. There’s, we all have them. I don’t. I don’t know what it feels like or what to consider directly when it comes to screen reading, not being able to read the screen. I don’t know, like what is better? What is a better experience, but that could be programmed into ai. Yeah. And there’s un like having unbiased testing supported with AI and then being able to be a lead, taking that information from like leaders in the space who understand it and plug those in as, excuse me, plug those in as models that that AI can use. So there’s so much opportunity to make it. To leverage this tool to create more efficiency, to create more impact, to be more valuable in the organization. But we’ve gotta, we can’t be scared of it. We can’t be scared. We need to look at it and be like, listen, you are mine and I am going to I know that you’re a hammer and there is a nail. I am not going to use you, to do these other things, but I’m gonna use you. Nail everything in cuz I know how you work. 

Matt: This is great and I love this. You have the eternal optimist mindset versus the pessimistic, it’s gonna take everybody’s job. And I love that cuz it’s an enablement view of, it gets me outta like the mundane, like stuff I don’t want to be doing. And it uplevels us as humans. It’s that, and that’s why I love how it’s positioned as, people talk about it as a copilot, we’re still in charge. Yeah. But it, but it’s helping enhance what we’re doing. Yeah. Really exciting stuff. I love where this is going and I love that you’re testing it and playing around with the tool versus waiting. Cause I think that’s where so many people miss, is once it becomes mainstream, then it’s like too late. And you’re like trying to play catch up mode, right? 

Erika: Yeah. Transparently, listen, I, when it first came out I was playing around with it and I was like, okay, here’s a requirement. Write a test case or tell me what the acceptance criteria for this requirement is. And it two seconds rattled some stuff off, and I was like, those are decent. All right, tell me what pesky now use this and tell me what the test cases are for this in it. Two seconds later, rattled off some pretty decent test cases. You know what? And I say that decent test cases with it. Not being informed, especially before it was had access to the internet, was it really not being informed? And just going off of if we’re talking about this thing, right? If you’ve told me this is the requirement, and giving it enough information to be informed enough so it doesn’t just say you’re gonna have to log into the system, so do that, right? Yeah. But instead if you’re, if you’re doing this functionality, here are some things that you’d want to connect. And then really deep diving in and saying what are some non-functional versus functional? What is security? What type of performance testing? How would I test these APIs? What type of data should I use? Where should I, what are some considerations? And just continuing the conversation. That was fun. Yeah. But it was scary at first cause I was like, oh yeah. Snapple. 

Matt: Yeah. It’s, and it, like you said it’s wow, this is decent. But what, connecting back to a point you made earlier where you had the example of somebody kind of being blinders on focus and suggest their organization, they didn’t think about how they were impacting others like this. This could be a use case right here where generative AI and the tools we’re using do have that purview across the entire organization to say, Hey, Are you considering this? Yeah. That maybe outside of your discipline, so like that’s an interesting kind of use case for this as it starts to evolve. I, I think it’s really exciting where 

Erika: to go. I want us to get into the point where we’re able to feed it. Privately feed it information and say, okay, now that you understand this ecosystem, Now that you understand our structure, our business flow, our business model, right? Now that you understand that, what should we innovate on? Yeah. What are the concerns with our product? Like now you’ve analyzed our tests and how our tests are performing. Should we innovate or should we fix tech debt? And what’s the impact, what’s the financial impact? AI can start to answer all of those questions just at a, just as a food keys, strokes. That is so exciting. Being able to like unpack, and I’m not saying to get to that point is significant. I get that yeah, what is the data that we feed it? How do we feed it that data? How do we protect privacy and da security, all of that stuff. I get that. Yeah. But man, Jetson’s opportunity there.

Matt: I always think back to the beginning of, cell phones, where they were to where they are today. Yeah. Nobody could have imagined where we are today, where, like, where the internet’s gone. I think it’s gonna be the same thing with generative AI in a lot of ways. So it’s gonna be fun to watch.

Erika: That’s to the iPhone, right? That’s what of AI is, it’s like that point and then, All products. Now all phones follow that same view. Every phone is that, smartphone view based off of what Apple did. Nobody has a razor flip phone. Some dudes still have, I remember the Verizon little like brick thing that slid up and stuff like that. That was the cool thing. Yeah, not more right? Like everybody, a sidekick. Yep, that’s right. Yeah. Generative ai. That’s where we are at right now and it’s. I cannot wait. 

Matt: That’s awesome. All right, so let’s do a couple quick rapid fire questions to wrap it up. Okay. First thing that comes to mind. What company is doing qa, right? Which is there somebody in the community that you’re like, oh they’re really good at it. 

Erika: I, that’s not a, that’s not a fair question. It’s it’s subjective, right? Everybody’s doing something right. Can I plea the fifth? Everybody’s doing something right? Yes. Everybody has opportunity. Yeah. I worked at Calendly. I’m gonna say Calendly doing it right? Yeah. 

Matt: Shout out Calendly though later. 

Erika: Yeah. Yeah. Teams are looking to improve. There’s, there’s a lot of great things that Realtor is doing. There’s still opportunity. There’s opportunity At Calendly, there was opportunity at cabbage. It’s just about the focus and so yeah. I’m gonna complete the fifth. 

Matt: No those are good answers. What about an individual? Is there anybody in the QA community that you follow or, Think is influential. 

Erika: Angie Jones is amazing. Lisa Crispin and Janet Gregory are the agile queens. Like just, the, they have the Bible. Three of them actually on agile testing and processes. Come top of mind for me. 

Matt: For sure. Like those, like  it, it send their LinkedIn to, we’ll put ’em in the show notes for some folks who may be interested to start following them. And what’s one thing that you wish you could go back to like your former self and give some advice to your former self if you could go back?

Erika: It’s usually not about, it’s not about quality. 

Matt: It doesn’t, has to be, it’d be anything.

Erika: Own what you know. Don’t worry about what you don’t own what you know, because what you know is impactful. It’s important and it’s valuable. And when you spend time worrying about what you don’t celebrate and champion and communicate to others what you are excellent at, and therefore you don’t continue to hone it. It’s okay to know that you have what the gaps are. If you want to work towards filling them, but some gaps I don’t want to learn how to surf and that’s okay. Yeah. I’m not a surfer and I don’t want to learn how to surf. I like to swim and I want to learn how to become a better swimmer Still in water, right? Yeah. So it’s Hey, what are you excellent at? And what are your, what do your passions lie? So own what you know and lean into that and don’t worry about. Don’t worry about. 

Matt: I love that. And one thing you mentioned earlier, and just to wrap it up yeah. One thing I love about your experience, what you do is your involvement in kind of women in tech and the diversity and inclusion space. Anything to speak about the, I see your kind of involved with the women in tech and career coaching there. Yeah. Anything that you are either excited about within this space or how you’re helping folks in this area.

Erika: Yeah, I, as a woman in tech myself I’ve spent the better part of my career being the only woman in the room, especially as a leader, being the only woman in the room also being the only black person in the room and that can be difficult. It has been difficult and I’ve had to learn how to manage my my own imposter monster. I’ve had to learn how to manage my voice and showing up the way that is right for me and not worrying so much about what others, how others think I should show up. I had somebody tell me I should be more docile and quiet because certain genders should be docile and quiet, that I should modulate my tone. And so I’m passionate about coaching women especially. Because I spent a lot of my career not being confident about who I was and how I showed up and second guessing and not speaking up when I should have spoke up or not owning what I knew, and so I’m excited about that and I love to talk to women about that in, in the space and help them.

Matt: Yeah. And then the community element’s so important too, I think. Having that community of folks that are going through, the same thing, they can trade stories and I got two young daughters at home, so I appreciate you pioneering the way for women in tech as they come up. You’re an awesome role model there. Yes. But where just to wrap it up, where can people find you, whether it be LinkedIn or what you’re doing? Anything you wanna. Plug here at the end. 

Erika: Ah, yeah. I’m Erica Chestnut on LinkedIn. Please feel free to reach out. I love to talk about quality. I’m a bit of a dork about it and obviously I love to talk about women in tech in general. But you can also reach out to me on ericachestnut.com. That’s where you’ll learn a little bit about my leadership consulting and my. Women in tech coaching and my quality leadership consulting and coaching all things I love to do. I’m really passionate about coaching and supporting people, either women in tech or in the coaching, or excuse me, in the quality sphere. Feel free to reach out. I’m around. 

Matt: Awesome. Thanks Erica. Appreciate the conversation. Thanks for joining Built Right. 

Erika: Thanks.

Matt Paige: Today we’re chatting with my friend Andy Mowat, CEO of Gated, and we’re gonna go deep into the land of product strategy today through the lens of Andy’s experience in building gated, as well as some past experience at big name companies that you all will recognize. But before we jump in, Andy would love for you to provide a brief intro. You know who you are, what do you do, some of your past experience to 

Andy Mowat: kick. Yeah, absolutely Matt. Great to see. I historically have scaled companies up to, large scales. I’ve taken Upwork. Box and most recently CultureAmp to unicorn status. Periodically I start something from scratch and I’m doing that right now. I’m running gated gated. The original product takes unknown email outta your inbox and challenges people to think Hey, is your attention worthwhile? And as we talked about we’re launching in a couple weeks, which probably is launched by the time we talk here our new right platform, which is helping people. Take control of their attention on all platforms. We’ll dive into that.

Matt: Yeah, so excited to get into this and there’s some nice kind of meaty pivots along the way throughout the journey, kinda leading to this new gated 2.0 here. But I wanna start with, a meaty kind of meta question here. And how would you define strategy? This is such a, I think, a nebulous question sometimes, and feel free to jump into examples from past experience, but how, in your mind, what is strategy? 

Andy: I think it’s being as clearly focused on a single problem or opportunity as possible. For us, when we’re building a tool that anybody can use. It’s understanding user behavior really well and then figuring out how to do that. It’s, and this kind of goes counter to my approach, but our ctl Allen’s very passionate about it, which is it’s understanding what’s the problem you’re trying to solve versus Yeah. The features, and then, yeah, figuring out how do you deliver the right solution to that problem.

Matt: Yeah, it’s perfect. It’s that element of choice, right? And making a decision and being deliberate. About where you’re going. And I, that’s missed so many times. I think in strategy. That’s what I love about, gated, you’re very focused in who you serve and which what you’re doing there. Another element though that I love about gated is this element of the manifesto. Yeah. And your view of the market. And it kind of encapsulates your strategy in a sense, in this kind of narrative format. But with, I would love to, you. Give your overview, your take on leveraging something like that to help convey your strategy. 

Andy: Yeah, I think we spent a couple months iterating on the manifesto. If people haven’t seen.com/manifesto, I think it helps people understand what the company’s trying to accomplish. And I think, you probably, I don’t know when you encounter a manifesto versus when you first learned about gated. I think if you’re like, okay, it’s just a tool you’re gonna do I need the tool or not? But I think a lot of people have also really gravitated towards like the change we’re trying to bring in the world, which is like communications less noisy, more personal and all of those things. For me, I think with the original manifesto came from the concept of not everyone can use our original email product, but everyone should believe in the mission we’re trying to accomplish. I think as we launched this new platform, literally anyone can use it, which is fun. And so we’re rethinking the manifesto, but it’s just small tweaks. But, it doesn’t fundamentally change, but it guides the decisions we make on the product. And so we take it very seriously and it’s not a, here’s three bullet points on a webpage. It’s it’s an interactive kind of flowing manifest.

Matt: That’s what I love about it. It puts your stance, your viewpoint in the market. And you mentioned like how I found out about I’m a great, a user gated now. I love it. I remember I saw y’all put something out there around noise canceling headphones for your email and that kind of just clicked for me. It’s actually how we first engaged and interacted. But it just clicked, which is awesome. But I want to get into gated, and this is a really interesting. Where y’all first started and where you’re moving towards. So we’d love, maybe start there with the where did gated originate? Where did you first come up with this idea? And then we can get into where it’s going, which I think is really the exciting spot. You’re l you’re learning from customer behavior as you go and talking to customers and iterating on the product. 

Andy: Yeah, absolutely. I’ve sent billions of emails. I’ve caused a lot of pain. Probably a lot of your listeners. Sitting there saying Godammit, all those emails I think I’ve sent push send on like 8 billion emails or my teams hat, right? And so we’re all guilty, right? I know all the hacks. I was sitting there at a series E company and I was just getting blown up. I wake up every morning and I was the buyer for a lot of tools and People would ping me and I was just like, God damnit. Like this stuff’s irrelevant most of the time. And so I wrote an email and said, I don’t know you. Here’s my Venmo. If you wanna Venmo me 10 cents, I’ll pay attention to it. And then I threw on my nonprofit. It was like, Hey, if you want to donate to my nonprofit, but people started donating, like they started donating like 10 bucks, 20 bucks off of 10 cent As, I’m like, this is interesting. I gave it to a lot of friends and they’re like, man, this changed my email inbox. So that’s the original product. I think what we’ve learned in the, so we’ve got tens of thousands of users that are using it and passionate. Yeah. But I think we’ve also, 

Matt: and I wanna pause there real quick, Andy, just on that point, super interesting. Just for the audience. Like the key thing I get from that is you noticed a pain point in the market in verse going into a hole and, trying to build something out for months on end. It’s like the best example of a… You put your Venmo in his email, 

Andy: kinda iterate there and Yeah, exactly. Yeah. So I built the first version in Air Table and, actually I built the first version, I just sent my demo in. Then I was, yeah, exactly. Then I built it in Air Table in Zer. Then I like hired a young kid to code it up and then I finally was like, okay, I need to get some actual developers. Yeah, and, it’s, and so I, it was more just a side hobby for a while until people be like, Hey can you so yeah, that’s, that was it was a lot of fun, but I wasn’t looking at it as what’s the market problem? I’m more looking at what’s bugging the crap outta me? 

Matt: Yeah. I love, it’s such a good point. It’s solve your own problems is such a good starting point. And you had this hypothesis of, okay, maybe this could be a mechanism to do that. And you did it in an early kind of quick, iterative way. It’s such a great, just lesson for folks out there building digital solutions. Start there, start small, start to it. But yeah. So you’re at this point, you’re starting to get some traction with this initial kind of idea, iteration of gated. So what comes next from there? 

Andy: Yeah, so we’ve, as I said, we’ve got like tens of thousands of users using it, loving it. But I think we, our vision is how do we change the world of communication? And I think the insight we’ve had in your user of the original product is there are people that can afford to just turn off the noise and. But most of us can’t. Most of us need to live in that noise, see that opportunity. And so our new platform is focused on helping people surface the right conversations out of the noise. So our fundamental thesis is this, on the new platform it’s hard to filter email after it’s been set. And everyone lines up and they just, and I’m talking not just email, but email, LinkedIn, dm, slack messages. And the problem is everyone can reach you on all these places. They, some places are like, oh, you’ve gotta be connected to me to reach people. But we all get the LinkedIn invite, it’s random that we don’t know and we have to deal with this. So our thesis is instead of letting people just blow you, Let’s guide people to engage around the topics that you care about. Yeah, so we believe you should be as available as possible for the things you want to talk about. And for the rest of this stuff, no. And so we are building a universal link. So you can get gated.com/. If it’s not taken, if it is, we can talk about this call. I gotta get that one early. Yeah. If it’s not, I’ll give it to you and if not we can get you page or something like that. There you go. And so you get that and you can, we can help you articulate what you’re focused on, keep it updated, and then let people that want to communicate with you around those things, regardless of whether you’re linked into them or connected to them or they have your email or anything, be as available as possible for the right opportu. Yeah. And the rest of this stuff, people gotta go figure out a different way to reach you. And so that’s what we’re building and that’s what data is all about going forward. Yeah. It’s such 

Matt: a real pain point out there in the market. A lot of ways we like to think about it, the, person that’s brought this out into the world, the Marty Kagan thinking around valuable, viable, and feasible. You’re hitting that valuable point head on. Cuz I know everybody listening has experienced this, especially our product in engineering. Folks who get inundated with emails, they could care less about. One interesting thing though, this is not something that’s just commonplace. You could even argue this is more kind of category creation lens, but what you’re combating, I feel in a lot of ways is just the status quo. They’ve always done it this way, and you’re looking to break a habit, which is difficult. I think it’s probably one of the. Under considered things, when you’re building a solution that’s driving a change in habit, how do you actually go about driving that change and getting people to adopt it when it’s something maybe a little more foreign or unknown to them? 

Andy: Yeah, I think you gotta, you gotta, we talk about creating the cultural moment. So when we’re launching, whether this podcast drops before or after that you. Wake people up and be like, whoa, that’s interesting. That’s different, right? With the original email product, people are like, wait a second I’m getting this email when I sent you an email because I don’t know you. It’s forcing me to think, I think with this new one, we won’t be like interrupting the flow of communications more way for the good stuff to rise above it. We’re launching with, tons of leading people that are sitting there saying, like, when you ask me Matt Andy, how can I help? I’m not doing a good job of articulating that right now. So we, if we can put that on everyone’s LinkedIn profiles, put it in people’s emails, we can start to, people be like, Hey, like what is this thing? And I see all of the other people like Matt articulating really well now, what he’s interested in and how I can help him. Like how do I get that? So yeah, it’s gonna be interesting, like we’re pushing the bounds of communication, but I think rather than there’s a better world out there, right? Which, I show up and I see you and I figure out like email, LinkedIn, dm, slack, whatever, and I’m like, here’s what I wanna send. Or even worse. Here’s what the AI tool that I use decided that it wants to send. Yeah. Copy and paste that. And so I think it’s gonna be a lot of fun. Like we’re taking a big swing at communication and we think that there is a better way, which is let’s be, it comes down to the in, I don’t know if you’ve seen it, but like the concept of. How do you want to engage? How do I want you to engage with me? Yeah. And we’re helping our users articulate that, right? I always love, I’ve written my own user manual. I’m like, how do I work? I love that. And there’s, it’s like when you join a company, like here’s our norms, here’s how we work, here’s how we wanna engage. And so I think that concept of let’s just not. Punching bags for somebody else, but let’s have our own gate. That helps people understand how we want to engage and on what topics I think becomes interesting. And so hopefully people listen to this get excited about that vision for a better world of communication and go get your gated profile. It’s it’s free. I think over time people are like how are you gonna make money in, on the original product we charged the sender. Here. I think we see, if we can change communication the right way and create the value. There’s lots of fun ways to like power features for power users type of thing.

Matt: Yeah, and it’s interesting, it’s not, maybe the traditional two-sided marketplace type of solution, but providing those kind of rules of engagement doesn’t just serve you as the person that people are trying to reach out to. It also serves the person trying to do the outreach, think of how more, optimized, I guess their outreach can be. If they know who they’re reaching out to and understanding how they want to be engaged, what they’re interested in. So you’re serving both sides of the market. You kinda have your core user over here. That person performing the outreach is very much probably in your consideration set, I’m sure as you’re building out gated.

Andy: Yeah. Yeah. And I think it’s, with the original product it was all about like how you stop people selling you things Here. I think it’s like, how do you actually get the people. They’re maybe in your peripheral tangential network to know how they can help you, right? There are probably people you’re trying to meet every day, right? Like for me, I look at it as I, in addition to like my core day job, like I love helping like high like revenue leaders that are trying to competition things. And if I, if there’s a half an hour call, like I love helping you help vice versa. Like how do we help each other? And so I think there’s a lot of those conversations we’re all trying to have that we haven’t articulated well. And you and I haven’t chatted in a little while and I don’t even know what are the types of people you’d want to meet and all that stuff. So I think there’s, yeah, it’s the sales people, which is the original product. But we also like forcing sales people to pay, like only creates so much value, right? Like they’re still trying to save. Yeah. Yeah. But helping people understand the topics you want to connect on is a much more powerful plan. 

Matt: Yeah, that’s interesting. So it’s deeper than just this transactional. Type of piece. It’s like, how are meeting and growing and working and networking with other people. It’s more than just the transactional level. Yeah. I love that. 

Andy: I, when I go on LinkedIn I know who Matt is. Yeah. I know what Matt’s done, but I don’t know what Matt wants to connect on and talk about. 

Matt: Yeah. No, that’s perfect. One piece that’s really interesting, this ties back to strategy and good strategy, and I heard you mention. There’s been this change in the market very recently. This, rise of generative ai. And it’s just changing things in terms of making it even that much more easy for people to spam, reach out to folks and just go crazy with it. I think that’s an element of good strategies when you. Tie what you’re trying to do to a change in the market, an inflection point in the market, and I think that’s an interesting piece. Maybe speak on that some. Just the element of strategy and being able to leverage market trends and things 

Andy: going on. Yeah. I see three major trends that we’re thinking about. Yeah. One is the proliferation channels, right? Hey. You need to join this Slack group this Discord channel. You gotta be on TikTok now or whatever. It’s, yeah. And so you’ve gotta to be able to be successful, like you keep getting pulled to more channels. Every single channel is dying, right? Like it’s, aI will kill every single channel. Over time. If you’re not overwhelmed by LinkedIn, you soon will be. Yeah. And then the final one that’s really been the most nuanced for us is the barriers to reach. Are being shattered, right? So every morning you wake up, you get like a couple to, maybe for me it’s 10 or 15, like LinkedIn invites from people. I don’t know. And I gotta decide, do I want to accept them and have them live in my dms forever or do I wanna ignore them? And it’s like that whole concept that’s a really interesting one we spend a lot of time thinking about, which is, it doesn’t feel like the right way to decide, do I wanna have a conversation? I haven’t decided if I wanna let you in forever. And so I think LinkedIn’s there’s an interesting trend where it’s become more of a social graph than like actually people, and yeah, that’s, that is it’s hurt them, but it’s just changed the world. So we spend a lot of time thinking about what are those trends? And then how do we take it, how do we position our plots in the right way? 

Matt: Yeah the other piece I heard you touched on too, this focus of building the experience and a product user’s love versus the focus on mod monetization. What’s your thoughts on that, especially early on in terms of trying to get scale and all of that? You wanna create a viable business that’s gotta be part of the roadmap in some way. But what’s your take. 

Andy: Yeah, I’ve learned a lot on that one. I think we gotta, 

Matt: I bet with Upwork and Box and all those I’m sure. Yeah. Past experience. 

Andy: I think it’s interesting. Like we’ve, we went, we have a very interesting model with our original product, right? Which is users get it for free. We take a percentage of the sender donations and fine for that. I think, that is, it’s a very innovative revenue model. And it. As we move to the new one, like that model doesn’t apply as much. And so what I look at is, I guess I’m in it to change the world. And I think if you can change the world, there are a lot of interesting ways you can make money on it. At the same time, like I’m conscious, there are products like Loom, right? Like it’s my favorite example of I freaking loved them. It was a hundred million people used the damn thing. And then. They got their pricing model and they were like, okay, now you’re using an infant, and then we gotta roll that back. Yeah. And I think a lot of people were pissed off and looking for alternative solutions. And so I think the way we think about monetization is first job, make sure people use it, love it, and can try change. Second job is like how you pay, how do you make this thing viral? So you can go change the world. And then if we’re providing value to. And I think that the, so we’ve really defined very early of what are the values with this new product that we can create? And we look at it as three things, and I was just pulling up on my side. One is, in the, in this order, in this like sequence of can we help you articulate the topics you care about and keep them current. I think that’s a subtle thing, but it’s, it’s hard to keep current. You go on your LinkedIn or your, your GitHub or whatever, and. This was me two years ago, or I’ve never updated this thing. Can we give you peace of mind of not checking other channels and can we help you make like new and valuable connections? If we can do those things, I think there’ll be a subset of people that will value it enough to pay for it, or we are, they like, Hey, if I’m using it for these things, get this one additional feature. So I think I look at it as you have to have a philosophy, and this is what we’ve been talking about internally, a philosophy of what would be a paid feature and what would not And yeah, also not dumping everything out there of every feature on day one. Keeping it very narrowly focused. Like I looked at a tangential comp and I was like, man, they got everything. Like it’s and so I think it’s, how do you keep it very simple solutions focused on one use case versus everything under the sun and then, You don’t need to bring additional functionality and charge for it versus trying to reel functionality back, which Ishmm like Loom did and I think it went down pretty badly.

Matt: Yeah. An interesting point you mentioned that I was chatting with the CTO O of Hockey stack the other day on one of our built right episodes, and they hit this inflection point. They’d built this product and they’re doing their customer research, which is something y’all do an awesome job at gated. But they had this insight where the person was like, I don’t want to use 90% of this solution, but this 10%, this is what I need. And they actually cut the bloat out of the pro product, created hyperfocus in this one area and it just drove a lot of their growth. But I heard you mentioning, focusing in on kind of core use cases a super interesting area there. One thing y’all do at gated, though, I always see you out there wanting to talk to more customers, more users. Yes. How do you weave that into your process? It’s such a big thing. We do AtWork with continuous discovery, but I think you’re one of the better added, especially, leading a company as a C E O.

Andy: Oh, thanks man. I appreciate that. That is, I’ve gotten an article on my LinkedIn profile about it. I can I can drop it. Sisi I call we, you don’t need to put it in the show notes there. Yeah. I’ll send it to you. But it’s a customer advocacy playbook. I think I look at it as, first off, the product has way, like interesting moments of joy and excitement. And then we can build little things off of it of awesome, you’ve got this really, we saw a $250 donation off a $2 ask last week. We’re like, yeah, hey that’s pretty cool. And we can reach out and we can talk to people. So I think it’s like engineering the product where there’s opportunities to connect with users and then building the motions to be able to do that. I don’t know when you and I first connected, but you probably had a fun donation or something, or like one of our teams. Hey Matt, congrats. And you’re like, ah, that’s awesome. And then you’re like, you ideally the product creates joy and then you can start a conversation around that. And, I’ll be candid, like we’ve templatized some of those things, right? I, yeah. Have this fun thing where if I see an interesting person make a donation through our, just feeds through Slack all day long. And if I’m like, wow, that one was cool, I’ll drop an be like, need to see your donation via gated. And people will be like, oh my God, I love this thing. I’d be like, okay, cool. I think I literally have a follow up feedback, which. Awesome. Thanks so much. We love feedback do you have anything that you’d improve? And people like, yeah. You bring ’em in, you throw ’em in, then you can turn ’em into advocates. And so there’s a lot of, I don’t know where I developed it, but I’m really passionate about engaging customers. Now there’s a balance to it too, right? You want your product to do that all. And we haven’t built that all into the original product and on day one, we’re not gonna build it all into the other ones. So it’s like the how do. This is what I’ve spent a lot of my career on. Which is how do you use data in product Yeah. To trigger the right team actions. And then over time, like with technology and ai, you can start on automate more and more of those things. Yeah. I don’t wanna 

Matt: pause, pause there. Anybody passively listening right now? This is your indicator to, to heighten your sense. But the piece you did there, you identified these triggers in, your user’s journey and you systematized actually talking to those users. So you’ve almost built this operating model or system for continuous discovery, which is a great way to do it cuz everybody says they want to talk to customers, but if you don’t consciously think about it, it’ll fall to the wayside with everything going on. So that is critical. I think you’ve identified. At what points, what inflection points are great opportunities to talk to customers and learn. And you built a kind of a system around it. It doesn’t have to be, some over-engineered thing, but you have these inflection points. So that’s super interesting there. Maybe the last point to get into here you start, you’re making this pivot in essence with gated. How do you think, in terms of making a pivot what is. What’s going through your mind where you decide, okay, we gotta make a change. You’ve probably been through some pivots in your career in the past. Is there any similarities you’ve seen through them When a pivot’s ready to happen, different things happening at the same time anything around a pivot in general that just kind of triggers in your mind where it’s more than just a small tweak, it’s something found. 

Andy: Yeah, it’s interesting, like generally I am growing companies really rapidly, right? Like my sweet spot. 5 million, hundred million sales. This is a little earlier stage than I’ve played before. I think for me it’s more if you see the friction in the growth motion, which, for us, I think we’ve built a to people love. Yeah. But it’s, not everyone can sign up for the original email product. And so what we said is, Can we change the world? And if we can’t with the original product, like we’re still gonna keep it over here, but like what? Like we’re always thinking of like, how do we get this to go to millions of people and drive that impact? And so yeah. I don’t think I’ve come across the pivots before on the product side. I think I’ve definitely encountered other things where okay, like we’re not growing fast enough. What else? What other products can we add? But I think here, I guess I would say. I’m conscious you can’t like, keep adding to the product bloat. And so we’re very much like on that day, in mid-May, the new platform will be our focus and yeah, that’s really important. And the existing platform will be there for the original users. We’re not gonna kill it, but I think, so for me it’s being comfortable making those shifts. If you took it maybe a step like. Every company I’ve ever been at has struggled with. We can’t ship more stuff. It’s hire 10 more engineers. You get no more growth no, no more product features. And so I remember I was talking to this dude at Facebook, I was like, man, you guys are constantly innovating. And this is eight years ago, right? And he’s yeah. What’s your secret? He’s we make the hard. And so I was like, okay, gimme an example. He’s messenger. And at the time, I actually still used Facebook and I was like, yeah, I freaking hate how now I have an app for Facebook and an app for Messenger. And he’s I know. We knew it would piss some people off, but by doing that we got the ability to move a lot faster with both products. And so what I’ve seen consistently within companies is they’re afraid to make the hard decision. And yeah, I, in my historical world, it’s go to market changes. And I think within product, which I’ve always been partnered deeply with product, it’s the hard architectural decisions, right? So everyone wants to make everybody happy. They do. It’s let’s add this feature. But no one’s saying, okay we have to kill stuff. And I think that very few c. Have that architect slash product person that’s empowered to make those hard calls. And that’s exactly what we’re doing here, right? Which is we built a product that people absolutely love. I think we send 700,000 challenge emails a month and the world knows about us, and we’re gonna blow it all up to change the world in an even bigger way and bring a new platform. 

Matt: And you just hit on one of the core elements of strategy right there. It’s being willing to make those. Decisions and many aren’t. And that’s where people fail. That is so core to strategy is being able to do that. So hats off to y’all for doing that. And the other thing I heard too, is it tied back to where you wanted to go. Tying back to the manifesto, the impact you wanted to make, and you realized we can’t do that with where we are today, so we need to make a change. So it, it ties back to that thread of where you want to go, what you want to do. And it’s just the core elements of strategy being executed really nicely. I’m really enjoying the journey here. Last question for you Andy. I’d like to finish up on this one. What’s something you wish you knew Cerner? Something you could go back in time and tell your former self, and I’ll leave this open-ended. You can take it anywhere you want, whether it’s on the product side, engineering side at go to market side. What’s one thing. 

Andy: You would tell your former self with gated? I think what it would be we saw massive early user love and we’re like, great hire. I think in retrospect, I would hold back on hiring and I see this across the board people like, let’s hire a huge sales team, or let’s hire a huge this or that. I think it would’ve held back on the hiring until we were like, we literally can’t deal with the deluge. Because then I think it would’ve given us the ability to. Iterate faster and all that stuff. And so for us, we probably staffed up early for the growth. We definitely had some, but we didn’t have like millions and millions of users on the original product. And so I think for us, as we personnel we’re like, how lean and mean can we be? Yeah. And for as long as we possibly can. And yeah, so that’s, I think it’s a, it’s also like the world has changed, right? Two years ago. Like when we raised money and when every company raised money, it was like you had infinite capital. And yeah, it was available right across the board. I see it on product. I see it on in, I see it on sales teams. I see it. And I think the world is changing. Now. Will there be another mad rush of this in a couple years where everyone’s go spend it as much as you can? Probably, yes. Yeah, but I think it’s a discipline that I took away. Yeah. I 

Matt: love that. So it’s almost go lean and mean until it’s painful or until it hurts that you it’s, yeah, you gotta have that pain point, right? Yeah. That’s great. Andy, I appreciate the conversation today. Thanks for joining Bil Wright. Have a good rest of 

Andy: your day. Thank you, Matt. Great talking. I’d say last thing I would say is 

Matt: yeah, where can people find you? Yeah, that’s a great point. I missed that. Where can people find you? How about they find gated gated.com?

Andy: We, you can go there, you can find it over the, in kind of mid-May new product will be available. And check out the website and if people wanna email me you can andy gated.com. And if you don’t know me, you’ll still get an ask for a donation. 

Matt: Yeah, and I’m a  user of gated, one of the early adopters of it. Love the product. Can’t wait to see where it’s going next. So you’ll definitely check this tool out. It’s a great one. gated.com. Thanks Andy.

Andy: Thank you, Matt.

Matt Paige: Today we’re chatting with Jackie, founder of Option One Partners serving companies through technology and product consulting. And this year she just launched Product X Agile, an on-demand training platform for product managers. And agileists and product and Agile are two of my favorite topics, Jackie, so you already got me one over there. She’s made the list of Atlanta Business Chronicles 40 under 40. So she’s a local Atlantian, which we love here at Hatch Works where we’re founded. And she believes that product management is the key to changing the way the world works, plays and connects. But welcome to the show, Jackie.

Jackie Flake: Hey, Matt. Thank you.

Matt: Yeah, great to have you on and I’m excited for the topic we’re getting into today. So you got a fun one for us. Today we’re gonna get into going deep into the five mistakes to avoid in your product career, as Told by Jackie, who clearly has a lot of experience and a lot to say here. N PS number four is my favorite, so make sure to stick around for that, but without fur further ado. Let’s jump into it. Let’s I feel like we need a drum roll or something. Maybe we’ll add that in, in post production, but what’s number one? Hit us with number one.

Jackie: Okay, sure. Yes. So I created this list because I have made a lot of these mistakes in my own career and I also see a lot of my clients making these mistakes. So very common. Yeah. And things to avoid. So number one is simply believing that you are not strategic. I. And so this is something I really struggled with early in my career as a product manager, and it comes down to three things. The first one is just the simple fact that you don’t know how to be strategic or you think that you don’t know how to be. And I think as product managers, We put this pressure on ourselves. On ourselves. We should be having these light bulb moments that are creating multimillion dollar ideas, right? Like the next Uber, Apple or Amazon. But really it isn’t about that. It’s about understanding how to look at both qualitative and quantitative data to inform product decisions. And while you’re, may not be building the next Apple or Amazon, you’re working on really interesting B2B, B2C, products, and those things can all benefit from just learning how to do discovery work and understand analytics and insights. Okay.

Matt: I love this one cuz when I first saw the list, this one like hurt my heart a little bit cuz I love strategy so much. But the way you’re framing it, it’s not that. You don’t need to be connected to the strategy. Understand the strategy, but don’t be burdened by like the whole company’s strategy on your shoulders, in other words. And it’s like, how do you exactly translate that into product, like you mentioned discovery. Go a little bit deeper there in terms of how you think about discovery.

Jackie: Yeah, so discovery is also something that is a really scary word for product people at times. They know they’re supposed to be doing it, but they don’t know h how to go about it, how often to do it, they know they’re supposed to be talking to customers, but when do you talk to customers? How often and how do you interview them and. And there’s so many different ways to do that. So I always encourage my clients and people that I’m coaching to we learn different discovery techniques and then we just practice using them. Because as an analyst at heart, I believe in experimentation as well. And that absolutely applies to product discovery too. You gain learnings and you iterate on them, and it

Matt: starts with talking to the customer. Any this is off topic, but any fun. Or funny stories talking with customers or doing discovery from your past experience or any my interesting nuggets from, I’m sure there’s too many to even name.

Jackie: Yeah. Yeah. Probably, I can’t think of a thing, anything specific, but probably like coming into a customer interview thinking, I’m gonna nail this. I’ve got all the right questions and just feeling so shut down by all the questions I’m asking and thinking like, I am getting zero insight into this customer’s head right now. Yeah. Things like that.

Matt: Yeah. And I think it’s interesting too I’ve been in discovery calls where, It goes a completely different angle than what you thought it would and that’s why you have the discovery is to actually Yeah. Understand how the customer thinks, how they tick. Yes. Such a big piece of it.

Jackie: Yeah. Okay. So that’s exactly,

Matt: so that’s number one. Let’s jump to number two.

Jackie: Okay, so number two is another huge pressure that I put on myself, and I think this is gonna maybe ruffle some feathers when I say this, but it’s believing that you have to be technical to succeed in product management. Okay. And so what I mean by that is, what is being technical? Like, how technical do you consider yourself to be, Matt?

Matt: Yeah. Not as technical as the developers on the team, or as much as I think I should. It’s like that imposter syndrome. If I’m working on a Yeah. Technology product, I feel like I should know how to like code and get in there. So I, I love this one. I totally

Jackie: get this. Yeah, exactly. But it’s that. Being technical is such a range and you hit the nail on the head. As product managers, our job is not to be doing the coding, right? Yeah. Our job is to understand what our customers want and need and build products that delight customers under the constraints of our business. And so I think this is an industry debate and I really despise when I see job. Job descriptions that are titled, or roles that are titled Technical Product Manager. Because now you’re eliminating a really big part of the market who are just gonna assume that they can’t apply for that role because they’re not technical enough. And unless you’re building, a heavy backend or API or database product, then I think as long as you’re a PM that. Knows enough to be dangerous. Our jobs are to understand what the technical constraints and abilities are of our product is, and then be able to translate those in layman’s terms to our business partners and our customers. And that is the skill that a product manager needs. Not necessarily being technical. And as you and I both know, as we’ve gone throughout our careers you pick up tech jargon and lingo and you learn as you go. So being, knowing enough to be dangerous is important.

Matt: Yeah, and I think you hit on the big point there. It’s being able to connect it to the business outcome. I feel like that’s such a big critical role of the product manager is understanding the business outcomes that they want to achieve and helping drive the roadmap, translate that for the team delivering. Cuz everybody has a role on the team. And as product manager, it’s not getting into the code per se.

Jackie: Exactly. And here’s my little tip about Yeah. How to get through that as a product manager is, I always like to say, this is what really helped me in my career. I would find a developer on the team or someone who’s extremely technical on the team, and I would, they would become my buddy and they would not make me feel stupid for asking questions. And so I always knew that when I was with that person, it was a safe space. And if I didn’t understand something, technically I could ask them to break it down for me because we’ve all worked with, more technical people that make us feel small for not knowing things that we shouldn’t know in the first place. So finding the developer that doesn’t make you feel stupid is a really good trick.

Matt: Okay, I want everybody to pause if you’re like listening passively, that is such a big point there. Find the buddy on the team, know how to network and find that person that’s like your your what’s the word? Your co-pilot or something Exactly. Within the team. Exactly. I love that. Cuz they can help translate stuff. And maybe now it’s generative AI as like the next evolution of it. That’s a whole other topic. But I’m curious though, what’s, what is the line? Because I’ve like you, I’ve seen some product managers that have no technical expertise and I’ve seen those that can actually be dangerous. They understand how, APIs work, how they can view the code. They’re not gonna get in there and write it, but they can start to understand the logic behind how things are developed.

Jackie: Sure. Yeah. And a lot of times the more technically you are, that can serve you very well, but then there’s times where it can hold you back. You do have a handful of product managers who came from an engineering background and wanted to transition into product management so that naturally they are gonna be pulled to wanna look at code and maybe even write code. But not to say you need to stay in your lane, but you need to be able to do your job and do it well. And with that comes focus. And so if you’re trying to be everything to everyone, then you’re not gonna serve anyone well or your product

Matt: well. Yep. You just hit one of the core elements of strategy right there. You can’t be, your product can’t be everything to everyone, and you yourself cannot be everything to everyone. That’s great. All right, then we’ve hit the first two. What is number three? Give us number three.

Jackie: Okay. So number three is only playing the role of pm. So I know we just talked about being focused in your role, but at the same time when When you are just a PM and you don’t understand how any of the other roles work, then you aren’t able to be a more effective and well-rounded product manager. So what I mean by that is we always talk about in product that we need to have empathy for our customers. However, we rarely discuss how we need to have empathy for our teammates because we cannot build a product in a silo as an individual. Yeah. Taking on other hats allows you to learn tactical skills to be more effective and well-rounded in addition to driving empathy for your teammates. And so the roles I always encourage my clients to learn how to do is QA because. First and foremost, you have to know how your product works and the only way to do that is to QA it up inside and out. And then, wearing your scrum master hat. I grew up in the Atlanta startup scene, so I didn’t always have the luxury of having a well-balanced product team. And so a lot of times I had to run the agile ceremonies and I had to report on the team performance data and product performance data. And so that forced me to learn how to facilitate effectively, how to block out for my teammate. And I think that’s a really core skill that you can take into any role in your career. I say, learn how to wear the dev hat, but what I mean by that is just to try to put yourself in a developer’s shoes because the way that they have to learn how to solve problems through technology and coding is a very different way of problem solving than how you and I approach problem solving. Yeah. Understanding dev and the challenges they undergo in their jobs is important and drives empathy. Other bonus roles would be learning the product design role because again, I didn’t have the luxury of having a fully balanced product team, so a lot of times I’m the one creating the wire frames and drawing out the wire framing. Yep. Yep. Totally. Scary if I look back on the ones I’ve built. And then just also just understanding how customer success and sales work together, and those are the frontline people. In your organization a lot of times, and when you don’t have access to customers, they do. So making sure you’re building those relationships with those people is really important. And then another bonus role would be product marketing. Because if you’re not involved in the go-to-market strategy of your product, then you can’t influence that function either.

Matt: Yeah. And for anybody in product they probably get this or somebody thinking about getting into product. That, that sounds scary because you’re talking about almost every area of the organization. But it’s true though. Like that product manager, I’m sure you’ve heard it before, they’re kinda like the CEO of their product, so they have to have tentacles into these different areas. And I love the way you put it, empathy. In two of the roles you mentioned, the developer side and the design side. One thing that I’ve. Notice painfully through my own experiences, again, make friends with them and get them in the process early on. Because if you’re defining things up front and then handing it over to design to do something and then handing it over to Dev to go build it, you’re in this kind of like transactional output focus. But engineers are super smart for a reason, and if you can bring them early on, They may have a different way of solving the problem that you have. Knowing how the thing is architects, I love getting them upfront in the process.

Jackie: Yeah, absolutely. Product idle, Marty Kagan says, if you’re not bringing your developers Yeah. Early into the discovery process, you’re only getting 50% of their value. Yeah. Yeah.

Matt: Marty Kagan’s like a one of the main people we love here at Hatch works. Anybody that has not same, read some of his books, like just, stop what you’re doing. Go to Amazon. Go purchase both of his, I think he’s got two books right now. They have a couple others through Silicon Valley Product

Jackie: Group, but yes. He’s great. Yeah. I have the opportunity to meet him and work with him. No way. December. Yeah. Really it was awesome. It was good. Good.

Matt: Deeper on that. I’d love to, I’d love to hear more about that.

Jackie: Yeah. Marty Marty and I agree on the same thing, that there’s not enough product co management coaches in our industry. And so he invited 50 product coaches from around the world to New York City in December for a two day workshop. Wow. Okay. Just to network with other product coaches so that he could then recommend them to his client. And but it was also work shopping, learning from the best. He brought the entire Silicon Valley product group team and I was fortunate enough to be selected to go as a product coach and meet him and work with his team. And I’m still on Cloud nine, and that was five months ago.

Matt: Wow. Jackie, why was that? You should have gave me that as part of your bio. That would’ve been the ultimate should have book like I’m best friends with Marty Kagan, that, that should have been the intro. I would’ve

Jackie: been that far. I don’t know if Marty would agree with that, but I still. I still look up to him so much.

Matt: Nice. That’s great. Okay, so we’ve hit the first three. Let’s do number four.

Jackie: Okay. Number four. This is very tactical in nature, but this is the inability to write really great user stories. And I say this is a mistake I see in people’s careers because if you as a PM are not able to write great user stories, or your team, if you’re a leader is not writing great user stories, then you are. Doing a lot of wasteful work upfront if we take all of that time to do amazing discovery and understand what our customers want and need, and then we don’t take the time to distill that down into the smallest unit of work to translate our product strategy and vision to our team, then the downstream effects of that are devastating to your delivery function.

Matt: Yeah, that, that’s your main mechanism of not your main mechanism of communication, but it’s your artifact in a lot of ways and having structure and I’d love to hear how you structure it, but it’s not just the as I want to do so I can X. There’s the acceptance criteria element. There’s all these different pieces that are critical in helping your developers, your designers. QA folks, you know everybody across the board, but I’d love to hear your thought on user stories and if you have a framing that you, you use in the process.

Jackie: Yeah, for sure. I love user stories. I love teaching people how to write them. And I actually have built, I’ve done several client trainings on user stories, and I’ve built a an on-demand course on user stories because it is so important to get a right. Yeah. But really what it comes down to is your job as a PM is to create a shared understanding of what the work is and equally as important, what the work isn’t. And so getting user stories right. Is re about removing assumptions and ensuring everyone is on the same page. Otherwise, we’re doing a lot of rework. We’re rolling back releases. We have unhappy customers. Downstream effects are just, very negative. And you’re not serving your team or your customers

Matt: yeah, everybody hates rework, or at least they should if they don’t. Yeah. And you hit on one piece there. It’s that shared understanding and. Knowing what to do and what not to do. It’s back to that empathy element for your team, because if there’s ambiguity, you’re leaving it in the developer’s hands to interpret, how they think. It should be interpreted. So having that clarity is important. And I’m sure like this is discussed in ceremonies, like your sprint planning or your refinement sessions where you get to have that discussion around the actual user stories and clarify as well, right?

Jackie: Yeah, absolutely. And what I hammer home to all of my clients is that I want you to share the user stories with your team at least twice, if not. Three times before they start working on them. Because you can’t just take a user story, run into sprint planning, throw it over the fence to your team, and expect them to hit the ground running, right? Like they need time to think through how they’re gonna approach it again as a developer, how they’re gonna problem solve it. They need to see it several times so they can iterate on it. And also, we’re humans, so we’re not gonna bring a perfect user story to the team for the first time. So we need time. For once the team pokes holes in it to go back and fix it and iterate on it and just work it and get it to a good place.

Matt: No, that’s a key lesson and I’ve been at fault of this. Your team should not be seeing the user story in sprint planning for the first time. That’s great. Especially with exactly different personalities on the team. Some people may prefer to, take time to understand something, think through it versus on the fly, so you’re not getting as much collaboration if you go that route. That’s great. Yeah, exactly. All right, last one, number five.

Jackie: All right. Number five applies to anyone in any career. This is just simply not meeting people where they’re at. And this one is really important as a PM because. Product management is, I like to say, is one of the hardest jobs because you have to get people to work for you that don’t report to you. And so that is a huge challenge. And I’ve seen people do it. Horribly, and I’ve seen people do it really well. And building those relationships with my team has really served me well in my product career. And just understanding that everyone has something to offer no matter if they have years of experience on you or, they’re much older than you or much younger than you. Just not making assumptions that you know more than them. And so I also like to say that, we assume that we need to find commonalities to connect with people. Like I have to, go play Dungeons and Dragons with a dev on my team in order to connect with that person. And that’s not necessarily true if I don’t enjoy that as well. Yeah. But it really is about developing micro connections and shared experiences and I have an example from From a job long ago. So I was working at a local Atlanta startup called since been acquired by Oracle and nice. We had a huge deadline. We were building a product for one of our clients, which was Sports Illustrated. And we were really running into Just this deadline with two weeks to go and we were not as far as we needed to be. And we did an estimation and the devs were saying, this is definitely gonna take us longer than two weeks. And so leadership had asked the team to work overnight until they could get the product launched. And so that was really hard pill to swallow for the devs, but they were agreeable to it. And so they were gonna stay. They stayed till three and four in the morning. We’d go home and sleep for a couple hours, come back around eight or nine. Get back on the horn and start coding again. Everyone else in the office left and would just leave these guys. To their loan summit. I was like, I’m the PM at the time, and I couldn’t leave my team. It was just I wasn’t, I couldn’t contribute to the quality of the code. I couldn’t contribute to the actual speed of the code, but I could be there to order them food, get them coffee, give them pep talks, answer their questions around the tickets and things and be there to review their work and. That time was really critical for us. And it really just, that created such a safe space for our team. And that’s an extreme example. We were up many nights in a row. And I lost years of my life not sleeping then. But that’s until you have kids and then everything and then,

Matt: yeah. I know we were just chatting right before this. Yeah. We both had some fun child, had a sick one last night and Yeah. A whole nother Exactly. But I love this though. And that story is such a great example. It’s, meeting people where they are, and I know you’ve experienced this, those teams that have that level of connection. It’s intangible. You can’t really put your finger on it, but they just perform so much better. Their, their velocity is so much higher the ability to get things done. It’s that connection and it’s intangible. But I’ve felt it on teams that had it and didn’t have it, and I’m sure you have too.

Jackie: Exactly. Yeah, you’re absolutely right. It’s about building that trust and just, like I’ve seen PMs that take that complete dictator approach where they just, hand tickets over build this devs can’t even ask why. Devs can’t even look at a roadmap or understand the big picture. And they’re not there to sit in a basement and code with a hoodie on, like they are there to do strategic problem solvers. That’s great. And so building that trust and building those safe spaces with your team will only serve your product better. Yeah.

Matt: It’s that servant leader approach. And yeah I love those that are not on video, can’t see this, but you actually have a. Kanban board, I mentioned this earlier. Yes. That you have a Kanban board behind you. And this is the Kanban board for, is it for your life or your work? You actually have a physical that just goes how far, you are a practitioner of agile and the product way of thinking.

Jackie: Yeah, for sure. I run my business on Kanban. I wanna be authentic to what I’m teaching my clients. All day long. And I know it works. Yeah. No, perfect. No, that’s the reason to use it. Yep.

Matt: So there’s the five. And just to recap, let me see if I can hit ’em all. Don’t put pressure on yourself to be strategic and let me know if I’m butchering any of these. Don’t put pressure on yourself to be too technical. Understand how to write a really solid user story, meeting people where they’re at, and the ability to wear multiple hats. And I think I got ’em a bit out of order, but did I nailed it? Did I sum ’em up there? Perfect.

Jackie: Nailed it. Yeah.

Matt: That’s great. It’s all right. So I want to try something new. We have not done this on an episode yet, but I wanna do a bit of a rapid fire round. We’re gonna test this out in the practice of iterate to improve. It’s one of that core values we have at Hatch Works. So let’s test something new out. So Rapid fire, I’m gonna ask a few questions and just give us your first thing that comes to mind. All right. First one, what company is doing product management right?

Jackie: This is a shout out to a local company here in Atlanta. The Home Depot is actually Oh, okay. I’m very impressed by their product transformation. I was I. Brought in to help them support their product transformation back in 2016. And they, just to see their growth since then has been amazing. They really focus a lot on discovery. They are very good at talking to customers and getting those quick feedback loops. They’ll go into stores, talk to customers, talk to associates who they’re building tools and products for. And I’m just so impressed by them. And you wouldn’t think that of a, a. A hundred year old retailer, but their tech Yeah. Tech team is doing awesome. That’s

Matt: great. That’s an Atlanta staple right there, home Depot. Yeah. And to your point, it’s like you, you think of those kind of older, large enterprises as not doing this well. So that’s great that’s an example of one that is, I feel like I’ve been living at Home Depot lately. We just laid out mulch across our whole house and Yeah. Hey,

Jackie: I almost over the

Matt: weekend still hurting from it. Yep. Exactly. That time of year. All right, number two. Who is inspiring you right now in terms of the product management community? An individual. I think I may know the answer to this one.

Jackie: Yes. All right. So my first, we talked about Marty already love him. I always will or gimme somebody else. Honestly, it is probably the career changer. I, like people who I work with a lot of career changers that are coming from other disciplines or industries moving into product management. And I really, I’m really passionate about coaching them into product. I work with a lot of former teachers, former nurses who are making that transition Oh, cool. Into technology. Cool. And when you think about it, those type, those people that are coming from those types of industries, they have all the soft skills they need to be successful in product or in an agile role, right? They care about people, they’re compassionate, they want people to succeed and do well. They’re used to moving things outta people’s way so they can succeed. They, they know how to talk to people and make them feel good. They know how to teach and educate. And I would say career changers who are, they’re just so brave to take that leap. And to learn something new. And, I find that so inspiring.

Matt: I love that. Okay. So you took the angle of a persona, a group of people. That’s really cool. I like that. Yeah. All right, next one. This is one I’m really interested in with everything going on right now, but what’s your take on how generative ai, G P T all, everything LLM will shape the discipline of product management.

Jackie: I’m really excited about it, just like any other new technology. I think learning how to use it to your advantage is what you have to do in order to succeed and, yeah. Have growth in your career. And I, what I would say is for product managers to figure out how to leverage it and use it for the things that anyone can do. And then, Still focus on the things that only you can do and not try to, fit it into a mold of something where you’re thinking through strategy or your ability to do discovery or your ability, like to connect with your customers and stakeholders. Like AI’s not gonna replace the ability to do that.

Matt: Yeah I love that kind of co-pilot analogy there. And we’ve started testing it, actually helping write user stories. And it’s, a big part of, it’s the prompt and what background you give it and all of that stuff, but it, yeah, it quickly iterates if you give it the right structure, based on, you mentioned you have, I think, some courses related to this. If you can prompt it, the right structure, it can give you something good to work with. And obviously you gotta fine tune it and all of that. But we’re starting to play around with some of that at Hatch Works now.

Jackie: Yeah, for sure. And I’ve tested it too in user stories and I think it can absolutely, make you more effective, but even you have to understand how to break it down and, cause it really wants to bulk up the acceptance criteria.

Matt: Exactly. Yeah. Yeah. And that’s it’s an art and a science I think with prompting as a, we’re starting to learn and that’s the key. I don’t think the product manager role. Ever goes away. I think it gets enhanced by generative ai, but I think it’s still gonna be one of those core run ones that’s always around there for all the points you mentioned. Yes. What’s okay, what’s one thing you wish you could go back in time and tell your former self before you got started, either in your career or the company you founded gi Give us a nugget of wisdom.

Jackie: Sure. For me it is probably in line with the five mistakes we just discussed, but it really would probably be not putting so much pressure on myself to succeed, I always, in my career, I’m always the one that got in my own way and yeah, it was, by no one else. I always had cheerleaders. I always had mentors telling me that I was gonna do amazing things and, really saw me do growing heavily in my career, and I just never believed in myself. And I’ve, I’m so happy to finally be at a place where I do have confidence in what I do and the value I can bring to my customers and clients. And I wish I had known that just starting out, but, it takes experience and failure and to learn.

Matt: Yeah, that’s the key thing. Iterating as you go. I struggle with that as well, and. At the end of the day, nobody’s paying that much attention to you anyway, so it’s okay if you mess up here and there along the way. Exactly. Exactly. Alright I got let’s see, I got two more for you. What technology are you most excited about or tool, technology, anything that you’re using today?

Jackie: Oh, man. All right. Let’s see. There is a really cool, so speaking of ai there is a, like a headshot tool that you can use. And I’m forgetting the name of it, but you can input your picture and it’ll spit out like 50 different headshots that look really professional and good. So if you’re like, sick of using the same headshot over and over again and it’s got some tweaking it, it needs, it. It kind of messes up here or there, but it’s definitely like making pictures in your likeness and everything, so there you go. You don’t have to go hire a photographer,

Matt: yeah. Has so many use cases. We’ve started playing around with Adobe Firefly. We’re in their early beta testing and playing around with it. Similar to like a mid journey text, two image, and it’s insane. Yeah. The stuff you can do with it, it’s really cool. Cool. All right, last one for you. If you weren’t in product, what would you be doing?

Jackie: Oh, that’s a good one. If I wasn’t in product if I wasn’t in product. Let’s see.

Matt: I stumped you with that one.

Jackie: You did stump me. I wasn’t prepared for this. I don’t know. Can I be a professional traveler for a living? Yes. That’s perfectly accept. I love traveling. Yes. Okay. Okay,

Matt: good. Fa, favorite spot to travel? What’s your favorite spot to go?

Jackie: Oh, I love Hawaii. Oh. Have you been there?

Matt: Never been to Hawaii. I’ve been to Costa Rica recently where we have a big presence down in Latin America. Yes. Costa Rica’s amazing. If you haven’t been there, check that out.

Jackie: I’ve been one. Cool. Amazing. Yeah. Yeah.

Matt: It’s such a cool place. A awesome Jackie. Thank you so much. But please let me know where can people find you maybe a little bit more about Option One partners or this learning platform you’re building.

Jackie: Sure. So yeah, option one partners.com. Find me on LinkedIn. We love to serve companies through consulting and staffing work. And then productxagile.com is where we have on-demand courses to help product managers and agileists upskill in their careers. And we actually have a promo code for you guys. So HatchWorks30. At put that in any on the checkout page at any of our courses at Product X Agile, HatchWorks30, and you’ll get 30% off a course.

Matt: Nice. Hatch works 30. Okay, perfect. And I think we had the user story course, so that one’s out there plus many more. Yes. I can’t wait to check it out. Yeah. Awesome, Jackie. Thanks for being on Built Right. We’ll talk soon.

Jackie: Okay. Thanks Matt. Bye-bye. Bye

Matt Paige:

All right, we got a good one for you today folks. Today we got Hatcher’s very own Joseph Memer. He runs our solutions consulting practice and has about 20 years experience building teams that build and deliver custom software solutions. And he’s one of the few people that I give the tag of Agile Guru to every time I talk to him, I’m learning something new. I keep the notepad handy whenever I’m chatting with Joseph. He’s just a wealth of knowledge with everything agile, everything, building software in general. But welcome to the show, Joseph.

Joseph Misemer:

Thanks for having me, Matt.

Matt:

Yeah. Excited to start chatting with you on a interesting topic today. And today we got a bit of a hot take with a bit of a contrarian view. What most everyone in our industry holds near and dear to their hearts. And that’s the beloved MVP approach to software development. Everyone’s heard of the mvp, the minimal viable product popularized by air crease. It’s become the sole defacto way of building software full stop. Like it’s just, it just is. Everybody knows the mvp. If you’re talking about waterfall, you kinda get shunned out of the room. And if you try and speak against m p, you sure to attract an angry mob with pitchforks. But luckily Joseph, for us today, we’re in the safety of our homes. This is recorded. Nobody’s coming after us, after the show airs, maybe. But we’re going there today. And don’t get me wrong, MG MVPs have their place. If you’re trying to test a market hypothesis, build a proof of concept, test the technical feasibility of a solution. MVPs are a great way to go. We use MVPs for those approaches when you’re building something new but you can’t, apply this blanket approach to everything and just assume it’ll work. And this is especially true when you’re modernizing or redesigning a software solution. Joseph, take us off the top. Why does an MVP approach not work when you’re modernizing software or redesigning software?

Joseph: 

Yeah, sure thing, Matt. And I want to, we’ll start with a little bit of a metaphor. We’re all familiar. I the MVP metaphor of we’re gonna start with the skateboard, move to a bicycle, move to a car. And I think that’s a great metaphor for MVP software development. But if we think about maybe an experience of driving a car on a highway, right? Because that’s the point that you’re gonna be at when you finish that MVP project front. Your customers have that car, they’re zooming along. In Atlanta, people move pretty quickly on the highways. So let’s think about what would happen if somebody already has a car that you gave them and now you want to tell them you’re gonna start them back over on a skateboard with no warning. On the highway, what’s gonna happen? They’re gonna be terrified, right? They’re gonna be zooming along at 65, 75 miles an hour, and then they’re gonna find themselves standing on a skateboard. And that’s probably pretty dangerous, right? So we can’t, if we’re taking an existing solution, we can’t start people over from zero. We have to meet them where they are now. And that’s using your existing software.
Matt: That’s what, yeah, I was gonna say, that’s what so many people do, right? They take this MVP approach. You hand your customer this shiny new object. It seems great, but it’s not meeting the needs that they have. And you can, you get some upset customers that if you take that approach,
Joseph: Yeah, it’s such a big, it’s such a big risk. People get used to, people get used to things, right? Every time Microsoft changes the location of a button or a color of a highlight, you’ll have coworkers that are flipping out. We all have our experiences when somebody changes the behavior of something inside our email client, right? We’re so used to it. And if it gets moved even if it’s good in the long run, it’s still such a shock in that intermediate. Intermediate phase.

Matt:

Yeah. And so if we’re looking at, MVP, building something new versus MVR like, let’s baseline on a few differences, a few points of difference.They thinking we have this checklist of things, right? If you have an mvp, a new thing you’re testing a new solution versus mvr, you’re trying to replace something existing. Yeah. Primary goal of an mvp. You want to validate a market, validate a product. In VR or modernization, you wanna migrate. Users to a new solution with Little churn. Yeah. User base. None on the new thing. An existing base on the existing thing. Target users, you’re trying to attract new modernization. You wanna retain existing competition’s a whole nother side of it. Are you competing with existing behaviors with a new thing or your own solution with mvp? So there’s a whole litany of different differences here, and that’s not even getting into the workflows and the technology and the culture and all those other pieces. So there’s a dividing line between an mvp, something new versus modernizing something existing.

Joseph: 

Yeah, and I think that, you touched briefly on it, but one of the ones that I think is really worth considering a lot is the question of whether you’re starting from. From nothing or starting from your existing solution, right? You may have already won over a bunch of people to, to doing, whatever they’re gonna be doing on the platform that you’ve built for them, which is great. It’s such a powerful place to begin. But you can’t pretend they don’t have that experience. They can’t, you can’t pretend they’re not used to something. You can’t pretend that they’re not comfortable maybe with the workflows that they’ve built around it. And if you try to take that away without making a really strong argument in return, you’re gonna end up with a bunch of people who are unhappy and who might walk.

Matt: 

Yeah, definitely. And people are attached to their existing processes. But let’s get into the six, or so principles of mvr. So there’s like some core tenets that really start to mold this approach and this methodology. So I’ll Hit one. Let’s hear the rationale and the details behind it. But the first one being with an mvr, you’re competing against your existing product.

Joseph: 

Yeah. We talked a little bit about this just a minute ago, but. Hopefully people are already using the system that you’re trying to modernize. And in that case you’re not just competing against people from the outside. You’re competing against your existing experience. And so there’s a possibility of scaring people away. And, this is a place where there’s actually been a lot of study done on people’s openness and willingness to accept change. And there’s a couple of things that we’ve found are really powerful drivers of maybe sticking with. Older experiences rather than making a switch to the new one. And one of them is something called the endowment effect. And this is where people are really, almost always value what they already have more than something new that they can get. And they’ve got that strong grip on, on the current piece. And you really have to pry that, that out from them. And the other one is this idea of loss aversion. We think that it might be the same. We might be just as happy if we win $5 and if we lose $5. But studies have pretty consistently shown that people feel two to four times as bad when they’re encountering, when they’re encountering a negative experience than a positive experience. And this, I think, touches on one of the big risks you have to be aware of, right? You might be able to keep something, but if you give somebody a bad experience where they lose a tool they’re used to, they’re gonna be way more upset. Then if you give them a new tool that they didn’t have before. So this is something you really need to consider carefully when looking at your modernization plans. Because that idea of loss aversion, making sure you don’t take something away that people value is way more important to your users than giving them a shiny new bobble.

Matt:

Yeah. And you, it’s like that bird in the handsworth too, in the bush. And it’s not just like you can’t supply a bad experience, you gotta. Supply something that’s better than what they have. Something the same just isn’t gonna cut it. Yeah. It’s a, it’s almost a level, a higher bar when you’re modernizing or redesigning something existing in a lot of ways. Yeah. And you mentioned the endowment effect. It makes me think of my, so my brother-in-law, he’s got a dog, sweet dog, great animal. You give him his treat and he turns into this Jekyll high type of thing, protects it. He, if you even get close to it, you’re losing a finger. And it’s the same kind of thing. People are protective of what they know. It’s like humans are weird animals. I guess dogs are too. People are protective of what they have. Yeah. Yeah. There, go ahead.

Joseph:

No, I, I was just gonna say that’s something that that’s incredibly true and it’s so important to keep in mind is that the way people are gonna fight to, to keep their workflows the same and not have to give something up when they make a change.

Matt: 

Yeah. And so the next one prioritize your most valuable users. I think this is a super interesting one and it’s a big kind of differing point when you think of MVP versus MVR. It’s who are your key users? You’re prioritizing.

Joseph:

Oh, yeah. Cuz in MVP, you’re not even sure. When you go in with that blank page, you’re able to, you’re able to draw all over it. You’re able to make whatever decisions you want and draw whatever conclusions you’d like because there’s nobody there to tell you’re wrong. And when you’re doing a modernization, you’re gonna have people that are valuable, whether they’re the folks that are paying the most, the folks that are doing the most lever use, whether they’re people that you’re leveraging to. To speak for you and do word of mouth marketing however you define it. There’s gonna be people that are going to be adding that extra bonus level to your user experience. And you have to cater to them. Don’t bend over backwards, but you have to be particularly concerned with how they’re gonna react to changes. And you can’t just go in and assume that everyone is gonna be reacting the same way and that the impact of everyone’s reaction is gonna be the same.

Matt: 

And those users that use your system the most, that are driving the most revenue, and we’ll get into this as another principle as well. They have some of the craziest workflows and ways of using your product too. But the example I like to think of is it’s like you’re building condominium, right? Floors one through 20, exactly the same, two grand in rent. They’re all the same ceiling height, same amenities, all that. But you get to the penthouse. And they’re paying let’s say, I don’t know, $20,000 a month, something crazy. And they want swimming pool. They want their own elevator. They want a helipad, they want 30 foot ceilings. They want all these crazy amenities. If you’re not considering that upfront you’re outta luck by the time you get to need to start building that pen penthouse. And you haven’t established a sound foundation before you got there.

Joseph: 

Yeah. And I think along with that too, you can think about maybe people are using your system in ways you don’t anticipate. And so that needs to be part of your conversation as well. When you’re talking to these high value users. You put in a closet that you expect people to use for coats and they end up using it for some other kind of storage or something, and be prepared to respond to people finding that new new exciting way to do something. Maybe you’re, you’ve got people who are used to having an elevator big enough to. Piano in, and you can’t replace that with a tiny little elevator if you don’t talk to them and find out how they’re using it. And so be aware and be particularly aware of your power users, right? Because they’re gonna be the ones who are going to dream up ways to, to take full advantage of everything that you give them. Yeah.

Matt: 

So everybody picture that in your head, your customers. Trying to hoist a piano up the side of a building and how pissed off that’s gonna make ’em. So just visualize that in your head when you’re modernizing something and the approach you’re taking, but you segue perfectly into the next one, which is understanding how customers use your product. And I think every customer or client we’ve engaged with, They’re always surprised at how current users are using their system that they just were not aware of. And as we’re doing the research these new interesting ways of how they use the solution become unveiled, but talk through that a bit. And also that leads to potential opportunities, right? Once you start to learn these things.

Joseph: 

A hundred percent. And I think that’s one of the things that’s so great is that People use whatever tools they’re given to achieve their goals. And so if we’re making a tool for maybe our in-house staff to use, they’re gonna find the ways to do as many pieces of their job with that tool as they can. And if that means that when you’re starting to talk about how to replace it, you discover that people have been using your free text fields, for really weird things, or that people have been, tracking assignments in ways that you hadn’t ever anticipated. You both have to be prepared to support that, but it also gives you a great visibility into some of the valuable pieces you can do as part of the modernization process. If people have been using a notes field to track who owns a task, maybe grant people in the future, the ability to assign tasks, and that opens up a whole space of ability to drive value for your organization. As you find some of those places so you can automate on top of those pieces. As you find these other non-traditional, non intended ways of using the software as you go forward

Matt: 

And then there’s always the the tool that is will be around, I think it’s like a cockroach. It’s never gonna die. And that’s Excel. If people are still, I promise you, using Excel somewhere in their process that you don’t know about, that you could, automate and put into your solution. We’ve seen that a ton of times as well. Yeah.

Joseph: 

Yeah. And I think it’s always informative. The time I’ve worked on a modernization project, there’s always been somebody that’s come up to me after a discussion of a workflow and been like I can’t tell you this when my manager was on the call, but here’s all the steps that we do that, that nobody knows about. And there’s always something in there that you can capture and improve upon because people are having to cook corners. And if you can give them the tool as part of that modernization, it’s gonna be better for everybody. Improve visibility less. Less opportunity for mistakes. All those pieces come into play. But having, being able to have those open and honest conversations about how people are using the tool and about how people are using other tools because the current one doesn’t meet their needs, those are gonna drive the ability to get really successful modernization questions.

Matt: 

Yeah. And that’s powerful. That almost gets back to the discussion we’ve had with Andy Sylvere on another episode where, Being able to get under that user experience, knowing how to ask the right questions. Like you mentioned, like when mom and dad aren’t in the room. I’ll tell you how this really works, right? But it’s real. It, people figure out the best way to use things and it may not be the instructed way to do things. Yeah. In getting into the next one, so this segues into now identifying the most critical workflow. I think with mvp, the idea is, let’s keep it simple. Let’s keep it end to end. Let’s start with an easy workflow. Modernization redesign, you can’t always go that route. Yeah.

Joseph: 

If you’ve been stacking MVP for a while, you might have accumulated 3, 4, 5 individual pieces. But much like when we were talking about the critical users, when we started with mvp, we had a blank page. We were able to pick a path and we were able to see if it. If it’s stuck with people and try another path and try another path. Now we’ve got people who have worn those tracks into the ground. And we’re not gonna be able to pretend that that we don’t know what’s coming. So we need to look for those spots where people are doing those high value workflows. And hopefully by this point you’ve got an understanding of where the value is in your product, but if you can understand that and look at the things that are driving it that’s gonna help you make the selection on the places where you’re gonna get the most most bang for your buck when you’re looking at going forward.

Matt: 

And this really starts to frame up your roadmap in a lot of ways. Yeah. Once you identify, okay, who’s our most valuable user, right? Who’s, what’s the ways they’re using the product? Identifying the most critical workflows, then it starts to frame up, okay, how do we. How do we sequence this thing out? And that gets to the next principle, which is, and this actually does connect back to an m MVP type of approach, but you gotta prioritize incremental enhancement over big bang. So I think that’s one thing with MV r this is not an approach saying build everything, launch it all at once, and pray that it works. That’s not it. Yeah. Talk us through that.

Joseph: 

Yeah. And it’s. I think it’s important to recognize here we’re not trying to throw away the entire idea of mvp, right? The idea of take a small slice and get people using it so you’re not sitting on value continues to be in play. And you can see a lot of that. As you get an understanding of your workflows, there’s gonna be places where you don’t have to tear down and completely rebuild. Take advantage of that. Get people using the new technologies. If you’ve got customer facing and internal facing, you can make choices about where you want to do your first, UI update. If you’ve got process pieces, you can pick a place and update a, a. Slice of the process. Get yourselves so that you’re seeing some of that value. But I think when you look at these too, these are all tied together, right? Your most critical workflow might depend on which type of user you’re looking at. Your incremental steps may depend on what your critical workflow is. And all of these are gonna depend on, who’s already using your system. And how it’s already being used and how you’re gonna try to support that. So that might drive oh, yeah. The most important incremental enhancement is to make sure that we don’t delete any functionality, but we modernize it and we get it working better. Cool. That’s a good informed choice that you can make sometimes.

Matt: 

Yeah. It’s a, it’s almost like the way to think about it, MVR is a combination of all the MVPs. Required to migrate your existing customers to your new solution. With minimal churn. That’s another way to think about it. It’s like, how do you chunk this thing up in a lot of ways? Yeah. And the last one being, focus on improving the function, the experience and the technology. A lot of times you’ll have people sway too much one way or the other where it’s, solely it’s just technology. That’s all we’re focusing on. Let’s lift and shift this thing to the cloud and not worry about anything else or. Let’s focus too much on the ui, but your technology’s old as dirt, right? But it’s all of those things that are, you gotta think through and prioritize when you get this opportunity to actually modernize something. Yeah. Yeah.

Joseph: 

And I think the big piece with this is we don’t want to do something for no reason. Let’s understand why we’re doing things. We don’t want to just pick up and drop something into the cloud. We better have a reason for it. It’s not free to pick up something and drop it into the cloud. Yeah. It’s not free to completely redo our ui. It’s not free to to change our workflows. So let’s understand, let’s make sure we understand why are we cleaning this up? Are we cleaning it up? Because we’re finding that our team members can’t do their tasks without making mistakes and we’ve now learned a better flow to put in place. Great. Are we finding spots where we’re not collecting the right information? And so we can’t do the reporting we need to be successful as a business. Awesome. Let’s collect that information. Let’s do a project to collect that information. Are we finding that we’re not able to scale and so we’re having performance issues because we have our application running on a single server in the closet in back, awesome. Let’s move to the cloud and give ourselves the ability to be responsive to our loads. But all of these are a reason, and it’s making sure that we understand why we’re doing it so that we’re not just Hey, I’ve heard of the cloud. Let’s do the cloud. No let’s do it for a reason. Let’s do it for a reason. And that reason is scalability or flexibility. Or a new technology. Not just because we want to. And I think that’s so important for all of these is that, if we understand why we’re doing something, we’ll be able to position to tell our users what the value is gonna be and offset that endowment effect. We’re gonna be able to, Make the right selection of which users we need to support first. We’re gonna be able to make the selection of which workflows we need to support first. We’re gonna be able to pick which incremental change we make because we know why we’re doing it, and we’re not just throwing a dart at the port.

Matt: 

Yeah, exactly. And hopefully by this point, you, you jive in thinking, okay I understand why MVP may not be the best thing for everything, and I’ll understand the why. But Joseph how the hell do you do this? How does this work in practice? And you hit on some of it too. And that’s one of the most important things to consider. That’s where I see some of our solution consult things, work through this. You gotta be intentional. Yeah. About how you’re releasing out functionality, how you’re communicating it to your users. Cuz the last thing you want is unnecessary churn. Which relates into either unhappy users or lost revenue or inefficiencies. And there’s, several ways you can approach this. We got three favorite ones, at a high level we like to talk about, but take us through the the different approaches. Kind get down to that how layer.

Joseph: 

Yeah. So when we look at it, there’s three ways that we can approach doing a modernization an M V R project. And we don’t always pick just one of these play very well together, either between or within each other. But there’s three approaches that we look at most often. And one of them is called the functional approach. And this basically is where we look at a function that somebody does, and we can try to, we can try to upgrade one slice of your function. And we can pick another function and we can follow that all the way through. And this is maybe a spot where you’ve got different users and they’re very siloed in how they use the system. And if there’s no overlap in how people are gonna use it, we can take a look at this functional approach and we can upgrade or update the flow for all of one type and then all of another type and then all of another type. And this is great if it’s super siloed. But if it’s not super siloed, maybe we’ll need to take a look at another approach. And there we might look at the process process approach. And this is a case where maybe we’re cutting across different user types, but we’re able to follow a single flow all the way through and it sits by itself. And so we’re able to track, maybe an entire deposit flow from nose to tail, and then we can follow an entire audit flow from nose to tail. And this may have different people using the system at each step. But everyone is going to be self-contained inside that process. They’re not gonna have to hop from something that we’ve updated to something that we haven’t updated. While following that flow, they may have to hop from updated flow to not updated flow, but that’s an entirely separate workflow and they can make that transition fairly smoothly. The third one, Is what we call the add-on approach. And the functional and the process approach have been very focused on you have an existing system that’s doing everything you need to do and we need to take something new and bring it into that space with the add-on approach. We’ve got a an opportunity to take something that you’re not doing at all yet. And this can be a new function. It can be a new process. It can be both. But we can take this and no one’s ever done it before and we can bolt that on. Using our new approach and then we can bolt on another system. We can bolt on another system. And this allows us to focus on these new places, use the technology, use the new workflow, use the new pieces and tools that we have available to us while not touching any of the existing pieces. And this is perhaps the most extreme example of not messing with anyone’s current play base, but it’s certainly an option, if you’ve got something that’s completely new, don’t feel like you have to build it the way you built stuff 3, 5, 10 years ago. Yeah. Look for a way to build the new stuff with the new tools and then you can come on behind and try some of the other approaches to bring some of the pieces that you’ve built in the past forward.

Matt: 

Yeah, and this is one where if you looking to get into a new market or maybe a new customer segment, that add-on approach is. Particularly interesting way to test that out. So we got functional process add-on. Those are the three right there. And, it’s all about getting to, what is that? Releasable increment. How are we thinking about the approach to really bite off this elephant one bite at a time? Yeah. We’ve seen it fail on one side where you try to release an MVP and customers are unhappy. We’ve seen it on the other side where you try to do too much and package this giant thing together and release it, and that’s not great either. So you gotta have a dedicated approach and have the teams aligned on the approach that’s a critical thing cuz. But something existing you’re not, you mentioned sandbox earlier, you’re not in your own little sandbox. You got customer support. You got finance, you got all these different areas of the organization that you have to consider.

Joseph: 

Yeah. All right.

Matt: 

Go ahead, Joseph. You got other, there’s

Joseph: 

There’s so many moving, there’s so many more moving parts when you already have, when you already have the car running. That you can’t assume. Are gonna play nicely together if you don’t have that conversation, if you don’t spend that time doing that the prep work to set yourself up for success.

Matt: 

Yeah. It’s what’s the example? It’s like you’re building the airplane as it’s flying. That’s a startup example. It’s, very similar. You got an airplane flying, you’re trying to completely rearchitect it in the air and keep everybody safe. Similar kind of metaphor there. Make this real for us. Do you got any. Examples, any use cases where either you’ve seen an NVR type of approach for great, or on the flip side, a fail where a company maybe did not take this type of approach?

Joseph: 

Yeah. I was thinking a little bit about this before we, we talked and I, I had a bad experience with one of my online banks and they maintained some of saving accounts for my family. And they created a new product that was anyways really great, but I think they, they took an MVP approach to replacing an existing product and they sent it out the door where it could do the absolute bare minimum of this product, right? It was a savings account, it took in money, and it let me take money out, and it showed me my balance, but it couldn’t do any of the other things that the old savings account could do, including set up a joint owner. Yeah, so I was I used this to move money around to, to handle banking with my wife, and we were unable to link her, ch her accounts to this savings account, and it completely defeated the purpose. Of creating the savings account, which was a place to to bridge the other accounts that we had. And it was incredibly frustrating because we had a, an established process that we were using based on the tools that they had made available to us. And by cutting out that core tool when we needed it to upgrade that account, we almost walked away from that, that banking institution because they made it impossible for us to do the core workflow that we had. We had dreamed up. And I think this touches a little bit on that idea of. They may not have expected people to use their savings accounts to bridge a couple of checking accounts, but that’s how we were using it. And that doesn’t mean that we’re not gonna be impacted by their change in the process. And it really caught us out and it was really a very disappointing and frustrating experience for us.

Matt: 

We’ll keep the name of the bank protected and un undisclosed. But yeah it’s, they didn’t understand your particular needs and I’m sure what they delivered. The new thing was probably, Great shiny, awesome functionality, but it missed that core element that you as a particular user group, I promise you probably weren’t alone. I bet you there were other people in that same scenario, customer segment Yeah. That had that same use case in terms of how they used their solution.

Joseph: 

Quite possibly. Yeah. It was, but it was a clear instance. Somebody getting to that bare minimum functionality that works if you don’t have anything to compare it to, but as soon as you got a comparison point, you’re like, I can’t go backwards. I can’t go backwards.

Matt: 

Awesome. That’s great. And I think we’re about at a stopping point here, but MVR, so what are those core principles you’re competing against your own product? You prioritize your most value, valuable user, prioritize incremental enhancement over your big bang. Identify the most critical workflow. Understand how users are using your product. They got those paperclip products processes out there and improve the, the function, experience and technology of it. And then we talk through the different approaches as well. And, we got some resources out there. You can find ’em in the show notes if you want to go a bit deeper. And I think, Joseph, this is one of those topics where we could go down a rabbit hole on an entire episode, on any number of these areas, and we probably will in the future. But appreciate you having you on the show. I know folks can probably find you on LinkedIn. That’s probably the easiest spot out there. But thanks for joining us today. My pleasure.

Joseph: 

Thank you for having me.

Matt: 

Thanks Joseph.

Matt Paige:

Today. We’re going to get into a topic that takes true courage, guts, and above all else, humility. It’s an approach used by some of the most successful product builders, and it provides the promise of quick iteration validation and getting closer to the almighty product-market fit. To make it real. I want everyone listening to close their eyes unless you’re driving. Let’s, let’s just pretend in that case, but take yourself back to high school. Imagine you’re being asked to perform in front of the entire school, but you have no time to prep, no time to practice, no time to study, no time to perfect your performance and work out all the kinks. And you have to start right here, right now on the spot.  Now, imagine that’s amplified to the entire digital world. And PS everyone’s got an opinion and they’re able to comment with their thoughts, good, bad, or ugly. I’m talking about building in public, and that’s exactly what we’re gonna go deep into today with Rui Nunes to explore and deconstruct this approach to product development and discovery, and understand how you can start using it today. Rui, welcome to the show. Welcome to Built Right.

Rui Nunes:

Thanks for having me.

Matt:

Awesome. And Rui, just for context, is a growth and product marketing expert. 15 years of experience recently led growth marketing at OnDeck and product marketing and growth at Flybits before that. But what I’m most interested Rui is you’ve been building it public lately. You’ve been building a tool called Social Digest, and it’s kind of having some pivots and interesting stories. But, but that’s what I’m most interested to get into, is the story behind this. You know, you’ve experienced learnings, failures, successes, pivots, iterations, uh, which just makes for the perfect story. Uh, but all these things in such a short period of time. But that’s, that’s the beauty of building in public, is the shortened duration of feedback, right?

Rui:

Yeah. It’s, uh, like, I, I guess I should preface this by saying that whatever I say today should be taken with a big grain of salt, uh, just like anything else that you hear online, uh, and their individual experiences, right? So a lot of the stuff that I learned and how I learned it is likely gonna be very different than folks listen to this and how they learn, and, um, and then what they take away from it too. But yeah, it’s, it’s been a wild ride over the last, you know, I think 90 days of just building in public. Um, and, uh, but it’s not new, like building something for me isn’t new. I’ve, I’ve tinkered around with stuff, uh, a lot and, uh, saw what some other folks, uh, we’re doing online and how they were sharing their journey. I was like, I, why not try this out? Put my ego aside for a little bit and see, uh, see where this takes me.

Matt:

And that’s the biggest piece. I think it’s that humility element. You gotta have this element of humility if you want to even start on this approach because it’s hard. Like I, I, I have trouble with this today, like putting myself out there, uh, in, you know, just putting something out there that’s, that, you know, like the saying goes, it’s a shitty first draft, but you gotta start somewhere, right? 

Rui:

Yeah. There’s, there’s a lot of stuff that, um, so I guess, you know, when you build in public, one of the hardest things, like you said, is being able to have a little bit of humility, at least, is, is understanding that you’re sharing stuff that isn’t done and you’re gonna have other people see how you think and what you think at any point in time before it’s polished. Um, just like your intro, right? You don’t have much time to prepare for any of it. You just essentially share what you think, you know, at any given point in time. And that can either be a good thing, it could be a bad thing, and things can backfire. But to be completely honest, unless from what I’ve seen firsthand, unless you have, you know, hundreds of thousands of followers and have built, and, you know, this, this like portfolio of businesses, nobody really cares what you’re building. Nobody really cares what what you’re saying. And, and I mean, I use that loosely, right? Cause obviously there are some people that support, you’ve been a great supporter of the stuff that I’ve been doing and working on with Nick too, and, uh, appreciate that. But it’s, you know, you’re not gonna get this like tribe that’s gonna come after you and tell you to stop doing what you’re doing.

Matt:

Yes. Yes. And that, that’s such a great point. Cause that’s one of the blockers, I think, to people actually starting, just starting in general, take throw building in public out the window is that fear of rejection of, you know, everybody on name your platform is gonna just, you know, viewing it under a microscope, everybody’s so focused on themselves half the time. Um, but maybe take a step back, like how, how do you define what building in public even is? Like how would you define, uh, you know, kind of a base level, what, what is building in public? 

Rui:

Yeah. I don’t know if I can come up with any sort of Webster’s level definition of what it is, but the way that I interpret it at least is sharing your progress as you’re building something. And, and, and the, something can be, you know, interchanged with any other word you want, whether it’s a course a, a, a book, um mm-hmm. you know, a product, a software product. It could be literally anything. But the premise is that you are sharing your progress before you get it to a point where, you know, you believe that it’s the best or it’s perfect. Which obviously that’s never attainable. And I think that’s, uh, where a lot of people go wrong is they get the project to, you know, 90% done and then the, then they don’t share it because they’re worried that it’s just not perfect. So that’s the way that I see it. Obviously a little bit of a rambled definition of building public, but, you know, know, I guess the TLDR is, um, just sharing, sharing your progress to building the thing that you want to build.

Matt:

Yeah. And you know, so many folks view this as like a, a marketing growth hack type of thing. Uh, but at the end of the day, and it’s truest form, it’s a mechanism of continuous discovery and early discovery. I know we were chatting about this the other day, being able to get what you’re building out in front of people and having them either using it, providing feedback, that’s it. It provides for such quicker iterations on what you’re building, uh, before you go and, you know, sync a ton of money into something or go too far down a path of an idea. Uh, but, you know, I think that’s the interesting piece to it. It’s not, it’s not just some like marketing ploy type of thing. It’s, it’s all about the discovery element.

Rui:

Yeah. If you, if you go down this path and you expect that your building public, um, approach is going to yield thousands of users, then you’re setting yourself up for failure. Um, yeah. Because building the, the, the desired outcome of building in public, or one of the desired outcomes of building in public is definitely the reach and distribution you can achieve as a result of sharing this stuff on social media. Right. And I’m gonna go on a couple tangents here just to provide contact. Um, so one of the big blockers of people building a region and audience online is generally writer’s block or some form of it. I don’t know what to write about mm-hmm. I don’t know what I’m doing. People don’t care about what I have to say. Right? You get that part of it, and then what you do is you essentially take that away and you say, Hey, look, I got a bunch of stuff that I do wanna write about because I know the things that I wanna talk about. So if you go down the path of saying, Hey, I wanna reach out to tens of thousands of people, you’re gonna see, and even me to this day, I, I have posts on LinkedIn that have five people that like it. But now if you reframe your objective of why you’re building in public from, I want this massive reach of distribution to, I want to draft and write about my progress, right? Almost think about posting on LinkedIn or posting on Twitter or whatever else, or a newsletter as a way to write your memos that explain the progress you’re taking and being able to go back to it over time to, uh, to reflect on the things you’ve done. And the byproduct of that is ultimately sharing progress with other people that have similar interests as you, with a similar path, uh, uh, that, that you ultimately have.

Matt:

Yeah. And you, you, you mentioned the word memos. I think that’s an interesting way of putting it, cuz the way I see it’s like this, the beauty of, it’s, it’s this story that you’re watching unfold before your eyes, but in the truest sense of the word, you don’t know what the outcome’s gonna be, right? I, I see you putting these things out here, building in public, um, and it is like that, it’s kinda like little memos, little like diary entries along the way, but it’s a story like, we, we don’t know where this is gonna end up. You even the person building don’t know where you’re gonna end up. No idea. Which is the, the coolest part about it. Like, um, and it’s, we’ll get into this in a minute. There’s several folks I’m kind of following that are doing this right now, and it’s, it’s just like, you know, I get my popcorn and gotta watch, watch what’s going on as, as they go. But, uh, I’d love to get deeper into like, you’ve been building this tool called Social Digest as a, let’s, let’s tell this story. Like, take me through kind of the starting point. You know, you’ve had ups, downs, uh, learnings along the way. Start to like tell this, this story for us. Um, and it’s been a short period of time, what you mentioned, I think like 90 days of Yeah. Starting building this, right?

Rui:

Yeah. So I guess, um, maybe we can decouple two parts of sort of how this story, how the story started mm-hmm. between people that know what they’re doing and people that don’t. So I’d put myself in the bucket of people that have no clue what they’re doing, which probably the vast majority of people out there, right?

Matt:

Like I Don’t, which I love have, Yeah.

Rui:

I don’t have a background in product management. I’ve never managed a product at a larger organization. I figure this stuff out based on learning. Fortunately, I have a great group of people around me that, uh, influence how I think, uh, and develop, uh, better habits and learnings in the process. So what happened was had an idea just like everybody does, right? Mm-hmm. And they say, Hey, cool, like, I wanna do this thing. It wasn’t the first product that I built. I’d already built stuff before. Uh, and I decided that I wanted to put something together and I wanted to mess around with bubble a little bit more. So the NOCO tool mm-hmm. And I wanted to explore that cuz I enjoy building stuff. Yeah. And I reached out to some folks and then I chatted with Nick Bennett and I said, Hey dude, like, this is what I’m thinking about doing. And he’s like, I love it. And the hypothesis was that, um, the, uh, we all understand that the, the limitations of LinkedIn’s algorithm and the fact that we miss LinkedIn posts regularly, and if we can find a way to make sure that people don’t miss this post, they’re gonna be willing to sign up and pay for access to this, so on and so forth, um, in advance. Understood. There were risks with this process, as you should always understand before you go down a path. Um, but thought that I’d be able to solve those risks along the way and find solutions for it. Mm-hmm. didn’t want to, you know, hurt, um, uh, or didn’t want to negatively impact momentum, um, just because there might be some risks, uh, to be seen. Obviously nothing that was detrimental to anybody’s wellbeing or anything like that, but there are business risks that come as a result of building something that’s on another platform.

Matt:

And, and just to like pause there real quick, this is a big point here. Uh, and we, we see it with customers a lot. Like there, there’s no way to solve for all risk before you get started. There’s this concept of you have no knowns the things, you know, no unknowns. Like I know there’s some stuff I don’t know, and then there’s unknown unknowns you don’t even know that you don’t know yet. And that’s a big hurdle a lot of times with our, even our clients, is you can’t draft a perfect requirements document before you start a project. How you start is exactly how you said you have a hypothesis, you have a desired outcome. And yes, there is, you know, ideas around what you’re building, um, but, but you can’t be, you know, handcuffed by the fear of the unknowns. And that’s powerful. I think just starting from, it’s such a good example of you’ve gotta start to start to uncover some of those risks.

Rui:

Yeah. And, and there’s also fears of the unknowns. So that’s a great point. But in this case specifically, there were definitely, uh, you know, unknown unknowns mm-hmm. but there were definitely no known risks, right. For example, yeah. Pulling posts from LinkedIn and being able to share that in an email that was a known risk. We understood that that’s something that LinkedIn probably looked at and say, Hey, we don’t, you know, we prefer that you don’t do that. Uh, which is why tried to essentially find ways to get API access and stuff like that to LinkedIn, uh, in order to do that. But because we’re small, we’re still figuring it out. There was no reason why we couldn’t get the ball moving start to bring users into the platform and figure that problem out later. Obviously if you’re a larger organization, that becomes a bit of a, a liability, but because when you’re starting out, there’s, there are tons of businesses out there that started out with scotch tape and glue and scraps of paper mm-hmm. until they validated and then they scaled up from there. That was essentially the same, using the same principles of just full M v mvp didn’t really know what was gonna happen and found ways around getting it done.

Matt:

Yeah. And when you think of the built right mindset, the way we think about it is you gotta build the right thing is one side of the equation, and then you gotta build it, right? I’d say to your point, you know, don’t get hamstrung on the built building it the right way Concept in terms of like maintainable, scalable, secure, you know, all these components that building the right thing is so important if it’s greenfield new idea type of thing. And it’s, it’s hitting on these core elements of is the thing valuable? Like, do users actually find value in it? Is it gonna be viable for your business? Meaning, can’t I make money off this thing? Uh, and is it feasible to build? And this is like, you know, Marty Kagan’s one of those big, uh, um, what’s the word? Not disciple, but just somebody we look up to in terms of how we think about product development. There you go. Right there, right next to, yeah. Yep. And, and that’s a perfect, uh, segue there, but I’ve loved how like, you know, I even like copied some of your LinkedIn posts going along the way, but I can see where you’re intentionally going about this in terms of checking off the boxes of, you know, is it valuable? Are people finding value in it, is it gonna be viable? So maybe like, take, take me down that path and however you want to go into it, but these different components of those core elements.

Rui:

Yeah. So let’s talk about feasibility, right? Feasibility we touched on, right? Can we do the thing that we want to do? Well, I think if you look at it from, uh, a best practice, the answer is no. Right? There’s no api, direct API access to get the information, information that we had, but we had a way to make it feasible. That’s kind of thinking outside the box, I guess, as they say. Mm-hmm. is like, are there other ways of doing the same thing? And there were other ways of doing the same thing, which allowed us to validate. Yeah. Was it viable? Well, now we know it wasn’t, but before we didn’t know if it was viable because we still had the hypothesis, which we can get into after in terms of viability and Yeah. Um, and figuring that part out and what was the next piece Valuable?

Matt: 

Like, is it actually valuable for the users? Do they desire the product? Do they find it useful? Right.

Rui:

Right. So yeah. So, and, and the second one is, or the third one is, is is arguable because mm-hmm. It, was it valuable? Well, apparently from user I abuse it was valuable. People said that they enjoyed it. Um, and I guess for context, which might help, uh, add some colors, that Social Digest essentially sends you an email every morning for the posts of people that you follow from the day before. So if mm-hmm. you were posting the day before and I was following you, I would get a recap in my inbox with your LinkedIn posts where I can click on it, go back to the posts. Mm-hmm. that’s essentially how it worked. So people said they loved it because they lost a lot of posts. Yeah. Because the algorithms don’t see it. So was it valuable? Yes. How valuable then is really the, the metric that you kind of have to look at, which gets into the viability factor.

Matt:

Yeah, no, that’s, that’s exactly right. It’s like, it’s that sliding scale in essence, like valuable to what extent, I think I even had that specific post saved, but, and maybe this gets into it, um, take me through that. It’s, it’s almost this willingness to pay type of component. Now, I know you, when it’s some interesting learnings through this building and public process around willingness to pay. Some people will say one thing versus another, and take, take me down that path.

Rui:

Yeah. So as a product marketer, I guess willingness to pay comes second nature. And I skipped that step somewhat, building in public which in hindsight was probably not a good idea, but still learn, learned, uh, learned my lessons a different way. Um, but the, obviously you can’t run a business and, and this goes into sort of the post the note that I released, uh, I think a couple weeks ago now that sort of explained the outcome of Social Digest. But you can’t run a business with zero revenue unless you have mm-hmm. a bunch of capital that’s been injected into your business. Yeah. Personally, I had no intentions ever of, uh, receiving capital, uh, or getting investments or anything. I wanted to boots bootstrap this thing, which I did. I wanted to explore it on my own, uh, and then build, um, you know, I wouldn’t, I, I wasn’t expecting to build a massive business, but expecting to hopefully achieve something with it. Mm-hmm. And what happened over time is you, you start to talk to users and you get a sense of, okay, how valuable is this to users? Uh, what do you think? How are you using it? Right. Tell me, tell me how you’re doing this before. Right. Rachel’s typical discovery questions to get a sense of where this product can go and, and how it can evolve and what features are missing, and maybe we build too many features or not enough, et cetera. Mm-hmm. and a lot of the feedback was positive. It was great feedback. It was just like, yeah, like loving this. I missed if we missed the social digest post. We get emails saying, Hey, I didn’t get the Pope. Like I didn’t get the digest this morning. Yeah. So all the indicators pointed towards success until we put a Stripe integration and asked people to pay for the subscription. And that’s really where rubber rubber hits the road, right? Like, you need to make sure that your product is good enough, that people have enough of a need and a want for it. They, if they’re willing to pay for it mm-hmm. And in this case, even though people enjoyed using it, they didn’t need it enough in their day-to-day to, um, to end up paying for it.

Matt:

No, that’s, that’s super interesting. And, uh, maybe, maybe talk to me about through this process, what, what’s been the hardest part about it, uh, to you, this building in public motion?

Rui:

Well, I think that, um, I think the first time around is probably the toughest mm-hmm. Um, and I think that there are multiple reasons why it’s a difficult, the first time you go through it, one is you’re still figuring out what you’re doing. Mm-hmm. you’ve never done, done it before. So you don’t even know what kind of posts you should, you should post or how you should post them or what things you should talk about, right? Mm-hmm. So you’re still really figuring out what to write about publicly first. Sure. Just in the built in public part of the business. And then you gotta know what you’re building. And if you’ve never built anything before, you are gonna make an infinite number of mistakes. And I think even if you have built stuff before, you will still make a ton of mistakes. That’s right. And there’s that lesson too. And then there’s like a functional lesson of like, running the business part of it. It’s like, how long can I sustain building this thing out? In some instances, folks will build this, this stuff themselves, which is great. I built the no-code element all myself mm-hmm. but I also had to hire a JavaScript developer to help build the Chrome extension and components that added more features to, to to social digest. 

Matt: 

Mm-hmm. Yeah. So what’s crazy about that, I don’t know you picked up on this, but there’s a meta element to this cuz you literally are building in public of building in public, if that makes sense. Yes. Like you, by building in public, you’re learning how to build in public. Yes. You’re doing that motion to learn how to do that motion. So it’s, it’s kinda like a meta, uh, thing right there. But that that’s, that’s really, really cool. Um, the, tell me this though, uh, what’s one thing you wish you could go back in time, not very ba far back in time, you know, three months ago, and tell yourself, uh, when you started, 

Rui:

Um, ask people, ask people for money faster, get paid faster. I think that peop how should I say this? I love building products just as much as the next person who love building products. Mm-hmm. And if you’re doing it as a hobby and you have no expectation that it becomes something and you’re investing relatively close to zero as possible in terms of cost, then you can do this as much as you want. You can go build a bunch of tools mm-hmm. um, launch a bunch of stuff. A ton of these things will fail. Maybe one will make you some money. And I think that that’s fine. If you do have an expectation for this thing to make money and you are investing capital, or you’re investing a significant amount of your time, then you have to understand how you can generate revenue as quickly as possible. Yeah. And that if that means you have to scrap your idea super, super early because you couldn’t find a way for people to pay for it, then that means you scrap the idea super, super early. Yeah. Um, that’s probably the thing that I learned the most is that discovery is important. Understanding the problem, what the problem is, what the, you know, the customer need is mm-hmm. um, but also being able to identify that willingness to pay element and merging those two as fast as possible so that you know, that you can scale out a viable business over time in a sustainable business over Time.

Matt:

Yeah. It’s interesting. It’s, you know, when, when you’re bootstrapped, it’s almost like the luxury of being bootstrapped to where you’re, you are forced almost to iterate quickly. Learn quickly, versus like something that’s well funded or a large company. You know, they can take these pauses, they can, uh, take a long time to, to launch something ultimately to, to learn the same result in a lot of ways. I think this is the beauty in what you’ve done. We’re talking 90 days, you started building a public, you’ve learned a ton. You’ve kind of, you know, pivoted from the original thing and kind of get into where you’re pivoting next. But in 90 days in, in minimal investment, relatively speaking from what I’ve seen from a lot of companies that will spend months, years putting something out there without answering some of these critical questions. And it’s painful cuz you spend exponentially more money, more time, more resources, and you get the same result, uh, or, or, uh, not superior result because you’ve, you’ve learned less along the way. Right.

Rui:

Right. Yeah. There’s, there’s a lot of wasted capital and a lot of wasted resources that go into building products and that happens everywhere. Mm-hmm. I’ve seen it firsthand. The companies that have worked at, um, or you go down development cycles that are way too long, building things that you don’t know if people want mm-hmm. and then at the end you’ve released a bunch of features that people never use and or never see the light of day because you’ve pivoted before you even launched them, man. And I think that’s the Manage Building in public, is people get to see what you’re doing and you can quickly pivot. And even though I mentioned like 90 days before, from the point in which we launched and accepted wait list registrants to the point in which we published that we’re going to essentially phase out Social Digest, it was probably just over 60 days at most. Yeah. And we just, I just essentially let it run. I think it actually technically turns off in two days, um, but let it run for a total of 90 days. But yeah. And if you can run it even shorter, even better if I, I ideally in the future want to be able to do enough discovery, validate financially, validate the product or service or whatever it ends up being within the first 30 days at most, the first 30 days. Yeah. And if you can’t then move on to the next thing.

Matt:

No, that’s great. I think that’s such a big learning. And what’s interesting, you’re kind of pivoting this other idea, but it’s even more kind of a, um, you know, uh, MVP of an MVP type of thing. Like you’re just lever leveraging kind of a plug-in as the initial concept, which is a lot lighter weight, a lot easier to get out there. So even within the short period of time, the thing that’s you’re pivoting to is, um, you know, something you can get to those answers quicker that are so critical.

Rui:

Yeah. Yeah. And I guess to to touch on that is, yeah, when, when I was doing user research, uh, and talking to users of Social Digest, I was getting an understanding of, um, what their current process was, uh, how they use LinkedIn and some of the challenges they faced. And, um, one of the things that came up often was folks were interested in knowing who was influencing the influencers that they were following. Mm-hmm. Um, and they wanted to get a sense of who those people were so that they can also follow them and learn from them. And there was another use case in which people were using it for prospecting reasons, right. I want to go and I wanna reach out to more people. Obviously we all know that there’s a bazillion prospecting tools in the market and they cost, you know, infinite amounts of dollars and, you know mm-hmm. it’s, it’s, it’s not really cost effective for many folks and in other cases there’s a monthly subscription, so on and so forth. So, um, after Social Dive just decided, hey, why don’t I just create this quick spinoff of a Chrome extension that essentially allows people to go to anybody’s profile or engagement, uh, or on their posts and get the list of all the people who have engaged on it and allows you to essentially increase the number, number of people that you follow. Uh, or if you’re prospecting, it allows you to find people that are influencing your prospects so you can engage on their posts as well. Mm-hmm. and like you said, super mvp. Right? The way the tool works is literally open up a profile, this little slide out pops out as a scroll, the page, it populates all the people that essentially show up on the page on LinkedIn. And the features could have been infinite, right? It could have built a crm. Mm-hmm. I could have integrated chat G P T, it could have done all of these other things, but that wasn’t the problem. People didn’t come back and say, Hey, I want to, uh, automate outreach. It’s like, well, there’s plenty of tools taught I me outreach. Right. There’s plenty of tools told to do all what they’re saying is that I want to see who else is around, uh, in, on LinkedIn that I could follow, uh, or I can put on on my radar. And that’s what we did. And the difference between one and the other obviously is still early days for Influencer Circle and now starting to talk to more people. But, you know, social Digest in the lifespan of the product generated $10 and um, obviously still a small amount, but an influenced circle of 24, 24 hours generated 60. Right. So you see a big discrepancy between the amount of effort, money, and time put into one project mm-hmm. versus the other. And there’s no correlation between the revenue you generate, right? Like it’s really comes down to how big of a problem are you trying to solve because you can create something that’s super simple, very cost effective, and in some cases, in many cases don’t even require software. As long as it solves the problem and people find value in it and there’s benefit, then they’re gonna be willing to pay for em.

Matt:

Yeah. And it’s, it’s so awesome that comparison, that learning there, these kind of micro durations of how quickly you got with the second thing, you know, a week or two just getting that, that going. Uh, it talks about the, the beauty of constraints. Like constraints can be a good thing. They kind of force you to be scrappy and, and work with what you got. Tell me though, uh, besides yourself, who, who do you think’s doing building in public well or anybody in mind for you that, you know, you look at and say, Hey, that’s a good example of somebody doing building in public the right way?

Rui:

Yeah, I think you go on Twitter and you can find probably tons of those folks, right? I think Twitter’s pretty common in terms of finding people, building in public, whether it’s on the bubble side and people doing no code, but you can essentially look up mm-hmm. build, you know, building public on Twitter and find a ton of people. Like Peter Levels is the one that obviously comes up a lot because he’s built, you know, quite a big success for the products he’s been building and, and building in public for a long time now. So I guess he’s seen as probably the being on the press, you know, on the, on the, on the upper echelon of, uh, of building in public. And then you have mm-hmm. and then you have, you know, I was fortunate enough to work at OnDeck, uh, who I worked with, uh, KP who ran the No code, um, program there at OnDeck. And he, he even wrote a book around Build in Public too, which is awesome. Uh, and he’s a good follow. And I believe now he’s also getting active on LinkedIn and not just Twitter, uh, which is I think good for the LinkedIn community, but seeing more of this come out, uh, because I think originally it, it had just been, um, restricted primarily to Twitter and that’s kind of where the building public stuff, um, entered. But yeah, and Jay, uh, Jay decides also on, on LinkedIn and we were just chatting recently too, and he is building some stuff. So there’s, there’s lots of great people right now. Maybe it’s worth creating a list of who’s building in public. I don’t know, but there’s lots of great people out there, uh, to find.

Matt:

No, that’s a good idea. Maybe that’s, uh, something we’ll get in the show notes before we launch this episode. Um, but it was, even seeing the other day, darash who created HubSpot, he’s got this new tool called Chat spott related to chat G P T. It’s kind of, you know, adding that chat G P T element with your crm like a, a HubSpot. And he’s, he’s kind of doing the same thing. He’s got a very, you know, kind of rudimentary alpha out there, very similar to you. He’s talking about like cool little use cases of how to use it. I saw today he, he just bought the domain chat.com for like eight figures or something like that, but it’s this crazy like, transformation going on, which is a whole other topic around generative ai. But, you know, this is not reserved to, you know, the bootstrap, like there, there’s big name people doing this thing.

Rui:

Yeah. I think that unless you are, um, constrained by a p and there’s something there mm-hmm. that you don’t wanna obviously share. Yeah. Um, I, I honestly, I don’t see any negative effect of sharing what you’re building in public. Like, if anything it’s the, the opposite of that and mm-hmm. And I think when you, when you consider like, you know, folks like Nick Bennett and, and, and everybody else that has essentially built a, a decent size following on LinkedIn who have, um, earned this significant amount of distribution and often talk about what is the cost of paid media nowadays, right? The paid media cost is significant and how do you compare that to building an audience and having reach on a social platform? Well, obviously you’re not gonna pay virtually anything other than maybe your time to write stuff as long as mm-hmm. you’re writing reasonable content to continue to increase in that. Well, imagine if you do increase a workforce or build a workforce of folks who share what they’re building, uh, on online and on social media, well, there are a lot of great co uh, side effects that come out of that, right? Like, you share more about your product and more about your business. So it’s great from a career perspective and, and hire better people cause they’re gonna wanna work there and you’re building a better product cuz you had an open feedback loop, uh, which you talk about often, right? Like how do you get more people in and tell you exactly what you’re doing? Um, and obviously you’re gonna take that with the greatness all too social media, you get trolls, you know, that is what it is and we all know that. Um, but you do get that feedback and that input, which ultimately also, um, gets boosted to increase a broader reach of individuals, uh, online. And as we all know, if we wanna learn better, we should write about it cuz this, you know, to teach, to learn as to teach. Um, and as we write posts on social media, we are further cementing our learnings, uh, of what we’re building, why we’re building it, and our vocation itself as well as we post this stuff online.

Matt:

Yeah. Last topic here, we see a lot of startups, early stage, you know, type of companies, boots, trap companies doing this building in public motion. Why do you think we don’t see more of this in larger, more established companies? Uh, which, what’s your thoughts there?

Rui:

I don’t know, man. It’s, uh, it’s a, it’s a tricky one. I think that’s, I think it’s the same issue that you get around, um, why people don’t know how to use social media, um, and why bigger companies don’t know how to use social media. And we’ve all seen it, right? We got all these companies that all they do post social media, compos, all they post on social media is, uh, birthdays, anniversaries, holidays, uh, yeah. White papers, webinars, like the, the templates are all the same, right? Hey, pay attention to me and kind of double, double clicking back on what I mentioned before. We have folks that are doing this well and building a business or supporting business where they work. Mm-hmm. on the back of that, you, you, you start to ex increase your reach. Now if executives or whoever or whoever’s making the decision doesn’t support this, that is really up to them to figure out why that is the case. They’re, they’re leaving money on the table, they’re leaving opportunities on the table. And in some cases, you know, if we don’t know how to do something we tr try or at least tend not to go down that path. Maybe that’s a reason. But I think there’s probably unique reasons for different businesses. Like I mentioned, IP might be one of them, but that’s probably pretty rare to be honest. I think that’s a terrible excuse. Like, unless you’re working in some sort of generative AI with like this super complex algorithms that you don’t wanna share and all this other stuff like I get, or if you’re, you know, if you’re doing something on the defense side and you don’t wanna share that, like there’s, there’s, I think there are certain parts of, um, and even in those cases, to be honest, I’m sure there’s tons of stuff you could talk openly and publicly about You just like at open ai I mean’s secret stuff. 

Matt:

Exactly. Yeah. Yeah. I mean, open AI to a certain extent is doing this, you know, a lot of development went into it, but it’s, they’re quickly iterating through it. Um, it’s back to that, that discussion on constraints, they don’t have as many constraints and I think it is part of where the power lies. But, you know, I think it’s such a missed opportunity. Um, I think what would be interesting, these like companies with like incubator innovation type labs, like test it there that could, we work with several companies that have these kind of innovation labs and areas there where they could, uh, they could, you know, value by doing this building, building in public motion. So, you know, could you activate it there? I think that’s an opportunity, but, um, we just don’t, I don’t think we see enough of it in some of the larger companies. They, they default to the, the big splashy launch, you know, with the, the, the big marketing spin behind it and everything. And God forbid they do that and they didn’t even solve the right problem. 

Rui:

Yeah. I think that that actually, uh, surfaced another thought, which is also the optics behind mm-hmm. um, building in public, right? You the, when you think about building in public, it is a very informal, a very messy process. 

Matt:

Yes. Yeah. 

Rui:

Right? It’s, there’s no editing, right? I’ve, yeah, if it wasn’t for Grammarly, like the majority of my posts would probably be incomprehensible, right? Like yeah. Like it just helps to kind of figure stuff out along the way. So shit, they’re not a sponsor, I’m sure, but shit out’s a grammarly on that. Um, yeah. So there is a lot of messiness in building in public, and if businesses are set and, and, and their ways of really caring about optics in a certain way, then I can see why they’re, they have a, they’ll have a bit of an aversion towards this sort of messy publication. Yeah. And here’s the thing though, if you think about this, what’s, what’s the, the, uh, partner with Messi? It’s authentic, it’s authenticity. What do most big brands lack? It’s authenticity, right? So it’s it’s like right in front of their, their faces in a lot of ways. This is an opportunity to put humans in the process, to be authentic and oh, by the way, actually build something that customers want and learn as you go versus after you’ve sunk millions of dollars into it. Uh, yep. But no, that’s, that’s great. So, Rui, this has been an awesome conversation, love chatting on building in public. Where can folks, uh, find you? What’s the best way to find Rui out there? Uh, well LinkedIn is where I spend most of my time now, um, and posting and commenting as much as possible. So you can find me on LinkedIn. I’m sure you’ll probably include a link there somewhere, but I’m news. But, uh, yeah, I think that’s the, the biggest, uh, the biggest area we’ll have links to, like, everything that I’m doing and stuff.

Matt:

Yeah. No, and a, a worthy follow. Um, hopefully this gets more attention. Cause like I said, this is like a story unfolding. It’s, it’s something I look forward to every day, uh, the stuff, the knowledge you’re putting out there and kind of sharing as you go. So really appreciate having you on Built Right.

Rui:

Yep. Thanks man. Appreciate it.

Matt Paige:

All right, I am pumped to be joined by Andy Silvestri. He’s a true master of his craft, with nearly twenty years of experience in digital design, including graphic design, creative design, user experience, customer experience, and even product strategy. Even ran his own customer experience strategy and design firm, Field for about ten years before joining us at HatchWorks to lead our Product Design practice. And today we’re going to get into the crucial role UX (user experience) plays in making sure you’re building the right solution, which is a big component of Built Right. But welcome to the show, Andy.

Andy Silvestri:

Hey, thanks, Matt. Yeah, it’s a pleasure to talk with you as always, and really excited to dive in deeper. It’s funny you mentioned Field. That was like another life.

Matt:

That’s actually how we met back in the day, when you were running Field.

Andy:

Yep.

Matt:

Yeah, and really excited to get into this. We’re going to go deep on UX. Andy is a master of the whiteboard in the workshop, I see myself in awe every time I see him lead a workshop. I’m just in the back taking notes like how is this guy doing this? But you’re going to love this episode today. Andy. let’s start off like high level, the tipping point of us because it does get confused, but what is the goal of UX? What’s the goal of user experience?

Andy:

Yeah, it’s a great question. UX is used in a lot of different ways. A lot of people talk about UX this that. I think that there’s this general mindset out there that is all about making things easier for your end user, minimizing clicks, and that kind of things. But it’s really more about just I feel enabling an end user or user base, making them or giving them the ability to be successful, and your solution coming in doing what they need to do. Getting the information or getting to completion of a certain task and getting out. That’s really it, So obviously, ease of use is a factor of that, but it’s really more about enabling your user because complexity is a good thing sometimes. There are systems that need to be complex because they are trying to convey very complex ideas, or they’re trying to allow a user to do very complex tasks right. So it’s not just about how many clicks can we minimize or how can we combine things so that it’s in one place. That’s it again, just all about that enablement.

Matt:

Yeah, that’s great insight and that’s a big thing. It’s like everybody is trying to get to let’s make this super easy, super painless. Yes, that’s good but some systems can be complex. It’s like how do you make the complex simple? And that can be a difficult thing to do. That takes a lot of a lot of hard work on the research side of it.

Andy:

Absolutely, I think that you touch on a key point there. I think that obviously, the further you get in a system, the more complex it becomes, the more specialized whatever it is that you’re doing for your users is, the more you need to understand what your users need right, and you need to be talking with them. So the research component, which I’m sure will get way deep in because it’s where my passion lies, is a big factor to being successful into having that validation in understanding.

Matt:

And when we talk about the goal of UX, you’ve got different stakeholders. Who are you considering when you’re looking at user experience? I mean, people typically go straight to the user, but it’s more than that. Who’s in your thought process when you’re starting your research on the user experience side of things?

Andy:

Yeah, research kind of goes both ways in the sense that you do need to factor in key stakeholders of a business. There is that component of making sure that you know whatever you’re doing within the system, whatever you’re creating for your eventual end users, those could be external customers, hose those could be internal employees. That kind of thing right there, your user. But the stakeholders do have you business needs that kind of drive the foundation of a system, so to speak. So really making sure that there’s a consistent kind of narrative, or a specific kind of objective that you’re getting to, and having the buying of the business is a big part of it, so that has to be kind of factored into, “Hey, how are we kind of approach enabling our users?” As I was saying. What do we need to offer them? How are they successful? What does it mean for them to be successful and how does that refer back to the success of our business, our platform? Giving them this ability equals something on the business end and it hits the bottom line somehow. So again, having both of those threads from a research and understanding perspective is critically important, I think. And, again, part of the process is marrying those together into eventually what becomes the end user experience.

Matt:

Yeah, and you hit on it too. Like if you’re building a solution, it’s got customers. It’s got to be viable. It’s got to make money. You can keep continuing to offer this thing to your customers. That’s that’s that balancing act where, if you sway one way too much on the user side or one way too much on the business side, you can be in a bad spot. It’s finding that kind of harmony between the two. Your user and your business. Something your users love and something that your business loves in terms of viable and continuing to be self sufficient in the future.

Andy:

Yeah, having that feedback loop is critical, right? Because what you want to do is, of course, have continuous research and have it be built in to the DNA of how you’re approaching the user experience of your product or just your business in general. At the end of the day you want to have that feedback loop from customers and users. What are they saying? Okay, how are we kind of bringing that back up the chain and presenting that internally and either debating that folding it into existing initiatives or at least having some level of conversation around, “Hey, this is what our users are saying. We’re looking here, but maybe we need to be looking there because of what we’re hearing.” So again, it’s very important to have that as kind of a backbone of how you’re approaching creating the experience updating the experience, having the experience expand. All those kinds of things.

Matt:

Yeah, it’s a good segue into the next thing I was wanting to hit on. So you talk about a UX Mindset. It’s not just reserved for a couple of folks in the product designer or UX side of the house. It’s a mindset within the whole organization. Take us through that. What do you mean by a UX mind set? Go down that.

Andy:

There’s a couple layers to that. On the surface, there’s obviously the idea of, “Hey, we want to be presenting a good experience, right? We want to delight customers.” You hear that all the time. And then there’s this idea of, “Do the UX! Just UX it.”

Matt:

Just do it.

Andy:

Yeah, just do it. but I think that user experience mindset really starts with that kind of design thinking mindset as well, which is starting with empathy for your user, understanding what they need, getting inside of their heads. Walking in their shoes. A lot of customers a lot of times just assume they know. “Oh yeah, we get it. That’s cool. We’re going to just do this because we know that this is what customers want.” You start to peel the onion and it’s something completely different. In a way, you start with that empathy, you understand, kind of walk in the shoes of your customer, your user. Then, from there, you start to define and you continue to iterate and prototype and present things back to the customers and continue to narrow the focus or get more and more validation. And so really like that’s the cyclical mindset. That’s really what you have to have throughout the process of creating your product or your solution. You can’t just go in with assumptions and lean on those. You have to have that validation. So the more you can just have that as as a directive, “We’re going to empathize. We’re going to get inside the head and try to understand. Not just ask them questions, but try to understand who they are realistically. Who is my customer?” There’s a lot of emphasis sometimes put on creating persona out of your surveys. That’s a great tool for organizations that need to have kind of that crystallization of understanding, but at the end of the day it doesn’t have to be that cut and dry or that polished. You can just still have a running understanding in a narrative of who your user is, just by talking to them and documenting that and continuing the process. So that’s really what I mean about it being a mindset. It’s more about, we’re not just here to do the UX, meaning we need some wireframes, we need some user flows. We need to understand. We need to have that understanding be the backbone of what we’re doing.

Matt:

Yeah, we weren’t planning on going here but give me your hot take on personas. You know, Jill. that’s forty years old and likes comedy.

Andy:

I think they can be very valuable when especially when you have a larger organization that needs to rally around an understanding and you need to disseminate this persona to multiple business units or multiple groups within an organization that might not be very closely tied to product or design or the management side of a product. I’ve seen this happen with typically much larger groups, but you can create marketing campaigns around your personas for internal use. You can do that, and I think it’s very valuable in those organizations when you’re trying to disseminate a particular perspective on your customers. Just so it’s top of mind for your employees and the people that are working within your organization. So again, they have their place. They do absolutely have value. But it’s not necessarily something that always has to happen. There’s this idea that, even in the early phases, of just user definition, meaning that, “Hey, our users are this and we have a dossier on them.” It doesn’t have to be to the level of its personified, as Jill from Vaudowa who does this or that. To your question, they do have very real value and I think they’re an important part of the overarching universe that is user experience.

Matt:

And so much of it’s like, “What is their goal? What are they trying to achieve?” It starts getting into that path of your jobs to be done. That’s where you start getting those nuggets of how you architect and design a solution for your users.

Andy:

And the cool thing actually, just to kind of keep digging on personas for a second. The cool thing is that they can evolve so you can have them established one way from an initial body of work, and again, if you’re keeping that mindset, you’re continually doing your research, are continually talking to customers, you can bring that back into reevaluating your persona. You might have started with saying, “Hey, these are the ways we thought they sliced out. Maybe that’s not true anymore. Maybe there’s more gray area than you initially thought. Maybe there’s more personas in your platform? Maybe there’s more ways in which they’re using our platform or your solution that you never really thought of.” You’re learning through this continuous discovery and research.

Matt:

You mentioned a word a little bit ago. Assumptions. Which, it’s great to have assumption. You gotta have some ideas of hypotheses. I think where we see breakage happening is when you’re not validating the assumptions. Take me through that. The importance of validating because I can’t tell you how many times we’ve been burned by this in our past. You think you know your customer and you have an assumption and you run with it, and then you build something and you find later that it’s not the case.

Andy:

Yeah, right, and again that’s a very common and it’s a very attractive way to approach creating something. Let’s go as fast as we can. We know what we know. We think we know right. And it’s typically, there’s a perception that to do the proper level of research and getting the right data to validate your directions will take a very long time, or it will be a very significant investment. But that’s typically not the case. Realistically, going into even a handful of conversations with customers or your end users can be extremely valuable to getting some level of that validation. There are a lot of thoughts out there around what’s the proper level of research you need to be doing. How many people should you be talking to? Going all the way up until you need to talk to at least twenty-four or thirty-six. It’s usually divisible by twelve. I don’t know. But in my experience, honestly, even after doing some pretty pointed interviews with again your target customers – that’s a pretty big caveat that you have to be targeted in who you’re talking, which comes with, you kind of get there from the earlier definition. But long story short is, if you even talk to half a dozen people, six to seven, you’re going to start to see themes emerge that should marry to helping validate your assumption or disprove them right. So if you talk to six people and they’re going one direction and you were thinking the other, it’s a pretty good indication that you talk to six more or twenty more or fifty more, you’re going to get that same kind of trend. Atthe end of the day it’s a lot about how quickly or how efficiently or how easily, so to speak, can we get people’s feedback. Can we talk to them? How can we access our end user? Right? Is that a fifteen-minute phone call? Is that, in terms of if it’s an internal project, can we talk with co-workers who are using this platform or this system? Can we spin up something that’s a really low effort, kind of objective, and get them into just the conversation? That’s real. I think that it doesn’t have to be some big long drawn out we got to playing out six months of research and we got to do this and that. But as long as you’re saying on top of it right, and you’re not just being like, we did six were good, you have the ability to continue to do that right over the course of your product. I think that’s enough to just keep it going right to always have that touch point and have it again and be like we’re talking about with being in mindset. Have to be a routine part of your mindset in your approach and your process.

Matt:

Two separate threads here that I think are really interesting. the first one being you mentioned a minute ago. When people think of UX and starting like they want to hit the ground running. They want to start getting value. They want to start building, and a lot of people think that the UX side of it is going to slow them down, but talk though that. It’s actually a way to save some money in the long run, if you can validate some stuff up front versus building, rebuilding, and all of that.

Andy:

Right, like I was just saying, I think the temptation is, let’s save the money. Let’s start quick and let’s just start doing things and creating things right. Ninety-nine percent of the time, if you do it that way without any sort of validation, any sort of feedback or input from your end user, you’re going to be changing things sooner than you think, so you’re going to lose that money that you saved up front in the know by redoing or having to rebuild, or god forbid, completely scrap something that you thought was was the right way right. That’s just one way to look at it. You’re either making the investment upfront or you’re making the investment later. And it’s much more painful the further down the path you get.

Matt:

Yeah, the pain definitely escalates. So in that same vein, UX can be daunting for a lot of folks that are looking to take this on, and bring it into their organization, But it doesn’t have to be. How can folks get started like it’s It’s the MVP of UX for an organization that has not done it in the past? It doesn’t have to be this big crazy thing. Talk us through where’s the best spot to get started in kind of a minimal way.

Andy:

Yeah, I would say the best place to get started is to really do that look in the mirror and say, “How can we define our users or end users?” Really, just start with that definition, because they’re going to be the ones who are ultimately determining whether or not, you’re successful. At the very least, drafting that up and saying okay, this is who we’re going for. This is our core target. And if that’s multiple targets, that’s fine. Doing the same process for multiple users is fun. Getting that as a baseline is something that’s really just about again that look at the mirror. The internal kind of let’s do this, let’s make sure that we have the right understanding of who we’re doing this for, who we’re creating this for. Sometimes there’s existing users that you have, of course, that’s easier for certain organizations than others, or you’re starting from a cocktail napkin. From there, then it’s about what are we enabling them to do? Like I was saying earlier, What are the jobs to be done, so to speak? That framework that you can use. What they have to be doing in this platform or the solution in order to be successful. And can we start to think about what that is once we’ve defined that a bit? What is that in terms of prioritization? How do we look at that from like a phased approach? We know that there might be a dozen things that this user needs to accomplish to be successful in various ways. But what’s the most important thing? That eighty-twenty rule. If we were to, frame up this, this twenty percent of a platform or solution, what would give them eighty percent of the value of it? That kind of mindset I think is helpful. And all of this is really whiteboard work. It’s nothing that needs a super fancy degree or anything, It’s more conversation and analysis and retrospection. And making sure that all the stakeholders that are involved in this product are present and have and have input. From there you start to get some traction right around who this customer user is, what their needs and pain points are. You might. even at this point, if you can bring in some conversations with those types of individuals to again start that validation cycle. And from there then it’s like this is framing up as a thing that makes sense. And we’ve started again to get that validation, that understanding, were aligned internally. Our users, who we’ve kind of defined have given us this level of input. We can start to look at, bringing in the more formal process of let’s start to frame up things to show them as prototypes or as wireframes or whatever it might be. Let’s do that high-level conceptual work and continue the process. So you’re just kind of ratcheting up the fidelity a little bit. But like it’s the same kind of process. Okay, We started with words. We’ve gotten validation. We’ve gotten people involved. It makes sense where we’re going. Now, let’s start to show them things right and those can be crude sketches. Doesn’t have to be using fancy high and tools for this are a million tools out there that allow you to do this kind of work. You know, That’s really the process, and just kind of again, like I was saying, ratcheting up the fidelity getting a little bit more and more clarity and invalidation as you go along and keeping that going until eventually you’re like “Hey, this is the system and you know we have a very clear picture of what we’re going to do first.” And then you start to get into, of course, implementing that thing. So that’s where user experience turns into interface design. But it also then starts to become a delivery and a development kind of process. So but you can prototype all the way through that which is great. And you can do so in a very crude manner. It’s not like there is a certain level of fidelity that you have to have to show people of a concept. I guess the one caveat there would be that. Just make sure that whatever your user is, who, whoever they may be, that they can consume what it is that you’re presenting to them from a concept. Sometimes your user base is savvy enough to understand what you’re trying to insinuate through a very crude wireframe or a very rough sketch of something. Other times that might not be the case right, so you might have to be more successful going into those kinds of scenarios with a more polished kind of prototype. That’s maybe more clear and might be more direct, but again, that’s part of you as an organization starting from that basis of understanding your user and saying what do they need to be successful? And how can we present them options or concepts and ideas that make sense to them so that we get the optimal amount of feedback and in the right way.

Matt:

Yeah, so many good points there. You don’t need like this focus group thing that’s this big, expensive drawn out. Just talk to some customers. Start there and I don’t know about you, for me, it’s all about the qualitative. You can only glean so much info from a survey. Yes, they have their purpose, but the qualitative talking, picking up on these nuances. I’m almost thinking we need some type of guide or tool because you see Andy work a customer interview, it’s a thing of beauty. So maybe by the time we air, this we’ll have something out there for that. It’s powerful and it’s this process and, this is where it gets interesting, gets back to that concept of the mindset. All right, this is not a one-and-done thing. This is not something we do in the beginning. All right, we’re done. Move on to the next thing. It’s this concept of continuous discovery and continuous delivery on the build side. This is a Tim Herbig thing that I know you’re a big proponent of. But it becomes part of the organization’s ethos, it permeates the entire organization. But talk us though that concept of continuous discovery, its touchpoints throughout all of the process.

Andy:

Yeah, absolutely. I think that’s what’s interesting there is that you know, at any one time you have those two threads going, You have discovery and delivery, so to speak, and at any one time, they’re kind of like one hand washes the other. Discovery is being done in a way to inform the next phase of delivery. You know, delivery is being done in order to create artifacts and elements that could be used for discovery. So it’s this interesting kind of ebbs and flow. And again, the intensity of one or the other is dependent upon where you are in the life cycle of the product. I think it’s like you were saying, this mindset, having that understanding of how are your teams structured. When you start to get into teams like we have at HatchWorks where we have a discovery, in a definition, product team working in tandem with a delivery team, from an engineering perspective, that team is one team realistically. And every single day there is an ebb and flow of like, What are we using to inform how we develop the next phase of the work? What are we hearing from research that we’ve been doing with customers to inform how we pivot? You know, if we need to, or if we’re enhancing something, or you know, creating something new, and similarly it’s you know, from a delivery perspective of like Hey, we can. we can now demo this thing right. We have this thing ready to go that we could use as an artifact in validation. Or it might spin up things in a sandbox to use as a prototype. You know, so yeah, I think that that’s It’s really interesting. you know, when you get into kind of the fluidity of that kind of team and that kind of process, it all just becomes the cyclical thing where you’re kind of inputing in what you get. It’s going through this idea of defining and building and validating. Then you’re delivering and continuously doing that. So yeah. It’s excellent. That’s the way to do it. 

Matt:

You always hear about the CI/CD but that continuous discovery piece often gets overlooked because it’s not the technical piece of it. It’s that dance, that tango between problem space and solution space, and it’s ever kind of expanding and contracting and expanding and contracting. And that’s the best products out there. That’s the process they follow. It’s not this waterfall manner.

Andy:

Yeah, absolutely

Matt:

One area. So you think of UX… a lot of people are always thinking like I got this back of the napkin idea of drawing it out. You’re going to design this thing and build it, but when you’re modernizing something existing, you’re re-designing something existing. UX, it’s one of the most critical parts, right because you’re users are typically using your system in unintended ways. Ways you don’t even know. Go a bit deeper in terms of like what to consider when you’re re-designing something existing. Same principles apply. but there’s some nuance there.

Andy:

Yeah. it’s great and it’s a really fascinating topic because to your point when you have something that’s established and you start to peel the onion, you go through the process of talking with your users, you’re going to see all the duct tape and the paper clips that have been put in there and all the things that people are maybe they’re using a third party system to kind of augment what they’re doing within your platform. And kind of coming back to it, you’ll see some really, typically the older the systems is, you’ll see some very unique and interesting workflows that you may not even be aware of. Right. So, again, hat’s where it’s critically important in modernizing something and making it new that you take the time to unpack all of that. Because there is a very real challenge in getting people to give up those processes that they have established. You know they’re kind of baked into their workflow. Especially, we see this a lot in internal software that groups use for you name it internally. There’s a very isolated user group, a very specific user group that’s using this thing to do a certain thing, and they have created their own ways to be more efficient and more effective. And you’re hearing things in the pain point side as an owner of this product, of this initiative, that you want to solve. But the pain points that surface to you know to the top are really the downstream of what’s happening underneath the folks are doing with your platform. So it’s really about okay. We got to unpack. We got to do some digging. And that’s that can be a daunting thing ro think about it. We do you start, right? Again, though it’s about just talking with your users and maybe shadowing them on their process. We’ve done this with customers where they’ve been in the middle of moodernization, and there’s been interviews with you prime stakeholders, and also users of a certain platform where they’re trying to get everybody to go on to a new platform that’s kind of being built in parallel with a more modern interface, bells and whistles, looks very fancy. It’s really nice. It was a great design for the new system, but nobody was using it because it didn’t do a handful of the critical things that people were doing with the old system That hadn’t yet been addressed in the new system. So you think about that. If you had known that from the get go, that’s how you would have prioritized your road map. That’s where it’s one thing to start an initiative and say we’re going to just rebuild this. We’re gonna just take what we have and we’re gonna do a coat of paint to it. Everybody’s gonna move into the house and it’s going to be awesome and everybody will live there and be happy. But to get all your stuff out of the old house, you got to move it over.

Matt:

Exactly

Andy:

I think that’s why it’s critically important to take this mindset into a modernization project because you have to be able to do that deeper dive. And from that use that prioritization to inform where you’re going to start phase one right. Like eighty-twenty, We do this in phase one. That accomplishes eighty percent cool. We can deprioritize. These other things that we thought might have been the focus, but are really more nice to have when we really unpack what our customers are doing with the platform.

Matt:

There’s so much opportunity in how users are using your system today. I can’t tell you how many times you and the team have gone in and started talking to people using the system today. And you come back with, “Hey, They’re doing the XYZ thing. It’s not in the platform.” Our customers or stakeholders are like, “Are you serious?” It’s like a surprise factor. 

Andy:

Right? Everything funnels back to Excel or something like they’re using Excel. What are you doing about it?

Matt:

Yeah, That opens up new ideas though, which is cool because you’re like, well, we could solve that problem with our solution. And that’s where the real fun stuff starts. And with an existing solution, human nature is weird, right? You got the endowment effect at play where people are more likely to retain something they have, even if something else is better, because they just value it more. And then loss aversion. They value the pain of a loss more than gaining something. Like the old adage of a bird in the hand’s worth two in the bush, right?

Andy:

Yeah. go with what you know right?

Matt:

You’re Competing against that.

Andy:

I mean, it gets into a habit-breaking kind of process. You have to prove that the new thing you’re doing is truly better for that adoption to happen. And it can’t just be the same thing with a better coat of paint. It has to be better for me to be like, “Okay. I’m going to make the switch.” And this happens across the board with any sort of interaction that you or I use or anybody, right? It’s like you know I’m big on Sonos. I use Sonos all the time because I feel like the interface and the interaction and the integrations were fantastic for putting music throughout my house. But I have friends who use other services and are like you gotta jump over to this. I’m like, “No way, man, I’m invested.”

Matt:

Yeah, and the last thing we’ll wrap, this has been awesome, but the bar is at the level of what customers experience with other things. It’s not. It’s not your thing, but their bar. It could be how they experience stuff with Apple or Google or name your thing. That’s the bar that they expect. That’s another reason why user experience is so critical.

Andy:

Yeah, a hundred percent. We live in a fantastic time, I sound like some old timer, a very interesting time where we have all of these fantastic like there are groups out there like you mentioned, Apple, and they have put a ton of work into really nailing user experience. And the proof is in the pudding. Their products are ubiquitous. Everybody knows them, everybody uses them. There are wars about Apple versus Android. They are an entity within just human experience. It’s not just user experience. And so from that like realistically, you’re being measured whether you like it or not on that bar like you said, You can’t like you’re coming into the game with your service, your solution you, you have to, it’s reasonable to expect your users to expect that kind of experience, right? So again, the more you can kind of adopt, and and really, Um, you know, study and understand what it means to have good user experience again. It’s not about visuals. it’s not even about ease of use, right, it’s about true and utter enablement. Right. The more you can understand, the more you can kind of ingrain that into what you’re doing with your product or your solution, the better off you’ll be, because then you’re gonna be right up there with those heavy weights that have been doing it for years and years and years.

Matt:

Yeah, and not to end on a daunting note there. Definitely, just get started with talking to your customers. You have these examples out there too. That’s the thing. Look to tertiary to other things. Even outside of your category. There’s so many great experiences out there you can tap into. Andy, this has been awesome. There are so many rabbit holes I want to dig deeper into, but we’ll save those for some. Future episodes will definitely have you on again, but thanks for joining us on Built Right.

Andy:

Thanks. Yeah. It’s been a pleasure and any time you want to talk, you know where to find me

Matt:

Yep, sounds good. talk to you later, Andy.

Andy:

All right, yeah.