Why AI Projects Fail: The Change Management Problem Nobody Talks About

Last week that Open AI has started running these TV advertisements that I think are incredibly effective and I'm surprised they haven't come out sooner. It's a couple of elderly women who don't speak the same language and they're trying to communicate with one another and so they have the phone in front of them and they're speaking into it using the voice function to translate the conversation that they're having with one another. >> Here's the question. Where are we going with AI? What's the big picture now? >> I think management skills are going to be the most important things to develop when people are effectively using AI tools.

One of the big projects that I was involved in I co-directed a year-long AI adoption study among 175 government employees across 14 agencies. This was in partnership with Open AI. >> AI, is it hype or help? >> Good day and welcome to the show. I'm your host Pete Cooper and our topic is AI hype or help. Today's guest is Terrence Milstead from Second Mind Solutions. Terrence is involved in helping companies change their workflows and adopt AI. So without further ado, let's get into it. Welcome Terrence. Welcome to the show. How are you today? >> I'm doing well. Thanks for having me. Appreciate it. >> coming in from? >> So I'm joining from just outside Philadelphia.

I'm in the suburbs, a place called Devon, Pennsylvania. >> Devon, that's right. We talked about that and I'm just up the road from you in in Lehigh. So we're both in that northeast corner of the United States for anyone that's out of the US. So Terrence, tell us what you do and we'll get into it. >> Yes, what do I do? That often depends on the day. But you mean at a general level. So these days I identify as an adoption strategist because that's sort of been the through line throughout my my varied career.

And so by adoption strategist, I mean anytime somebody is developing a product or a service, or it could be a policy or an internal program, I'm the guy who comes in and helps make sure that thing actually gets adopted by the people it's intended to serve. Uh and so, that that could be a digital product or or app. I mean, it could be home heating, air cooling appliances. I've done that work for clients. I've done work for financial services. But basically, what I do is I go out and I make sure I talk to the end user of the product or service and understand, hey, what do they need, want, expect, but then also, you know, what's the framing that's going to resonate with them and the why.

So, I try to get at the why behind consumer behavior or end user behavior, I should say. >> Mhm. >> And really importantly, I try to understand the context into which whatever that thing is is going to exist. >> Yeah. >> Because I've seen a million times in my career where you can create something that functionally works exactly the way it's supposed to, but nobody adopts it because they didn't take into consideration the social context, the economic context, the political or administrative, and just the human side of of um tech adoption. >> There was a thing called change management. >> Uh-huh. >> Is this something that resonates with you?

>> [snorts] >> Yes, that I mean, that's still very much a thing. There are people who are certified in OCM, organizational change management. Uh and so, I would consider that adjacent. So, without kind of going down many rabbit holes at once, I'll I'll tell you that I I have combined experience professionally in user experience research, but also in strategic communications. So, I worked for several years at a big marketing advertising firm called Ogilvy as vice president in their government practice. And so, that part of change management, the strategic communications, um that's informed by really understanding the dynamics of an organization, that is something that I have experience in.

>> Yeah. >> People who are certified in OCM, they have a wider uh toolkit, I would say. Um and that's why they get certified in it, but it's an adjacent field uh certainly. >> Yeah, yeah. So, I mean, yeah. A lot's coming through right now. Um and I just got out of a meeting this morning with somebody who we reminisced over uh software. We said, "Well, you know, back in the turn of the century, it's 25 years ago, we were both actively working, and software was going to take over the world. It was all software. " Well, and adoption of software was obviously something that you've been involved in, but now you can't turn a corner in a conversation without bringing up AI.

And I'd imagine there's a lot of new changes coming around with AI. Is that your experience? Are you working in that area? >> Yeah, absolutely. Yeah, so I mean, I think that's critical. And And we can get into what I'm offering sort of as as one of my key services through my consulting firm, Second Mind Solutions. But a lot of it is about um helping people at an organizational level get past the fear that they might have about technology and and and the first step in helping them to get past that fear is understanding where the fear is coming from.

And that's where somebody in my position, I go out and I do focus groups, I do in-depth interviews, uh contextual inquiry where I'll actually follow people around and with their permission, of course, and and watch what they do in a real-world work environment. Um so, understanding the why behind some of the the blockers to tech adoption, and then developing communication strategy to help honestly but effectively communicate from a management level about um what's happening with that technology. Cuz I'm sure you know this, a lot of the fear that I've seen, and I'm I'm sure others have observed, is of course the fear of replacement by AI. >> Yep. >> Right?

And And I I have And I'm I'm going to be careful here about um you know client confidentiality, but I've certainly seen places where AI is allowing people to work more efficiently, >> Mhm. >> but they're not sharing that information with their managers. >> Mhm. >> Because depending on the context, what's going to happen? >> Right. >> You're going to have more work thrown at you. Or or somebody's going to say um oh, you didn't know how to do that? You didn't have that knowledge already?

You had to So, there's a lot of um and again, it depends on the institutional context and the culture of the place, but I think understanding the organizational dynamics and the psychology of of the team itself is critical for AI adoption. >> Interesting. Yeah, and I think um it's a very real fear, too, in the sense that some people are getting displaced by the power of AI, right? So, I know this from my own experience. Uh we we rolled out a tool and uh the the the offshore, in this case it was an offshore uh labor force was no longer needed because the tool was doing it um well enough. And then there's been other cuts in in the business.

That's a very real fear, and I think that puts this extra burden onto the the the management or the business owner, in the case of a smaller business, you know, to to how to juggle that, to say, "Well, look, AI is coming. We have to adopt it, otherwise we're going to get left behind. And we're going to have to handle, you know, some workforce changes, um whether they be positive or negative. " >> Yeah, certainly. And I mean and um I want to be really sensitive here because I I understand the the the real fear. I mean, I've been laid off in my career. I've laid off others. It's not pleasant. It's not fun, but structural mismatches have occurred throughout uh history for reason.

So, you know, industries aren't we're not frozen in amber, right? Things are going to change. I think what's important is obviously um minimizing sort of the the the negative impacts of that and upskilling, allowing people to develop the skills and the understanding so that they they can be adaptable. Uh and so, again, that fear that I mentioned earlier about people who are hesitant to embrace AI because, you know, um it might not be hesitancy to embrace it, but hesitancy to share with your leadership team that you are using it in certain ways. That's a real obstacle.

Because in my experience, and I also have a professional educational background, I teach um at the university level, a lot of the learning that comes within an organization is through shared knowledge and shared understanding and crowdsourcing learning, right? And so, if you don't create a company culture where people are able to share, you know, what they did, "Hey, I tried this and look what happened. " or "I tried this and it really didn't have a positive outcome. " Which also leads to a company needing to understand its risk posture. Like, where are those places within an organization where your team can experiment, right? It's not life or death if things go wrong.

Um so, we're kind of dancing around a bigger issue, which is that I think at two levels, a couple things should happen. One is there needs to be encouragement of individual experimentation and growth because we all learn in different ways. >> Hm. >> At the organizational level, there needs to be a cultural attitude that um first of all, A, like I said, understands the company's or the organization's risk posture and where [clears throat] there is your no AI zone, is what I call it. Uh but then, it's the responsibility of the organization's leadership to communicate then down that, you know, um um we we encourage you to use these tools. Here are the guardrails, here are the parameters.

Uh and so it has to happen, I think, at both of those levels simultaneously. >> So perhaps the easiest way to to have people relate to this is some of the what I would call retail forms of um AI, which is the ChatGPTs, the Claudes, the Geminis. They're very accessible. I don't Honestly, I don't know anybody that I work with that doesn't use those tools. So, is there a need to encourage people to adoption? Or are they just starting to use it and now we're just we're already running? >> Well, so I think it's an important distinction between access and adoption. >> Mhm. >> Just because a company makes 200 licenses available to its team, uh that's not adoption.

Adoption is when Monday morning looks different, right? Because workflows have changed. So, that means that at an organizational level, there has to be the belief that we can't just bolt AI onto existing processes. >> Mhm. >> We can't just give licenses to our team and say, "Go run with it. " >> Mhm. >> There has to be this effort to look at your workflows and specifically to take the time to look at where are our bounded workflows where we are right now bleeding time, bleeding margin, bleeding quality, whatever it might be, throughput. >> Mhm. >> And then to prioritize those. And forget about the technology.

I'm I'm talking about just looking at workflows in terms of what's happening right now, um regardless of any AI applications. But once you've mapped out, and it takes time, doesn't have to take a lot of time, but it does take time and effort. Once you've mapped out where people are getting stuck in the workflows and why, then to say, "Okay, now let's think about where does it logically make sense to apply AI? >> Mhm. >> Um and it doesn't have to be an army of AI agents that are fully autonomous and, right, integrated with your internal platforms and external platforms and all of that.

I mean, there certainly [clears throat] are use cases where that makes sense, but there could be very low-lift ways of using AI that unlock efficiencies. But, what's >> Right. >> critical here is you can't look at it in terms of um even if you look at a single workflow, ideally you should still be to see if we make this improvement here, what's the downstream effect? Cuz you're not gaining anything if you unblock a blocker in one workflow, but then it creates a bottleneck downstream in a different workflow because you haven't really thought comprehensively about everything that's happening. Does that make sense?

>> Yeah, so I would say you're a couple of levels I have this kind of mental model of zero to five in terms of capability individual capability. A five is someone who's a grand master at AI and teaching others, and a zero is someone hasn't even chat started with a chat yet. I would say level one is someone that's starting to use those tools, whether it's Claude or ChatGPT. To me, and you can tell me if I'm right, the first thing that someone does is they start asking it some questions about something that they want to know about. They kind of use it like a browser, right? Oh, can you find me a company that does this?

Can you So, they sort of have that very transferable I mean, to me it's almost frictionless adoption because it's just sitting there waiting for you to say something. And as soon as they get a response, they go, "Oh, oh, okay. Well, it wants me to do this. " It even prompts you to go down deeper or take different directions. So, I would say tell me if I'm wrong, Terrence, that adoption at that level is is pretty frictionless. Right? >> No, I would respectfully push back and say what you're describing is a use case. It's not adoption. And there's That's the difference. Can I give you an example? I don't want to cut you off. I want to make sure that you finish your thought, though.

>> No, no, it's it's good. I mean, I just look at the way I I started with these chat tools a while ago and I just thought, "Oh, okay. Well, if I need to know something. " And I just Now I use them more and more because I know the capabilities that the sphere of that that it goes for, but that very first instance was just simply, "Oh, okay. It's like typing something into Google's browser, you know? " And I just and just do it the same way. And that to me is a very It's like getting out of a car and getting into another car.

It's just, you know, they're so similar that Yeah, you got to know a few buttons and switches that are different, but fundamentally they're very similar at that starting level. And Okay, go down the path of it. What's the difference between that and an adoption? >> So, when I was head of user experience research at the Commonwealth of Pennsylvania's Digital Services Agency, which is a role I stepped away from earlier this year. I've been there for almost 3 years. One of the big projects that I was involved in I I co-directed a year-long AI adoption study among 175 government employees across 14 agencies. And this was in partnership with OpenAI.

And basically, we made licenses to ChatGPT Enterprise available to these employees. And then my team and I followed them for a year. We did focus groups, diary studies, in-depth interviews because we wanted to see I mean, ultimately, one of the goals of this was if the Commonwealth is going to scale the use of this technology across the workforce, how can we do it in a smart way? A way that really unlocks the most impact. But also, what we wanted to see in the study was if you just give licenses to somebody how are they going to use it? Without any sort of training or prompting or anything, just give them the licenses and sort of see what use cases naturally emerge.

And so one of the things that we saw early on in this year-long study is the adoption curve very much depended on people developing muscle memory. Just sort of knowing I'm going to reach for the tool now because it makes sense, right? So I think you're absolutely right. The default is, "Hey, you know, using it like a Google search engine. " Um but but beyond that really sort of rudimentary use, uh it was getting people to just think, "Okay, it's Monday morning. Here are my Monday morning tasks. Where does it make sense to use AI? " And it could be for information retrieval. It could be for categorization of information.

It could be summarization, so summarizing content from reports or emails. Um and again, these are pretty low lift. So nothing that's much more complicated than the way they're using it as a search tool, but it's just getting that um that as part of their their standard operating procedure and getting them to remember to think to do that. So one of the ideas that we had at the end, and I left, so I'm not sure if this has happened, was to kind of develop for people little recipe cards that they could just carry around in their pocket that would say to them sort of, "If you're in this situation or this scenario, you know, here's a simple, low-lift way that you could use this tool today.

" And giving them something cuz I think you need to see, right? Those those sort of um very uncomplicated use cases laid out for you. And I'll I'll I'll stop in a second, but I noticed last week that OpenAI has started running these TV advertisements that I think are incredibly effective, and I I'm surprised they haven't come out sooner. It's a couple of elderly women who don't speak the same language, and they're trying to communicate with one another. And so they have the phone in front of them and they're speaking into it using the voice function to translate the conversation that they're having with one another.

It's a 30-second commercial, but it's so spot-on because anybody can look at that and say, "Oh, that's perfect. That's exactly that I mean, you know, it's not you don't have to take a class to know to use it that way. " >> Yeah. I that's great. I had an example doing that exact thing with one of my staff. He speaks fluent German. Uh he's an Aussie. Um but he's brought up in a family that spoke German so he speaks it very well. I said, "Can we have this little experiment where he spoke German and then I spoke in English? " And it actually was it was effective. It actually worked.

I I think I used my phone, held it up to the Zoom call, and then I would read it off in English and it would then I would then give it to him in German and he'd reply back in German. I mean, we could it was a little clunky because of the way I had it set up, >> Yeah. >> but it actually did work. >> It's incredible. Listen, I spend a lot of time in the car driving to client sites and just around every single day. I probably spend a few hours a day speaking to both Claude and ChatGPT in Spanish because I speak Spanish, but I'm always wanting to improve it and you know, expand my vocabulary.

So I've actually Yeah, I created a custom GPT that I you know, I I built with some knowledge documents related to grammar rules and things like that, but often I'll just like pick up the conversation thread that I had the last time and and we'll just speak. But having that conversational, you know, um um aid at my fingertips at all times Yep. And just a couple weeks ago, a few weeks ago, uh I think maybe it it coincided with the release of Astra. I'm not sure. The voice function in ChatGPT is markedly improved in terms of lack of latency and and just the behavioral realism of it. So >> I think that that that's a a great point.

I'm going to I'm learning and the all the time about how to change my workflow for the better. And I'm going to use that word better in brackets for a second because there's just I guess it's an emerging. I mean, I wouldn't If somebody had told me 5 years ago you'd be talking to your computer, I would have said, "Look, that's science fiction. " You know, that's they've tried it. It didn't work. I tried software 20 years ago. It was terrible. Now, it's incredible good. I mean, it's to the point where we just tested out some software for voice like telemarketing. It's using using a voice tool. And it called me and I answered it. And it was so polite. It was it knew exactly who I was.

It was guiding me through. I And And didn't matter what I said, it was It was a This is a much better experience than a human. Because if you Let's say you had to ring up for a doctor's appointment. You You hold on. You get through. The person says, "Can you hold, please? " And And then, you know, you're waiting. And then, "Oh, yes. And what was your name? I'll just look you up. " And then, you know, there's It's just all these little interruptions that because people are human. And you know, you've got to be very polite. And you don't have to be polite. I'm not saying we shouldn't be polite. I'm just saying that in a way, it cuts through what is a difficult interaction.

If you say you're a receptionist having to take the phone calls every day and you don't know who you're getting. And There sometimes they're abrupt. I mean, it must weigh on you, too. But I a chat has infinite patience, you know? And it can have the information at the fingertips. There's no looking you up. It knows who you are from the phone call. >> Yeah. Yeah, certainly. I mean, I can already kind of hear some of our our potential viewers kind of coming up with the the flip side of that, which I also see. Which is that, you know, having to learn to develop patience is is really important skill. >> Mhm. >> And we don't want to lose that.

We don't want to lose those opportunities for actual human interaction where we develop interpersonal skills that we really need. Um which, you know, it kind of loops back to a point that I was starting to make earlier. In workflow mapping, when we go into an organization, we try to map out where are people getting stuck? There are moments of friction that I think, even in a business setting, are positive.

You don't want to smooth those away because in those moments of friction, even though things might be slowing down, that's where the learning is occurring or that's where the the team is cohesing together uh in a way that that we might not be aware of unless we go out and actually talk to them and understand it. Or or maybe there's a deepening of a client relationship at a particular friction moment. Uh and so again, I think um thinking about AI and large language models a partner, and I say that almost tentatively, but as a partner, but in the service of creating something that neither one of us could have created alone or doing something that neither one of us could have done by ourselves.

>> Mhm. >> And I've actually asked um my my um my custom GPTs, "Listen, I I want you to come up with an idea, but it has to be something that neither you nor I could do on our own. It has to depend on qualities that we both have that are mutually exclusive. " And it's a fun little thought experiment to try. Um and I've gotten some interesting responses back. But again, I I think about it as a >> That's a really good prompt. >> fun. >> Yeah. So, just just to recap and how we somebody could do that is listening in. What you would do is you go to ChatGPT and you would actually ask it a prompt.

"Hey, how can I use ChatGPT and me as a person, Pete Cooper, in the best way to give the best combined um outcomes. >> >> And see what he comes back with. >> Certainly. But sort of the you can do that absolutely, but the level up from that would be um obviously create a custom GPT that's configured with knowledge documents that gives it the contextual understanding of who you are >> Yeah. >> as a professional, as an individual, um right?

So that it's not just randomly giving you I mean, you know, you have to give it that foundation to be able to suggest something or, you know, even better than that is you know, create a custom GPT, but then access it from a project that you would create in Claude, right? So that you um So I mean, we can keep kind of layering on things, but I think the first step is experimentation. Exactly what you just said. Do that and then kind of work your way up to building something that's a little bit more contextually aware. And by the time you get to using things like Co Work or Chat GPT Work >> Mhm. >> your AI actually has access, if you allow it, to your desktop files, to your emails.

To me, that's been a game-changer just in terms of productivity. For me to be able to stand in line at the grocery store, pick up my phone and say "Hey, I just had a thought. I want you to go into my desktop. I want you to go into my consulting folder. I want you to summarize the notes from this client engagement. Then I want you to draft an email using this structure and have it waiting for me in my inbox so I can review it and hit send when I get home in 15 minutes. " I mean, that's all possible now through Co Work and through Chat GPT project. And again, most people don't realize I'm a non-technical person. >> Mhm. >> I really am.

I'm I'm, you know, um a social scientist with a humanities background. But most people don't realize that you have that capability now and it only takes 15 minutes to learn how to do that competently. >> I think that's a really good example. I haven't tried that myself and I I think I would like to try that. I'm certainly using the voice a lot to I guess riff off. So, you're you mentioned an example like a virtual assistant. I tend to use it more like a business partner. I would say, you know, [snorts] I need to discuss this with you. You know, what how can we reframe this word? Can you do some research on this? Can you find something out for me?

And can you structure a response, put it in a document for me? It's much more I use it at that level. And I've found I mean, how much do you trust these these things, but I would like to think I got enough experience I know when to reject things and when to accept things. But I could be wrong from time to time and AI is known to hallucinate. But, you know, when I compare it to a human like a business partner, it's it's in a way the same. I mean, people are human and flawed and they sometimes say things that are wrong, that they know are wrong, or that they mistakenly genuinely mistake make a mistake, or they just want to say something to say something, right?

And I think all of those flaws that are in that relationship are the same with AI. >> Yeah, that's a great point. So, I used to before I got into consulting about 15 years ago, I was a a tenure track faculty member at a university. And so, I would have a research assistant assigned to me to semester because, you know, I had to publish and enjoyed it, but sort of that was a requirement to stay in the job. And I think about AI the way I think about a research assistant. Somebody really intelligent, but they need parameters, they need guidance, they need to know what good looks like, they need to know what an abject failure looks like.

They they need to know what the things are that would get me fired Mhm. >> Uh, would get that person sort of, you know, um, facing recriminations as well. So, you have to I think management skills are going to be the most important things to develop when people are effectively using, uh, AI tools because if you can manage well and and and part of that also is where are the the check-in points, right? Where are those specific moments when I need to be brought in to set my eyes on something. Um, asking it, making sure that it always QCs its own work. That it shows you the evidence.

That it gives you the links to the sources so that I can actually go in and and look myself with my own eyes, um, to to test the veracity. So, it's basic management skills, um, that are so important. >> Yep. So, I want to explore the counter to this cuz I've had these experiences and that is perhaps over adoption or perhaps mis-adoption. And this has happened to me multiple times. When I bring new staff on, it's usually when I have the the biggest problems. And I'll say, "Oh, can you prepare a plan for me, uh, about, you know, something that needs to be done in your, you know, in your field, let's say the marketing campaign or something.

" Uh, and then I get this 10-page document which is just going on and on. And I'm like, "Okay. Can you explain this to me? " And it's clear that it's not been created by the person who who sent it to me. So, it's been a a flick to the I'll call it a flick to the LLM. And then that job that task is done in in a minute and then back to me. Now, when I was developing plans, it was the process of developing the plan was the important part. It wasn't the document. It That was just the end result. It proved that you had done all those things.

And when you sat across the table with your subordinate or your project manager, whoever it is, you could go through it and it was very clear that because it was well presented and well structured, then all of that work was done prior to it. And it seems that we've got to the end without going through the process. >> Got you. >> Why is that process important? Because if you're working in an organization, you need to get the buy-in from this person, you need to get them somebody else lined up over here. They know about the legal ramifications to some level of detail. Whatever it happens to They all has to be synthesized into this plan. >> Yeah. >> You can't skip over that.

And so, I've had to really push back on this overuse or misuse of LLMs to create plans. >> Mhm. >> What What's your thoughts on that? >> Yeah, it's such an important point. So, I to me, what's key about what you just said is the importance of us internalizing processes. Let me give you an example, and you tell me if I'm I'm sort of left of the mark here, but one one of the earliest custom GPTs that I created a couple years ago was what I called strategic communication assistant. And so, here was my thinking at the time. If somebody came to me and said, "Hey, listen, I hear you teach a course on business communications at Penn. Can you help me write a sensitive letter to a client?

" I would never sit down and just start writing. What I would do is walk that person through about 20 questions, all of which are super important for for us to even begin to craft a message. So, I built a custom GPT that's trained to walk somebody through each of those questions one by one. It won't go to the next question until you've answered the first question. >> Mhm. >> And at the end of it, what it gives you finally is a communication that's still draft, obviously, but it's a lot closer than it would have been if somebody had just sat down and started writing because it's it's you know, it's gone through this process.

So, what's important for me about that little exercise is not that you've got this thing that saves time because it doesn't save time on the front end. It might take you longer to work through that custom GPT to get the draft of that letter that's 80% done. Where it saves time and McKinsey actually did a study on this a couple years ago is when you send out a poor communication in a business environment, it costs you money downstream through misunderstandings, through revisions, through the phone calls that you have to get on explaining.

So, your the value is in making sure that thing that goes out the door is impactful, which means that it incorporates all of the elements that we teach in in my class at Wharton. But the other really important thing about that custom GPT for me is that it reminds me every time I use it what those questions are and the answers and why those are important. So, it's it's helping me to internalize and I've been thinking recently about building one. I just haven't had time. I've got it actually on my bulletin board here a similar custom GPT for different types different cognitive styles like first principles thinking, second-order thinking, systems thinking, adversarial reasoning, right?

Building these things that I can have conversations with that walk me through the steps in these different types of strategic thinking that become really really important in business settings. Um so, the bottom line is I think what you said about processes and making sure that we're not delegating the entire process away, but instead we're doing something that helps us remember the process steps ourselves cuz we have to supervise. We have to manage >> Right. >> the AI and make sure that it's following those steps. >> Yes, right. And I think that I feel that the burden of managing AI is is is at the management level. It's not at the it's not at the uh employee level.

I I feel at the employee level, what I'm seeing is a huge acceleration in what they can do and capability of what they can do. But, the side effect of that is a lot of noise, a lot of AI noise, like the example of the 10-page document, which they don't understand, right? And so, who has to break through all that? Well, it ends up being the manager or the leader or the business owner has to recognize that and then go to work to try and correct it.

Whereas in the good old days, you could trust that if somebody had a nice 10-page document, and they had taken the time of typing it out with Word, with the running the spell check and all that, rather than any kind of AI generation, that they must have thought about it, because they created the words. >> [snorts] >> Well, I mean, this is where uh and again, this is still pretty basic, but in uh Anthropic and Claude, you know, you've got skills, the skills feature, >> Mhm. >> which so you can build a skill that would say, and and it's something that you don't have to keep triggering.

You build the skill and it would basically say, "Hey, anytime you produce a report for me, it has to have this structure. It has to have these features. It has to go through these uh QC checkpoints, right? " So, you can templatize processes um pretty easily uh in in a way that sort of will then be applied to any interaction that you have. It's just a question of again, raising people's awareness both at the management level and at the individual level that that's now a feature. It's something that's there to help you. >> Right.

Well, [sighs and gasps] if we created these other tools around that, we're relying then on the tools, we're relying on the LLM, and relying on Well, we're not really relying on the person again, are we? We're not really saying, "Hey, this is what I This is When I have people, this is what I like to be able to say to them. I'm trusting you. I'm giving you the responsibility to execute this project, this initiative, this and I try to elevate it to the business level, return on investment, you know, those sorts of things. Show me how that that equates to that and I have to trust them that they're going to follow through.

I feel that with the LLMs coming in, I'm almost having to go a level down to to do a lot of extra work because I can't now just say, "I'm trusting you to do that. " because that document has got in the way of of of that. The old way of sort of trusting the document was proof that they can execute the plan has gone. So, now I have to rely on these other systems that I've built, which I may not and be able to, right? Especially if I'm outside my field, right? If I'm in invoking somebody in an area that I know nothing about, then I really have to trust them. And I can't use the old tool of a nicely presented report or a pitch or whatever as a gauge. >> Sure.

>> And that's that's creating a more a difficult complication that I haven't really found a solution to. But let I mean, maybe the skills is something I need to develop or maybe that's where I need to invest more of my time into the skills that enable me to to to cut through that. >> Yeah. Well, I mean, it's there as a resource, but I I definitely Peter, I don't think it's trust. I think it's calibrated trust. And the situation that you just described to me where you're working on something that's outside of your your field of expertise, to me that's clearly an opportunity for human in the loop and for collaboration.

I mean, I would want to bring in somebody who is an expert in the field and you know, either to truth check whatever it is that I'm that's been produced or that's being produced. It should never be something is produced. Now let me make sure that it's accurate. I mean, it it should be a system built where you can check the veracity of what's being produced as it's coming out in in smaller increments. >> Well, I had just this example that I and Terrence of of one of my colleagues saying, oh well well I we're working on this document together was a pitch and they said, well, I got this from chat GPT. They said this. They said that and I'm like cringing and I said, no, don't don't don't react.

>> >> I wanted to say, that's not good enough. But I didn't and we had to probe around the whole issue about where was the underlying basis for that information because >> Yeah. >> we know that chats can hallucinate and you don't know to what context that question was answered. Um it's a little bit like Chinese whispers in that case, you know, it's oh so and so said, but they heard it from so and and was the original source correct in the first place? >> Yeah, that's a huge red flag.

If anybody ever says, I got this from chat GPT or it is like you know, there's a deeper conversation to be had and I'm I'm not you know, trying to be overly critical because there are people who still think that you can interact with the chatbot and get back some and the hallucinations I will say in the latest models and you know, as the models have continued to improve, the hallucinations have become less and less frequent. So the quality of the models is improving, but still that's not an excuse.

Um you know, I think what you should hear somebody say ideally is hey, I I again, I I created and or orchestrated the development of an AI agent that was able to pull credible sources that I then QC'd and I collaborated with a colleague who's an expert in the field on this other input and together, right, we created this thing. So there there still has to be that element of external validation. I mean, even just and again, I don't want to punch above my weight class here, but a couple of weeks ago the whole story about OpenAI's chat GPT solving this math problem that had been unsolvable, right? >> Was it the Stokes-Navier? Stokes? Was that the one? >> I think so. Yeah, yeah, yeah, yeah.

And so I I just heard yesterday that apparently and again, I I think this is from a credible source. There are a lot of other math problems that have been solved, but they're holding off releasing those until they've talked to the math community to kind of verify and and do everything that they need to do. So I that's a way of saying that while I think the quality of the models is probably improved very dramatically and then hallucinations have have become less and less frequent. You still have to have that dialogue with the human experts in the field. >> Mhm. Yeah.

Well, even if it's not a hallucination, it could be from a source that it's it's genuinely believed but has that hasn't been cross-checked. I mean, how does this happen in the science world? There's a thing called peer review, right? Where which is you kind of alluded to the fact that you've got multiple people with expertise analyzing something and saying yes, we all agree or we don't and we debate it out. This was the engineering world that I came from.

When I worked for Nokia, we had these milestone reviews and you would get people from the factory, the engineering, the mechanical, the supply chain all coming together for this big milestone and it was there for they were all trusted around that milestone to sign it off or sign it off with open items or reject it. So, that was a kind of a peer review process in an engineering environment. I feel that it's kind of short-circuited into some extent and I had this really good analogy of um which I'll repeat repeat here. If you wanted to get to the top of a mountain, you could climb. You could walk past the river.

You could experience and smell the forest and you could walk up through the get to the snow peak and what have you. Or you could just fly a helicopter there and go and land there. And in the worst case with an hallucination context, you could say, "Well, the helicopter took me to the wrong mountain. " Right? Cuz you just wanted to get to the top of a mountain. But in the other sense, you never knew how you got there. You didn't pass the river. You didn't pick up the flowers. You didn't do this. You didn't have a journey >> Yeah. >> top of the mountain. You just got there.

And I feel that when you ask people to generate documents and it's uncontrolled, the tendency is to just arrive at the destination and deliver the document without taking the journey. >> Yeah. >> And then the question is, "Well, what's the value in the journey? " You know? And I think if you're picking up the peer review pieces all along the way um in the business context or in the the in the experience of walking up to the top of the hill, that's a life experience. Like you have a life experience to climb that mountain. Getting a helicopter to the top is not a life experience. Well, not really. I mean, it's just a it's a helicopter trip.

>> It's such a funny analogy because I you know, um I've been thinking something similar and and if I can share with you a little something about my my recent experience trying to stand up an AI consulting uh practice, I think you'll see the the relevance here. So, I I set up Second Mind Solutions with the idea that So, the first thing that happened was I had clients coming to me and saying, "Hey, can you give us AI enablement trainings? " And this was a couple years ago, just AI 101. We want our team to understand.

So, I started doing that, and then I thought, you know, a better thing for me to offer would be using my user experience research background and my strategic comms background, let me come into a team, map out your workflows, figure out where people are getting stuck, figure out where AI makes sense, then create a prioritized AI road map for you, and I'll do it in 4 to 6 weeks, right? That's that's the basic service offering. And then if they like the road map, then that leads to me coming in and redesigning the workflows. I will tell you, and I've been getting on calls over the last several months with COOs, CMOs, CFOs, and a couple CEOs. Nobody is buying that.

And I'll my ICP, my ideal customer profile, is small and medium-sized firms in the knowledge industries and creative services world. Um so, it's, you know, it's pretty narrow. But nobody is buying the prioritized road map. What they want is somebody to build a vehicle to get them to the destination on the road map. And they want that vehicle this week. They don't want it in 4 to 6 weeks. Which means that what I'm seeing in the solicitations that are going out and what I'm hearing in conversations is that they're hiring automation engineers to come in and build AI automations for them. They're automating parts of workflows, right?

To get them to a place where maybe they're seeing increased ROI in in the first couple quarters. I don't know. But to me, if you don't know how you got there, right? And you don't know sort of what the terrain looks like and and what the again, what the downstream effects are across the organization of automating certain things, like you're missing an opportunity. Um and partly, you're you're also missing understanding of your own workforce. And sort of where, like I said, your no AI zones might be. So, uh your analogy about the helicopter is is very apt. >> Yeah. So, I can very much relate to that. I'm seeing the automations being uh what I'm feeling the pull in the market.

People want automations. They want to they want to find ways to speed up work, get work done quicker and cheaper and easier. And And you know, that's one of the projects that I was telling you about which where we did that. One of the things that we did and you can comment this is we made sure that there was always a human interacting with that automation. So, we had the 80/20 rule. 80% could be done via the AI and then 20% would be done. And that can often be just checking that it looks valid, right? It looks reasonable. Uh and then one button. So, there was they get thousands of transactions a month. And that most of that can be done with AI and they wouldn't have to see anything.

But we insisted that you know, they have to at least verify each of them manually just to see there's nothing completely weird going on. Like if it's in Australia when all of sudden there's a whole lot of you know, foreign names popping up that would red flag, right? >> >> Just as a simple example, but >> [snorts] >> Yeah. Yeah, I mean that that certainly sounds like a a reasonable approach. I I guess my sort of my concern and and I've seen this in in again solicitations RFPs for consulting services. Somebody will say, we want somebody to come in, map out our workflows, figure out where we're getting stuck, figuring out where AI can be um applied.

And we also want them to build these um AI agents to do XYZ. But for me, if you're starting with a solution before you fully understand the problem, right? Then that that's a problem in itself. And so, um again, it could very well be that the people who posted those solicitations had already done their due diligence. They already did everything that I'm I'm talking about offering as a service. Um but the other thing Peter that I've heard in these conversations that I think is really important intelligence, and I heard this from again from CMOs, they're under pressure to do something with AI. >> Mhm. >> Contribute to an AI-ready company, right? >> Mhm.

>> But at the end of the day, the CMO is still being evaluated on the same criteria that she or he was being evaluated on before AI ever came along. >> Yeah. >> Which is convergence and all these other metrics. And so it's like they've now got this other thing that's been added to their plate that is introducing friction, but they're not going to be evaluated on how well they integrated AI into their marketing functions. >> Right. >> And so it's again, it's an organizational dynamic that I think is important to understand and map out. >> I agree. I think this this burden of adoption has has been is been thrown at businesses um like the one in Australia I was talking to you about.

And that person's my sister and she runs a business with 40 staff. She is also a clinician, so she's busy busy. She doesn't really want to spend her weekends learning AI, right? She doesn't really want to take the journey of learning what all AI can do and then but she knows the potential. I mean, she saw it ahead of me. She saw the problem. She saw that AI could possibly do it. She didn't want to map out the workflow. She didn't want to know the minutia, but she did want the result and she's very happy with the result. So, how does she manage that? She know how She trusted me to do it and to do it well. And I trusted my team to do it well.

But um she didn't really have any visibility into that. It's not like she could watch her one of her physical therapy uh people in an example of taking, you know, doing some therapy on someone where she can see and witness and and interact like there and then. It's very abstract and very hidden. It's It's very hard to get a handle on for a business owner. >> Yeah. Yeah. Well, I think some of this is going to you know, the proof is going to be in a year. Or I mean, obviously she had the benefit of a trust relationship with somebody that that she believed in, you know, um a lot of of people working in industry don't have that and they're being assaulted by vendors.

They're going through the vendor gauntlet. Um you know, people like me trying to say, "You can trust me. " >> Exactly, yeah. >> Yeah. Yeah, so I mean, it that that is a pressure, uh obviously, but my hope uh is that Well, let me frame this a slightly different way. As a user experience researcher and somebody who believes that it's important to take a step back, be strategic, invest a little bit of time, and really kind of again, understanding the context in which this thing that you're building is going to exist. Uh because you want people not just to buy the first product, you want them to buy future iterations, you want brand loyalty, and all of that.

Um Anything that adds friction to an environment where product managers are being evaluated on their ability to ship product is going to be resisted. Um and again, it comes down to incentive design, I think, within an organization. If you're a a product owner, product manager, and you're being evaluated in your annual review on the product that you ship that's functional in the near and the the medium term, right?

But you're not being evaluated on whether or not that thing that you shipped 2 years ago is actually being adopted and still used by the user, not only are you not going to have an incentive to do the kind of user experience research that I'm talking about, you're going to have a disincentive because it's going to slow you down. Even if it's a little bit, even if it slows you down a couple of days or a week doing the contextual inquiry, right? There's going to be friction introduced. Now, I'm of the opinion that's a necessary friction that's going to lead to positive outcomes in the long term, which is adoption and long-term use. Um but but it's about incentive design.

So, I think the CEOs, people who've invested their their their heart and their soul into building a company are the ones who really um the the ones who are really going to be open to that argument that like there's a long-term gain in being strategic about how and when and where we're using AI and where we're not using it, where we're preserving um human relationships. >> Cool. I want to shift gears to some thoughts about the future of AI cuz you alluded to it, you know, in a year's time and it made me think, well, I got to ask Terrence the question, where are we going with AI? What's the big picture now? >> Well, >> >> oh, that's a small question. Where are we going with AI?

Um well, I mean, everybody said that 2026 was going to be the year of agentic AI. >> Mhm. >> Uh and it certainly seems like like it's moving in that direction. I mean, I'm not building um agents on the raw AI myself, but I have orchestrated um the creation of autonomous agents that are doing things um you know, without being prompted and delivering deliverables to me on a regular cadence. So, again, even as a non-tech person, I have jumped into the agentic workflow world in 2026. >> Mhm. >> Um I think what's happened in the last month with the public discourse around AI and and the fear factor maybe was not predicted a year ago and and certainly has an influence, I think.

Um whether it's going to slow things down or not, I don't know. I'm not an economist. >> What do you think Where do you think the fear You said the fear factor is What What is Can you put a nugget on that for everyone out there? >> I'm talking about Jay Jacob Coxen, the guy from OpenAI from Anthropic who came out a couple weeks ago and talked about the fact that, you know, AI uh people on the inside believe that it has the capability or will within in to wipe out humanity. You know, that whole conversation that's been all over the the news cycles. >> Yeah. Yeah.

>> I don't think anybody would have predicted, even if people were thinking that, having those conversations, the fact that this young man came out and made that statement, which then forced the heads of the companies to come out and make their statements, which were basically in agreement. I mean, that was something that came out of a little bit out of left field that I think now, you know, has probably affected the the trajectory of where AI is going, because that's your question, where's it going? >> That's Okay, so we we've got a Well, that that fear factor has gone gone back to sometime sometime ago, I forget. It may have been 10 plus years ago.

Elon Musk, he's saying, you know, that open AI needs to be controlled and he invested in open AI to make it a nonprofit and for the good of humanity. >> >> And then that caused open AI did something different and there's a lawsuit with Musk. And now he's almost saying saying the opposite. He's he's not saying that that the world's going to collapse. It's going to be actually all all good, sunshine and rainbows even. Like no one's It's going to be checks for everybody and no work. I mean, he's almost shifted. >> Um >> I would really hope so. But he also came out last week and sort of agreed with um the the the dire prognostication that came out.

But my point is, yeah, that this obviously has been something that's been discussed and people have gone back and forth. What I'm saying is we can't predict when somebody's going to come out and make a a statement to the press that that is going to start off this political cycle and, you know, public sentiment's going to come out. So, I think that and what we're seeing with the data centers and a lot of opposition to the data centers and, you know, people mobilizing around that, those are unknowns for me in terms of impact that they might have.

I would still uh suspect, and again, this is me just speaking in the moment here, that we're going to see a continued movement towards uh agentic AI as more and more people start to realize the capability of in a thoughtful way building these autonomous agents that can use multi-process reasoning and that you know can do things that are helpful without being prompted, okay? You sort of set them up and and give them a task and they're optimized for task optimization. And that's what Astra is sort of I think that's what makes it different. It's a model that has now been released that's really built for task optimization.

So you don't say give me a list of suggestions, you say hey here's a task I want you to accomplish it and sort of you know see where it goes. So I think we're going to keep moving in that direction but I wish I could tell you what's going to happen in another year from now. >> Yeah. Well, running out of time Terence and there's so much we could cover here. We'll have to to consider having you back on. If you don't have any questions for me I'll jump to our final question. AI is it hype or help? >> Well, I don't think it's inherently either of those things. It depends on how how it's being used. It's helpful if things change on Monday morning in the workflows, right?

It's helpful if the metric that you have thought through for what success looks like is actually being met and that metric can be again quality, ROI, throughput, it could be creativity, right? Then it's helpful. It's hype if you're expecting to just give licenses to people on your team and suddenly you're going to see this increase in productivity or you're going to see them figure it out and and use it in the best possible way. I don't think that's going to happen without again those sort of institutional cultural changes happening at the management level. >> Yes, they said, very nicely said.

Terence, this has been a This has been a really interesting podcast with you and there's a lot more there we need to unpack. We talked about workflows in businesses and I think that is where, you know, the the biggest impact it's going to have over the next few years. That's That's my prediction. With We've seen the the rollout of chat. We've seen the adoption very high rates in that. But now it's going to flow down into businesses. And that's going to be a big transformation. You're right in the middle of it. I'd like to be in the middle of it, too, so let's see if we can do some good work. >> Yeah, thanks again, Terence, for having me on. This has been great. Enjoyed talking to you.

>> Thank you. Well, I hope you enjoyed that as much as I did. Terence was a wealth of information about AI adoption in the business context. We talked about how it's not just a case of rolling out lots of licenses, but it's actually getting the AI tools used as part of the workflows, dissecting the problem first rather than just pulling the technology out of the air and hoping for the best. He's an expert in that field and we dug into all sorts of tools, techniques, and skills that can really help a business. So, if you're interested, go back and review the podcast. If you want to know more about Terence, please follow the links in the description.

If you want to know more about Skillion, we do, please also do the same. Hope that helps and thanks for watching.