The Future of Work Isn’t What You Think

Great Expectations Podcast – Episode 1: Tom Davenport

Paul Sephton:

Hi, I'm Paul, and you're listening to Great Expectations, a Jabra podcast that's your trusted guide to navigating AI's impact on the workplace in a world full of AI noise.

We cut through the hype and deliver the information you need to make smart future-proof decisions about AI.

In today's episode, we chat with Tom Davenport, one of the pioneers who saw AI's potential to transform businesses.

Years ago, a professor at Babson College and author of many books, the most recent of which is All Hands-On Tech, the AI Powered Citizen Revolution.

Tom has advised major enterprises and taught MBA students on AI's potential for years.

So keep listening if you want to understand why many companies are missing the mark, why only 5% of organizations have effectively deployed AI.

And the burning question we all want to answer, will AI enhance our work or replace our jobs?

I'm really keen to kick things off by understanding a little bit about the Gen AI hype versus the reality.

You're someone who probably saw the tsunami coming or at least the wave of it and was talking about AI long before everybody else caught onto what I'd say is now a huge trend.

What would you say in hindsight has been the biggest differentiation for you of hype versus reality?

Tom Davenport:

Well, I think there are a number of aspects of that hype.

One is that generative AI is the only kind of AI there is, which is, I think that's a serious mistake for many large enterprises because generative AI is great for what it does, but it deals with unstructured data and its objective is to create more unstructured data.

And a lot of organizations have structured data and they want to use it to predict something or others.

So I talk about generative AI and analytical AI.

The definitions are a little tricky because generative AI also uses analytics of course, but the purpose is an analytics.

The purpose is to create content of various types, and analytical AI is trying to take structured data and predict into the future with more structured data.

So if you're talking about what price to charge or whether someone is going to contract diabetes or whether they're going to respond to a marketing offer, that's all analytical AI and still for many organizations I think is very important.

So I think it's really critical that organizations realize that generative AI is not the only thing out there.

Then I would say there is a bit of hype about how much benefit we're going to get from the productivity advances that come from generative AI.

And to be honest, I don't think we really know yet.

We really won't know within particular enterprises unless we measure and hardly anybody has done that.

I wrote an article recently, I co-authored an article about six disciplines that an organization needs to have to succeed with generative ai, and one of them is measurement of value and productivity and so on.

And I was presenting this weekend an AI conference, and I said, raise your hand if your organization's culture is characterized by discipline and nobody raised their hand.

So I think it's quite possible that we won't get that sort of measurement, and so organizations will be just be going on faith.

There are a lot of other issues about how do we get that value that generative AI is capable of right now with the need.

I think the very critical need for a human in the loop, both to prompt it in the first place and then to review the output is still somewhat problematic as to how much productivity improvements there are.

I've worked with companies that said we tried to generate product catalog content with it, and it was great until we realized we had to have people actually read it and make sure that it was all good.

And even in my teaching, my students sometimes say, well, you're making a show our work, we have to show the prompts we used and we have to show the edits that we made and so on.

And they said it was easier when we could just modify Wikipedia content and that would be our essay.

And I think maybe there's some aspects of the hype that are overly negative about hallucinations.

And so I do think that under highly controlled and managed circumstances, we can reduce the frequency of hallucinations quite dramatically so that that's less of an issue.

There was hype about intellectual property leakage because you put your secrets into prompts and then they'll retrain the model on those prompts and so on.

That doesn't happen very often.

And B, most large organizations have their own enterprise versions of large language models and their prompts are not used to retrain public models.

So that's less of an issue.

Paul Sephton:

I think you've categorized companies in the past and to those that are ahead, those who are waking up to AI and those who think that gen AI will replace traditional AI investments.

And speaking about the hype, it feels like everyone's trying to jump onto a moving train for the risk of being left behind.

But like you say, there's a lot of experimentation and we haven't quite figured out that implementation.

You mentioned sort of the six guiding principles that organizations should take.

We've also discussed briefly how we can successfully measure the implementation or success or room for improvement with AI.

So what's your sort of 1 0 1 crash course for organizations at the moment who are trying to not get left in the dust and are trying to raise ahead but doing so responsibly rather than in a complete state of craze?

Tom Davenport:

Well, I think to start with, organizations need to spend a fair amount of time thinking about where they really apply this technology, and you can have people experiment with it, and I think that's generally a good thing.

But for serious commitments to generative ai, I think you need to say, well, what's our competitive advantage now and how can generative AI improve upon that competitive advantage?

A little bit of productivity here and there probably is not going to help all that much because everybody else in your industry is going to be doing that.

So if you're competitive advantage is the knowledge that you have about the application of your products and services, say, I don't know the engineering knowledge or the customer service knowledge or whatever, then you say, okay, we're going to make a really serious effort.

We're not just going to experiment.

We're going to try to create a production deployment of that particular use case.

And that's a big issue.

My surveys and other ones as well say that production deployments are down in the five or 6% range of companies having any production deployment at all.

And so we need to decide what to do and then we need to get it into production.

And that's where those measurement disciplines come into play, where you have to, if it's a customer facing activity or something really highly regulated like medical, you have to make sure that you have all the behavioral disciplines in place.

You have to think about the design of your business processes.

So in a way, it's not surprising that production deployment percentages are pretty low because it's a much larger commitment, but then again, you don't get much economic value unless you are putting it into production in your business.

Paul Sephton:

With regards to your students that you just mentioned, I'm really curious around the future workforce and how generative AI will likely shape that in terms of what companies should be looking for in candidates now, how people can be preparing, because the quote that circulates so often is AI won't replace you, someone who knows how to use it, we'll replace you.

So as someone who's in touch with a lot of MBA students and guiding them as well as consulting to companies, what are you seeing as some of the core skills gaps that are emerging or areas that businesses are far more interested in potentially beefing up based on the workforce transformations that we're likely going to see?

Tom Davenport:

I do generally subscribe to that cliche term that you mentioned.

I've written two books about the impact of AI on the workforce, and they were both pre generative ai, but still, I think until proven otherwise, I'm a big fan of augmentation rather than large scale automation with lots of people thrown out of their work, wandering the streets, looking for jobs and income.

And we haven't seen that yet.

Even with generative ai, there have been a few experiments where, I dunno, journalism organizations say, oh, we don't need reporters anymore, let's get rid of them.

And turns out the quality of the output was really quite poor.

We've had the traditional form of ai, for example, in radiology now for more than a decade where we can analyze images.

I think it's fair to say not a single radiologist in the world has lost his or her job, but I do think it's really important to know what these technologies are capable of doing, not just in the abstract or in general, but how it might augment your own efforts as an employee, how you can add value.

Maybe you can add value by just making sure that generative AI output is of high quality, maybe that sort of critical thinking to say this is crap that chat GPT has produced and it really needs substantial editing.

And it's quite amazing how these systems are affected by past content that particular vendors or whatever have poured into the world in a big way.

I think it's fair to say that IBM Watson, while it has some, always had some strong capabilities, it's not the leader in AI now, and it had some quite significant failures yet, if you ask who's the leader in healthcare, ai, IBM Watson is one of the first to come out of chat GPT, just because they pour tons of money into marketing that technology.

And that's the content that chat GPT was trained upon.

So you have be, I think, very aware of what's really happening in the world to know is this stuff useful or not?

Paul Sephton:

So if we talk about productivity and it almost feels like people are so ready to welcome generative AI into their work days because it's potentially a silver bullet for a lot of busy work, a lot of emails and communications that we're on right now.

So if I'm not in content generation, it could be something which helps take care of a cluttered inbox meeting notes that I don't feel like taking or sending out afterwards, presentations that I don't have time to put together, et cetera.

So what do you think in regards to productivity we will see with generative AI beyond just content generation, and particularly with your book coming out on the democratization of ai, I'm curious to get your take on the productivity gains we will or won't see in your prediction.

Tom Davenport:

Yeah, I thought a fair amount about that.

I wrote a book a number of years ago, not a terribly well selling book as I recall.

It was called Thinking for a Living.

It was about knowledge work and how to make knowledge workers more productive and effective.

And as I suggested, most organizations find it very difficult to measure knowledge, work output productivity.

Even if you think about one of the primary applications for generative AI is generating code.

We don't even have widely used measures of system development productivity there.

We've used lines of code in the past, but that turns out not to be very accurate.

We've tried, there was something called function points that I think have largely disappeared as a way to measure how much code has been generated.

So I am not sure what companies are using when they say they got 20% productivity benefits.

You have to be able to measure the output.

And then there's the quality issue as well.

What's the quality of a blog post?

As an academic, I'm supposed to write articles.

We can't really figure out how to measure the quality.

So we leave that to other people and see, well, how often was it cited by somebody else?

So we've kind of crowdsourced the quality issue.

So I of course try to cite all my own work so I can get my citation numbers up.

Just kidding, sort of.

Anyway.

I think it's very hard to measure productivity, and there's the issue of most knowledge workers don't really like to be told how to do their work, and they like a lot of autonomy telling them, oh, you have to use generative ai, you have to use it in this way, not going to be very popular.

And in my own work, I haven't seen it make a big difference.

I started this year for the first time, I decided I would use chat GPT for grading purposes, but I thought it was irresponsible for me not to read it and grade it myself as well.

And so it's added work.

The grading was, the grading wasn't that great.

Everybody got a B basically all the scores were in the eighties for some reason or other, even though there was pretty widespread variation.

And in general, I gave better grades, so they didn't mind so much that I was the real grader, but it takes more time, not less.

And then we're also moving into the era of ag agentic ai, which I think is largely viewed as something that will enhance productivity because humans won't have to be in the loop as much.

Paul Sephton:

How does agen AI differ from generative ai?

Tom Davenport:

Sure.

Well, agen AI is one of those technology domains that has much more hype than reality at the moment, but everybody in the tech industry is very excited about it.

These are autonomous agents, AI agents that can not just generate content but actually do something with it.

They can send out emails, they can make particular decisions about, I could develop a grading agent, or somebody could develop an agent for how good was that credit card application?

And some people think they'll be self-healing, kind of a widespread belief that they need to be kind of small and standalone and they'll operate together as an ecosystem.

And then something is going to have to kind of orchestrate that.

That might be traditional robotic process automation or it might be something else, but a lot of excitement, but not really much experience yet with them.

And if it's using generative AI and there are hallucinations, imagine there are five different agents that completed that, finding out which one created the error and maybe the errors got magnified as they move from one to five and being able to, how do you reverse that transaction?

So we don't really know that stuff yet, but that doesn't stop the tech industry from being very excited about that.

Paul Sephton:

So would this be something like the adoption of co-pilot or are we talking about specific agents who are tailor made for each organization?

Tom Davenport:

They would be tailor made at least for a particular task.

And co-pilot, for example, is the term that's usually used for code generation or pair programming where you pair a program, a human programmer with a code generation copilot or assistant or whatever.

But there are some code generation agents that are totally autonomous that purport to be able to generate programs without any human intervention.

Now obviously you'd have to tell it what want in the first place.

Somehow that seems like a form of human intervention to me, but it would just, you'd send a command to an agent that say you're going to develop a website or something, you'd send a command to somebody saying develop a little application to do credit card fraud checking or something along those lines.

And probably in that vision, the MasterCards and the visas of the world would make that agent available to merchants that they could use easily.

But we're a long way from that now, and I don't even think we've figured out exactly how it's all going to work.

Paul Sephton:

I think the other core piece with that is training.

So it's one thing to implement a technology, but then to train your staff on it obviously is the question of is it worth the corporate overhead?

How much time am I taking out of people's days?

What do you think the right advice for organizations is who are considering what type of AI to invest in?

And then what type of training needs to be delivered hand in hand with that in order to see sort of minimum viable success?

Tom Davenport:

Yeah.

Well, there are some general findings in the variety of surveys on ai, as you know.

And one general finding that comes from surveys of workers employees is that there's not enough training and not enough explanation about how this might affect people's jobs in the future.

So we need more of it.

I think that, again, we try to be economical in terms of the types of effort we put into training.

And so we say, okay, here are five generic video MOOC programs on different types of ai, and you might want to look at all of those.

Or one do a general, one company that I was recently working with, a big insurance company developed an AI awareness video that every one of their employees had to look at.

And I think that's a useful thing, but eventually it's going to come down to how do I do my job differently with ai?

That's going to involve some thought.

It's going to be labor intensive.

As I was saying, if it's in an area that's critical to your business and could lead to some competitive advantage, if you use these tools well, you should start to educate, work with people about how they think it could be used, develop a set of approaches to using it effectively and try to motivate them to do it rather than ignoring it.

One of my favorite example is you have radiologists who say, hold it.

I don't see cancer in that radiology image.

The AI system is saying that there's cancer.

I can't tell my patient that there's cancer if I don't see it myself and I can't explain it, so I'm just going to ignore it.

And I think we'll see a lot of that unless we really train radiologists and everybody else who's using these tools and how to make sense of them, how to work them into their jobs effectively and so on.

Paul Sephton:

So do you think it's some type of confirmation bias where we'll initially be looking for the answers we want to hear in AI and then dismissing the ones that we think are irrelevant or

Tom Davenport:

I don't know.

That's an interesting question.

I mean, you could argue that it's the opposite, that we have higher expectations for AI than we do for ourselves, at least in certain areas.

An autonomous car hits somebody and drags them along the streets of San Francisco.

That's big news that happens every day when humans are driving, and we don't seem to think anything of it.

So I think we do have pretty high expectations.

We get bad advice from humans in customer service calls.

We get even madder.

I think if we get bad advice from an AI system at a customer service call, although at least you may not have to wait as long for the bad advice if it's coming from an AI system.

Paul Sephton:

Yeah, that's an interesting one.

We've just completed a survey that for me, one of the most telling stats was we surveyed around 2000 AI decision makers, and what we noticed was that very few of them were in the C-suite.

Very few of them came from technical backgrounds.

And I guess to a degree, it's such new technology that you've got ask what is the right type of qualification to make these decisions?

And also, they were younger than the normal decision maker profile that we see when it comes to implementing tech.

Tom Davenport:

I'd love to see that data.

I have a little bit of data from looking at different types of tech executives, chief information officers, chief data officers, chief analytics officers, chief technology officers, and now Chief AI officers.

And it was quite interesting that we asked them to assess each other in terms of how connected they were to the business leadership in the firm.

And the chief AI officers were the least well connected to either other tech executives or to business leaders.

So it may be explained by A, they might be newer since it's a new job, and B, they might, as you say, they might be younger and less able to really discuss important business issues with senior leadership.

But I think that's a big problem, and I think you should not appoint someone to be an AI decision maker unless they understand your business.

They have your trust, you have their trust where you can really make some good decisions jointly about how to use this technology in the business.

Paul Sephton:

Do you think they should be on speed dial with the ceo?

Should they have sort of a buy-in across the C-suite?

Because obviously then you've got your CIOs and your CSOs and your CFOs, and they're all weighing in some way on those decisions.

So do you see a matrix of stakeholders who really should be involved in all of these decisions for them to be as successful as possible?

Tom Davenport:

I do, and it'll vary by organizations.

If you say, well, marketing is the way we're going to really succeed with ai, then clearly one of the primary partnerships should be with the chief marketing officer in your company.

But a lot of, when I talk to data and analytics and AI officers, they tend to say that they're opportunistic about it.

And because everybody doesn't get it and they don't want to waste the time to try to persuade them, they go with the people who are already interested and sympathetic with the idea of doing something with AI.

And that might be okay if those people are able to be sponsors and stakeholders for some really important initiatives and use cases, but if they're just enthusiastic and it's not going to help the company's perform as much, probably you should be more systematic and go to more effort to persuade the people who really matter in that regard.

Paul Sephton:

One of the other interesting things that we saw come up with the difference in what decision makers think and the difference in what knowledge workers think is when we asked decision makers what would time freed up from generative AI be spent on instead, most of their feedback was around more work, so more development training time to get more tasks done.

And when we asked knowledge work is the same question, their feedback was spending more time with my loved ones getting better work life balance.

So it's a really interesting difference in terms of the expected outcomes that we all have from this type of transformational evolution.

Tom Davenport:

Yeah, well, I guess that makes sense if we have a stereotype view of bosses and workers, but I think it's important to think about the benefits of this other than just doing more work.

I was talking recently to some of the researchers at Microsoft and GitHub about programmer productivity with generative AI.

They said, yeah, they have had some productivity increases, but a fairly high degree of increase in job satisfaction.

So that's worthy in itself.

It's really expensive to lose people and have to replace them and so on.

And happy workers tend to do a better job anyway.

So we need to look at that whole suite of metrics for how we evaluate the impact on the job.

Paul Sephton:

And Tom, I'm conscious that we're short on time, so I'm keen to get your final thoughts in terms of strategic vision and guidance for leaders in terms of you've laid out the six disciplines companies need to get right for things like behavioral or controlled experimentation.

What would you say businesses should be considering now so that when they look back at the end of 2025, for example, they have checked off everything and haven't been left in the dust?

Tom Davenport:

Well, I think, as I've said, I think those disciplines are important, but I also think it's important to democratize these capabilities and to sort of let a thousand flowers bloom and maybe you should be having the AI innovation of the week or something like that that people can learn from.

You can have little meetings where people share ideas about how they're using the technology.

So the highly disciplined stuff I think will be more top down, but you also want to have that bottom up level of innovation.

I think it will be useful to have these sort of many laboratories for innovation around your organization.

That would be one key thing.

Make sure you're using it responsibly and ethically and that you're not violating any regulations.

Unfortunately, we in the United States don't seem to believe that we need any regulation in the AI space.

I wish we were more like Europe in that regard.

But those are the main things.

I think.

Paul Sephton:

Following up from that briefly, there was an article I saw earlier this year that said, no one wants to sound clueless about ai, especially not your boss.

I think you talk about top down implementation, and then at the same time, bottom up, do you think there's some type of sweet spot in terms of having the right overall communication and guidance on AI policies, but at the same time leaving some wiggle room for people to bring their own types of AI into the workplace?

Because we've got to find a balance between regulating organizations and having that sort of leadership, but at the same time, leaving the space for employees at any level to bring new ideas to the table or try different things out.

Tom Davenport:

I think that it's really important.

I've found some companies that are doing a great job of democratizing it, having bootcamps really facilitating that bottom up activity.

But beyond that, I think people should be allowed to experiment.

There should be some limits in terms of what they can and can't do.

And if it's customer facing or involves regulation or whatever, then clearly there need to be some controls.

But to your question, sadly, I think time has passed when an executive can say, I don't really understand this AI stuff.

I wish you'd tell me more about it, because they'll feel stupid now.

It's been in the media for a couple of years, virtually nonstop, but still a lot of people don't understand it.

And I get a lot of people thinking AI was invented in late 2022 rather than 1956.

So I think there's still a lot of education to be done as well of senior managers as well as the employees that we were talking about earlier.

Paul Sephton:

We spoke earlier about the potential to implement a few MOOCs, but what do you think the best way to go about staying up to date on AI is?

And there's so much hype, like you say, if I'm reading headlines or wanting to educate myself, where do I go where I will get valuable information versus just a lot of noise?

Tom Davenport:

Keeping up with generative AI in particular is sort of a full-time job these days.

If you're really serious about it, we all have to get other jobs done.

So I think it's really important to think about as an individual and as a company, how can I really use this to make myself more effective?

And if you do a lot of presentations, then you explore how do you use AI to generate better presentations?

Or if you're a writer or if you're a podcaster.

I was just exploring this Google Notebook, lm, I don't know if you played around with it.

You can make some pretty amazing ones by just sending it a document.

But I think again, it's important to think about what do you want to do with this technology before spending a lot of time following every possible development?

Paul Sephton:

I think that's a great point.

Often people are looking at what can it do and then not thinking about the use cases they have for it first before figuring out if it can do that.

Well, I really appreciate the time today and the scope of content that we've covered.

So thanks so much for coming on today, Tom, it's been great to chat to you.

My pleasure.

Nice talking to you, Paul.

And that's it for this episode of Great Expectations.

If you want to find out more information about implementing the right AI solutions in your workplace, see more episodes or find our latest research, go to jabber.com.

I'm Paul.

Thanks for listening.