In this episode of The Parlor Room Presents: Hello AI, host and Harvard Business School Online Creative Director Chris Linnane speaks with HBS Professor Felix Oberholzer-Gee about AI’s impact on productivity, jobs, and competitive advantage.
He explains how organizations can use AI to create lasting value, why human judgment remains essential, and what the future of work may look like in an AI-powered economy.
Resources
Learn more from Oberholzer-Gee in his HBS Online course Business Strategy, which can be taken individually or as part of the HBS Online credential program Core Business Essentials
Catch up on previous episodes of The Parlor Room, featuring Oberholzer-Gee:
Compilation Episode (Part 1): Early Career Advice in the Age of AI
Compilation Episode (Part 3): Mid-Career Strategies for Thriving in an AI-Driven Workplace
You can also watch The Parlor Room Presents: Hello AI on YouTube.
Transcript
Editor's Note: The following was prepared by a machine algorithm and may not perfectly reflect the interview's audio file.
Felix Oberholzer-Gee: If we look at AI today, that's like looking at a Toyota in 1970. We have seen nothing yet.
Voiceover: This is Harvard Business School Professor Felix Oberholzer-Gee, one of the world's leading experts on competitive strategy.
Felix Oberholzer-Gee: You're thinking about the employment effects. It's always two things.
Voiceover: The conversation is just getting started.
Felix Oberholzer-Gee: Maybe the biggest open question, big opportunity has to do with—
Voiceover: Every organization is asking the same question.
Felix Oberholzer-Gee: Oh, absolutely. The productivity of individuals who use new technology goes up. It doesn't go down.
Voiceover: The future of work is being rewritten right now.
Felix Oberholzer-Gee: Many, many people will just be more productive. But if there's good demand for these kinds of people, people will actually make more money.
Chris Linnane: This is the best news I've heard recently about this.
Voiceover: Welcome to The Parlor Room Presents: Hello AI.
Chris Linnane: Before this episode starts, a quick message for you. I'm coming to you from one of the weirdest, coolest places on campus—inside the Chapel and the koi pond. I just wanted to say thank you for listening to this season of The Parlor Room, and please consider following the show wherever you get your podcasts. Thank you, and I really hope you enjoy this episode.
Before AI became something you actively studied or taught, was there a moment when you first came across this new wave of AI?
Felix Oberholzer-Gee: Oh, yes. So I think I first started with summaries, and I found it incredibly helpful. I get sent a lot of books, and I see a lot of articles, and there's no chance I can read everything. And so getting a first glimpse at, "Is it really interesting, is it close to the research that I'm doing?" that proved super, super helpful. And obviously, it got better over time with every release of a new model. The summaries became denser. I think they became more interesting. Also, they had more nuance. So that was maybe my first really positive experience.
And then I also ran into trouble right away. So I'm the treasurer of a small building in New York, where we live. And so the co-op is a legal entity. It's a weird thing in housing markets. And we needed a new mortgage. And we don't—because it's such a small co-op, we don't really have formal financials. And so I thought, "Oh my God, I have AI." And so I took, basically, all the transactions that we had, and I wanted to create an income statement and a balance sheet. It was a total disaster.
Chris Linnane: Oh, no. Why was it a disaster?
Felix Oberholzer-Gee: It didn't work at all. So it couldn't even add up. I was so specific about, "This is the sum in column X row, and so on and so on." And then the funniest thing was—because it always got it wrong. And then in the end, it asked me, "So, can you tell me the result?" And I thought, "Oh, I could, but that's not exactly how it's supposed to work." So it was both really interesting and impressive, and then also, it failed at things that humans wouldn't generally really fail at.
And I think this is—people now talk about the jagged frontlines of AI, where it's really amazing at some things, and then other things, almost inexplicably, it fails in ways that are really surprising. And so that was exactly my first experience.
Chris Linnane: And imagine if you—a lot of people just trust what they get, too. So someone might have just sent that off to their people who live in the building, and they would've been like, "Felix has really got to get moved out of this position."
Felix Oberholzer-Gee: Yes, that's right. Or we may not have gotten the mortgage because we can't add up—
Chris Linnane: Oh, boy.
Felix Oberholzer-Gee: —which is not a good idea.
Chris Linnane: So the summaries, that was a positive experience. When you did that, did it feel like something big was happening? Were you thinking, "Wow, this is pretty impressive," or were you thinking, "Now I'm a little cautious here?" What was the first impression of that?
Felix Oberholzer-Gee: It was impressive, I think, in two dimensions: the speed with which it would read and summarize, and now, of course, where we have the models, where we can actually see what is happening in the background, that's even more impressive how, it just consumes vast amounts of data in basically no time at all. And I do remember you couldn't really trust the results completely, as hallucinations have gone way down, even in the last year, year and a half or so.
So not so much in the sense that it's literally wrong information, but say, if I would ask, "Tell me a research paper that supports a particular point of view," it would give me the right set of research papers, but then when you read the papers, they don't actually say exactly what you would expect it to read.
So I think there is—I think, which is the future anyway—this machine-human collaboration where there are some things that the machine does really, really well, and then there are other things where humans are just much better. And it'll move over time. But I think collaboration between machines and humans is what the future is all about.
Chris Linnane: And you mentioned that it started to show you what it was thinking along the way. I find that unsettling myself.
Felix Oberholzer-Gee: How so? Why?
Chris Linnane: Because I put it in, and I was like, "Oh, this must be an error." I'm seeing behind the scenes, it's telling me what it's thinking. And it says things like, "I need to be honest." And I'm like, "Of course, you need to be honest." The fact that you're checking that along the way, that is crazy. I don't know if that's sent way to be shown to me, so I feel like, "Oh, it's really thinking about the right things." Or, if it truly is using those plot-by-plot directions.
Because if it's using those plot-by-plot directions, I can see where the confidence drops, because you're like, some of these things should be just given. But it's telling me accuracy is important. I'm thinking, obviously, but why is it telling me that? I know that. So that part just makes me a little uncomfortable.
Felix Oberholzer-Gee: I guess it has two dimensions. One is, even when you download a document or when you save a document, it takes a little while. It's typically visualized. And the reason that we do this is that it feels much less onerous. So you feel like, "Oh, people are way more patient if they even see a little circle." That is basically nonsensical because you're waiting the same amount of time, but it's different because you're thinking, "Oh, something is happening in the background, and it's looking up all of these documents."
And then the second thing to say is this is, of course, a little bit state of the art that the big foundational models can do really amazing things. And I think one of the biggest tasks of the engineers at Google and Anthropic and everyone else is, how do you rein in what the model would do in a way that it gets closer to what users expect? So Gemini reportedly has around a million lines of code to rule out the kinds of things that AI might do if it were completely unsupervised.
Chris Linnane: Wow.
Felix Oberholzer-Gee: And so we should think that there's probably—depends a little bit which company we're talking about, but there's—a year or maybe even two years' worth of progress already exists in foundational models, but they're not released at this moment in time because the engineers are working on reining in what the model reasonably can do. So we have, I think, a big task in that these models can be really powerful, but they can also do many things that are undesirable. And I think this back and forth, I think, is in part what you see and, in part, what then creates the differences between the different models.
Chris Linnane: You mentioned speed a few minutes ago. A lot of AI feels like speed rather than real change. And how do you think about productivity gains that actually last and don't just fade, and all you got was speed at something that dies out?
Felix Oberholzer-Gee: I think of speed as something very positive. And I'll give you an example. In the Walmart Fulfillment Center, productivity has increased by about 30 percent. That means Walmart can do now much more business given the same warehouse infrastructure without hiring additional people. And that, of course, influences the unit cost very dramatically. And eventually, probably some of that goes to the owners of Walmart, and some of that will be passed on to consumers.
So the productivity gains that we start seeing now, many of them come from the fact that you can do things more quickly than it used to take you a long time to do X, Y, Z. I think JPMorgan had, like, 400 use cases of AI. And if you looked at what is actually happening here, many of them revolve around—we're doing the things that we've always done. We're just doing it much more quickly.
The bigger question, I think, that is related to that—so say you're using AI for some tasks that you have always done, and now you're five minutes faster. For productivity questions, of course, the big conundrum is like, "What's going to happen with those five minutes?" If you then think about the results for five minutes more carefully, that might be valuable. If you use it to eat your lunch a little more slowly, it might be good for your health, but it's probably not super productive for the company.
And so some of this, I think, is shifting time around. But some of this is already impacting unit economics, actually, in quite an interesting and dramatic fashion. And one of the things that I see that is very interesting to me is that some people's view of AI is that it's going to increase the willingness to pay of customers, which, of course, would be value-creating, so that's really good. That's good for customers. That's good for the company also.
But maybe at least at this moment in time, the more consistent gains will come from really boring things where unit costs fall by 10 percent. You get your results a little faster. So I would not be surprised, at least in the beginning phase of AI, that we see many, many things that just reduce cost. And then for companies, the big question is, so that's really amazing, a 30 percent decrease in cost for a transaction in fulfillment centers at Walmart. That's a really big number. What if everybody does that?
And then, of course, we're back in this situation where we get technology, everybody uses it in very similar ways. We get productivity gains in the beginning, and ultimately, because of competitive forces, most of these productivity gains will go to consumers. This is a little bit of the history of technology since, say, the early 1990s. If I show you total labor productivity in the economy over time, what you would see is really dramatic ups and downs that correspond, essentially, to the business cycle.
So, the big 2008-2009 recession, you see COVID, really, it's a really big change in productivity, but you don't see technology. So it's not as though we went from an average of two percent productivity gain per year to now three percent, or four percent, or percent. There's no acceleration.
And in part, I think it's because so many of the benefits ultimately went to consumers, because companies did things in more efficient ways, but everybody started using Outlook, and everybody started using SAP, and everybody started using a very similar set of tools. And that means, yes, we can do much more, but who benefits in the end? Not really the companies and the shareholders. Many of these benefits went to consumers.
Obviously, nothing wrong with that. But it does open up this question: If both the production of LLMs but also the applications, if they tend to be really expensive, the return on these investments are not completely clear. And so much will depend on: Can companies find ways to deploy AI in a way that is differentiated from the competition?
Chris Linnane: So differentiation will definitely help for sure, but are there other things people are missing along the way, like the benefits going to the customer? Like we said, that's great. Are there things that they're missing at the same time, besides differentiation, that they should be thinking about as well, or is it truly just standing out from the rest because we're all using the same tools now?
Felix Oberholzer-Gee: I would say maybe the biggest open question, big opportunity has to do with if you automate the processes as they exist today, you'll get a little bit more efficient. And that's nice. And in particular, if everybody around you is doing it, you'd better do that as well, because otherwise, you will fall behind very quickly. But if you look at the history of technology, actually, most of the gains most of the time came from reinventing the processes.
So I'll give you an age-old example. So you remember Edison invents the light bulb in the late 19th century, and then, actually, quite amazingly, in a very short period after that, there's a power plant in Manhattan. And so electric power—there's a market. Electric power comes available. And you think, "Oh my God, this is going to change everything." You look 20 years after electric power comes available, less than 10 percent of factories actually are using any sort of electricity.
And the question is: That is such an obvious novel, interesting piece of technology, what is it about these companies that they don't understand that they can do many things with electric power that they cannot do with steam? Now think about what a factory looks like at that moment. You have a central steam engine that produces power, and then from that central steam engine, you use basically connections to drive all the individual machinery. So what if you just replace that central steam engine with an electric motor? There's a little bit of a benefit. You don't have to worry about coal and all of that stuff. But the benefits are really small.
Where do the big benefits come from? So once people realize, if I use electric power, actually, I can now have a motor at each machine and reconfigure the whole production process, that then brings the big productivity gains. And that's a little bit the way to think about AI today. If you superimpose AI on the processes that we have today, you're a little faster, maybe you're a little better, your cost might go down, but it's not going to be the big thing. The big thing will come from rethinking what these processes need to look like in the first place.
So, I'll give you one example. If we onboard someone, how do we make sure this new person actually understands what we do, does the work in good quality, even though the person hasn't really been in the job all that long? The answer is process. We tell you, "OK, so here's a customer order, and this is what happens next, and so on and so on." So we've tied quality to a particular way of doing things.
Now, one of the big strengths of AI is that it can figure out how to do things radically differently. That's interesting about the history of AI more generally. If you think back to Deep Blue, the first supercomputer that beat humans at playing chess, it's essentially a superhuman. So it plays chess like a human person. And the only difference is really that it's incredibly fast. It can look at tens of thousands of potential combinations of moves in ways that humans just cannot.
But the really big breakthrough comes much later, when Google and others try their hand at chess and go where the process is largely unspecified. AI figures out how best to play chess, how best to play Go. And that's the metaphor to keep in mind. I think we've always had technology that ultimately required us to rethink what these processes look like, but now, interestingly, I think for the first time, we have technology where, actually, maybe I don't really have to think about what the processes are. Maybe I can let the technology figure out, you get, I don't know, a stack of resumes with applications, and I have a way to review these applications. Maybe AI can do it in a very different way.
And the intuition to have is that usually, you can tell the technology, "What does good look like?" So, say, if I had really big success with hiring a particular type of person, just specify, "This is what success looks like." The way we did it when we taught AI to play a game, "This is how you win. This is what success looks like." And then how exactly it's going to do it, I think, we will not know. And that, I'm guessing, will make people like you a little uncomfortable.
Chris Linnane: It will make me uncomfortable.
Felix Oberholzer-Gee: Because you like to know what is happening under the hood. But as long as the results are great, why worry about the process?
Chris Linnane: Can't complain if you're using it and you're happy with the results. Where do you see AI creating the earliest and most reliable productivity gains in organizations?
Felix Oberholzer-Gee: So there's definitely big progress when you think about coding. You look at companies like PayPal, and their developer productivity is up anywhere between 10 percent and 30 percent. So I think that's an obvious application. Anywhere where you need to consume large amounts of data in a short period of time and get reliable summaries, I think that's a really interesting area. And I think the part that is already very interesting, so now people think, "Oh, AI can do it much more quickly, and we increase productivity. And then, before you know it, we will all be out of jobs."
And then you look at the data. In the last three years, in the US economy, I should say, we added about three million white-collar jobs. We added seven percent software engineers, 10 percent radiologists, 20 percent paralegals, exactly in the areas where you think, "Oh my God, catastrophe impending." And then you look at the data, and not really.
Chris Linnane: So that's the one specifically everyone uses as an example.
Felix Oberholzer-Gee: That's exactly right.
Chris Linnane: Why are we getting that wrong?
Felix Oberholzer-Gee: So maybe I'll tell you another history lesson.
Chris Linnane: I love history.
Felix Oberholzer-Gee: OK, good. So imagine an old-fashioned bank. Part of what bank tellers do, what keeps them really busy, is dealing with cash.
Chris Linnane: Oh, so this isn't the old-fashioned Wild West with the thing on their head. OK, I was imagining that bank. Sorry, old-fashioned 1980s bank. OK.
Felix Oberholzer-Gee: Yeah, so we're not going back quite as far. So they're really busy. People come in and pick up cash, the businesses bring back cash, and so on and so on. And then the ATM comes along, and everybody thinks, "Oh my God, bank tellers, doomed. No chance." And then you look at the data, and over the next 20 years, we're adding a lot of bank tellers. And you go, "What on Earth is going on?" And I think it's very similar to what's going on in software and so on and so on.
So the story is twofold. As ATMs come along, the cost of operating branches falls, and so banks have many more branches than they used to have. And then we shift what bank tellers do from something like handing out cash, which is really not that valuable for the bank, to giving advice, to encouraging investment, and so on and so on. So, what you should imagine when you're thinking about the employment effects is always two things. If any sort of technology drives down the cost, and if, ultimately, we pass on some of these savings to consumers, demand will rise. And so we get more business activity. That alone, I think, cushions a little bit against big employment losses.
And then the second thing that's always happened in technology is that, oh, actually, the productivity of individuals who use new technology goes up. It doesn't go down. You can do more, and you can do smarter, better things. They used to hand out cash. Now I can give investment advice. And as a result, actually, both the number of jobs and, very often, the compensation goes up.
It's exactly what we see in software today. Compensation goes up very quickly. Why? Because, oh, now having a software developer who can actually use all the AI tools—you just have more output, and the more output is more valuable. And in a semi-tight labor market, we can sometimes pay people less than their marginal productivity. But generally speaking, if you try to understand why is it that some people make more money than others, I think looking at differences in productivity is a good first step.
And so your expectation should be, yes, there might be jobs that are negatively affected where we use fewer people. But the bigger story for white-collar work in general is probably that many, many people would just be more productive. And if the labor market is maybe not super tight, but if there's good demand for these kinds of people, people will actually make more money.
Chris Linnane: This is the best news I've heard recently about this, so I'm quite happy to hear your version of things. I'm fully subscribing to.
Felix Oberholzer-Gee: So I should say there's academic research out there that talks about these entry-level positions that we do away with, and the evidence is not as clear-cut because it actually, the paper that has become quite famous starts about a year, a year and a half before even the release of OpenAI. Think of every job as a bundle of tasks. There's really no such thing as a job. Every job includes many, many, many, many different tasks.
And if you happen to have the kind of job where every task can be automated, I think we will see far fewer positions, and probably you're not going to get paid that well. And there's some indication that in the market for back-office work, that has started to become visible. But for most tasks, it will be parts that you can automate and then other parts where human judgment remains important.
One of my favorite examples is air traffic controllers. So, air traffic controllers now use pretty amazing technology to make sure we don't get collisions, and so on and so on. And so their most important task revolves around these edge cases. "Oh, it looks a little dicey. Should we? Should we not? Is it a problem? Is it not a problem?" And so essentially, it's this collaboration between air traffic controllers and a lot of data and a lot of machinery and a lot of history that we have.
But because air traffic controllers now provide these incredibly valuable services, their compensation has actually gone up dramatically. And so that's a glimpse of the future, I think, where we will have lots and lots of jobs. It's already true. The general notion is that if you look over the last 30 years or so, there are hundreds and hundreds of jobs that didn't really exist before computers because computers generated new types of jobs.
Already today in AI, we have—I guess what we now call "forward-deployed engineers," engineers who help companies implement AI products in the context of that industry or in the context of that business. That's not really a job we had before. And who knows how labor-intensive this is? You have to go in.
And yes, some of it you can probably automate, like, "How do I create data pools and so on and so on?" There are elements that are open to automation. But then the judgment, "Is this client going to face a revolt if I try to do X? Does this particular implementation match the general culture?" There's so much judgment at one and the same time.
And so I think these forward-deployed engineers, they will have good human skills to read situations, understand corporate cultures, and then see how technology can best be deployed. That is so much harder to automate. And so for a long time, we will see that these people are really valuable.
Chris Linnane: Wow. If you looked at your position—someone's sitting here, and they're looking at their job as a bundle of tasks, and they're seeing some things that, "There's a good amount of automation possibility here in my bundle of tasks," what's a question or what's a system that they can go through themselves to say, "How do I fix this problem?"
Felix Oberholzer-Gee: Maybe the first thing to think about is, if we look at AI today, that's looking at a Toyota in 1970. And I think our first mistake is to say, "Oh, look, it can now do X. I'm totally OK because X." No, we're looking, you're looking at that 1970s Toyota. And I don't even know. Did Toyotas exist in the '70s?
Chris Linnane: Yeah, I'm sure they did. I'm sure they did.
Felix Oberholzer-Gee: Yes, so we're looking at a way early version. We have seen nothing yet. These systems will become so much better. And so I would say, if you're thinking about the bundle of tasks that you perform and you see that the technology cannot do a particular set of things, I think that's the basis to believe that you'll be really valuable. And probably, as your productivity goes up, you'll be—like, say, project managers are in crazy demand right now because project management is exactly—in the life of a project manager, no day looks exactly like the other day. There's always all this stuff happening.
So you should feel pretty confident. If you're thinking, "No, actually, my day is pretty much the same every day, and I do the same set of tasks, and right now, AI is really terrible at doing these tasks," that should not give you a huge amount of confidence simply because the models will become so much better.
Chris Linnane: That's a good point. Now, since you brought that up, is there anything that people are trying to make happen for AI right now that it's not ready or it's not working well?
Felix Oberholzer-Gee: I mean, the degree to which it can reliably give you something that looks like human judgment, that is still super hard, because humans are doing something pretty darn complicated. And in part, it's because—stick five humans in a room—or I should probably say, stick 93 humans in a room and ask them for judgment, that's the HBS classroom, and you will never get agreement because this is a super hard thing to do.
And so I always encourage everyone who's working with AI: just think of it like if you're in a meeting, and your smartest friend says something. You probably have priors that, "Oh, he's a really smart person. I should give the view or that opinion a decent amount of consideration." But you wouldn't think this person's always right or this person always knows better.
And AI is exactly like that. It throws out pretty reasonable suggestions often. But, of course, just like when we interact with other humans, it's like, yeah, sometimes it's great, and sometimes it's not so great. And so I think for quite a long time in areas where there is a lot of judgment, there's a lot of surprise—imagine if we took, like, a project manager and we were trying to automate that person's life. Super, super hard to do. The bits and pieces, yes, of course. But the thing that the project manager never really thought about, and then it's happening against all expectations, that's going to be surprising to the person and surprising to the technology.
Chris Linnane: Can we play a quick game?
Felix Oberholzer-Gee: Of course.
Chris Linnane: All right. So what I'd like to do. So, I've got these paddles. Now, just to start it off, we were just talking about there's a lot of hype with AI going on right now. And if we talk about this as overrated or underrated, is the hype overrated, underrated, or just about right for different kinds of things? So I'm going to name a few common ways companies are using AI at work, and for each one, I'd like you to tell me whether you think it's overrated, underrated, or just about right right now, the hype part of it.
So here, you've got these. And for the people watching at home, they can see these. People listening, we're going to announce as well.
Felix Oberholzer-Gee: And I assume I have to explain myself?
Chris Linnane: I would like it if you did. Otherwise, we might get the wrong idea. All right, so let's go through—
Felix Oberholzer-Gee: That's the bad news.
Chris Linnane: Scenario 1: AI writing internal emails and documents. Is that hype about that right now overrated, underrated, or just about right?
Felix Oberholzer-Gee: I think that's, if the hype is that AI can do the vast majority of emails, so I'll give you—
Chris Linnane: So you said just about right, just for the people listening at home.
Felix Oberholzer-Gee: Yeah, just about right. I'll give you a good example. For reasons that are not totally clear to me, I get a lot of email from prospective PhD students who basically just write, "Mr. So-And-So, I'm interested in your PhD program. Can you tell me more?" And, of course, my answer is always, "Well, it's very nice that you're interested in our PhD program. And guess what? There's a website that will actually tell you almost everything." So that kind of thing, of course, I no longer answer these myself, because there's a very polite and nice and encouraging way to answer, but it's always the same.
Chris Linnane: It's obvious.
Felix Oberholzer-Gee: It doesn't really change. And so for that, I think for all that kind of communication, there's no hype. I think all of this will be automated.
Chris Linnane: OK, scenario 2. Are you ready? I'd like to say they're going to get harder, but I'm actually not sure if they're going to get it right. AI summarizing meetings instead of people attending live.
Felix Oberholzer-Gee: Oh, I think—
Chris Linnane: Just about right again?
Felix Oberholzer-Gee: Just about right. I think it's near perfect now.
Chris Linnane: Really?
Felix Oberholzer-Gee: One of the things that I have found is, so you have a summary of a meeting, and it's pretty good at summarizing where everybody came out. We agreed on seven things, and the seven things you'll see. But it doesn't give a good sense of the back and forth, or it doesn't give a good sense of the differences in opinions. You can now just go back and say, "Give me more nuance, give me a sense of the pros and cons on the seven points that you just listed," and it will do that.
There's research out of MIT that has these iterative prompting techniques where you feel like, huh, but I heard, if I remember right, there was a pretty powerful counterargument, and I don't really see it. Just ask, "What was the counterargument to what we decided?" Or, "For each of the points where we came out, give me the two most important pluses, the two most important negatives." And it will do that. It will do that beautifully.
Chris Linnane: See, I would have thought you'd say overhyped, because the nuance part does scare me. If I get a message from a friend and I tell them, I sent him a text message, maybe, and I say, "We'll meet at two o'clock, and the kids are going to play soccer or whatever. Is that OK with you?" And they respond, "Yep." Like, "Y-E-P." I'll look at it like, "Yep?"
That seems dismissive. They must be mad at me. They would say "yes" if they wanted to do this and they were happy with this plan, but they said "yep." And then I'll go into this thing internally being like, "Did I phrase this incorrectly?" And then I think, "Oh, I'm reading the context wrong because it's text." And I'm not getting the, who knows what they were doing?
The meeting notes feel like a wildfire for those kinds of missed cues for me. So I wonder, nothing's saying, I don't think Rachel was quiet the entire meeting. She obviously wasn't happy with what was going on, or Thomas used a dismissive tone or something. Those are things I'm going to pick up in a room, or in a Zoom room, or whatever, that I'm afraid it's going to be missing. It's going to be the "Yep" over and over again. I'm like, "There's 40 'Yeps' in this thing. I think everyone's mad." And I'd miss that.
Felix Oberholzer-Gee: So I will say part of it is, of course, what's the data that you capture. We're not quite there yet, but you will have video of the meetings in the future, and you will see a person with folded arms who looks angry, and so on and so on. You will see who doesn't speak up, and so on and so on. So that, I think we're fairly close to having these kinds of models.
Of course, does every perception that you have in a meeting, all the information that you pick up, is that going to be reflected in that document? I think the answer is no. But now think about we're in a meeting with five people, and we ask everybody to summarize. Everybody's experience of that meeting will have been different. And so this goes back, to think about AI as a really smart person in the room. That description of the meeting is not exactly your experience, but it's also probably not so different from what other people thought about the same interaction.
Chris Linnane: You probably wouldn't walk away with wrong information. You'd walk away with a variation on the same understanding.
Felix Oberholzer-Gee: Yeah. And always, I think one of the really fascinating things is if you're unhappy with what you get—I mean, the models already are incredibly good at responding to, "I need more detail. I need dissenting views. I need to know, 'Give me a sense of the time that we spent talking about X, Y, Z." So I would often say with bigger—when I try to decide, should I read a book or not, so I have AI read the book, and then I don't really have a, the summary doesn't give me a good sense. Did it dwell on two of these points? Basically, the book is like two of these points, and I happen to be not interested in these two points. So I want to know. And you can definitely go back and say, how much space does it take up?
Chris Linnane: That's interesting.
Felix Oberholzer-Gee: Is that really important? So in a way, the trick is to treat it like a person. And what you would, if you tell me, "Did you read book X, Y, Z," and then I talk to you a little bit about the book, the next logical thing for you is to then ask 13 follow-up questions. And I think the same is true for AI.
Chris Linnane: Yeah, I think the back and forth is a part I'm not thinking about as much. So I'm thinking, "All right, I'm going to read the summary or get the summary, I'm going to miss it." But the point you just made, which I hadn't thought about asking back, like, "How often did we talk about this? Did anyone bring up that?" Using it more like a, not just a one-way street, but a two-way street. I don't know why I didn't think of it as a two-way street. It does become a better idea as a two-way street.
Felix Oberholzer-Gee: Yeah.
Chris Linnane: All right. Let's move on to the number three scenario. So remember, this is—
Felix Oberholzer-Gee: I have these underused—
Chris Linnane: It's OK. If they have to be underused, they have to be underused. So is this overrated, underrated, or about right on the hype scale right now: AI helping leaders decide which projects to stop or prioritize? Is that overhyped, underhyped, or just about right right now?
Felix Oberholzer-Gee: So I'm not sure about—if the claim is that it can perfectly do that, that claim is not right.
Chris Linnane: So that would be overhyped if they're saying you can rely on this thing, no questions asked.
Felix Oberholzer-Gee: Yeah, for exactly the reasons that, I think one good way to think about, two colleagues of mine, they had this interesting way of thinking about when should you rely on AI and when is it OK not to do so. So Andy Wu and Bharat Anand, they have—the source of information is explicit data that you have somewhere, versus tacit knowledge. The history of experiences that this particular executive has, that's a body of knowledge that is incredibly important in deciding should I buy the company, should I not buy the company. But it's not data that you can easily access. It's an intuition. It's a sense that you have. It's human judgment.
And then the other dimension that they talk about is, how costly is it if you're wrong? And even though it's a super simple two-by-two, anything that is in, there's a lot of tacit knowledge, probably AI will perform not quite as well as the typical human person who has a lot of experience, which doesn't automatically mean you shouldn't use it. Because then you ask—OK, so it's probably not going to be very good—is it costly if I make a mistake? And the answer is no, actually, it'll be OK. Then, of course, I can still use it. If it's the acquisition of a company and I get it wrong because I have AI decide, then that's probably not a place.
So this goes back a little bit to thinking about jobs as bundles of tasks. The more tasks rely on tacit knowledge, I think, the harder it will be to really take the human out of the equation.
Chris Linnane: Good. All right. AI forecasting demand or sales: Currently, right now, if it's saying it can forecast, it's got it under control, is that overhyped, underhyped, or just about right?
Felix Oberholzer-Gee: So I think, actually, it's even better than that. We know that many people don't really trust it that much, so you see it in debates around scheduling people and scheduling workers. It's been true for quite some time that both traditional methods already, but then AI definitely can do a much, much better job than humans.
Chris Linnane: So I think that's considered underhyped. We can get a lot more out of that.
Felix Oberholzer-Gee: But somehow, many people who, so think about retail, where we have this complicated scheduling mess. Many managers who do this, it's a big part of their job, and they think they have good intuition. And they know when it's raining, I use fewer people, and Kathy and Robert probably shouldn't work together because they don't like each other, and so on and so on, all of these kinds of things.
But actually, the reality that much of the research shows, no, the manager is not better. And so I would say this is, even technology before AI, and with AI, definitely, it can just forecast things in ways that are really hard for humans. And also, say if you're thinking about pricing models. Because your prices change in response to the prices of competitors, and that happens a million times a day. If everybody, like, say, competition between e-commerce companies. And there's pretty good academic evidence that if you do it more often and if you do it more quickly, that actually gives you a competitive advantage relative to people who move more slowly.
Chris Linnane: All right. Let's move on to the last one: AI screening or ranking job candidates. As it stands right now, is it overrated, underrated, or just about right? Screening and ranking of job candidates.
Felix Oberholzer-Gee: So I have a project, it's early days, where we trained a model to mimic the decisions of HR managers. And it's doing that pretty reliably. If you get a stack of resumes, and we ran these tests where the humans choose who are the 10 best candidates and the machine chooses. Super, super, super, super close. But this is also an example of, because we know we're not very good at picking the right people. This is a good example of maybe we should have a completely different process to do this.
So one of the things we're trying now, I don't know. Maybe it'll work. Maybe it won't work. We fed the productivity of individual teams into the model. And the idea was, don't pick a really good candidate who looks good. Think about, ultimately, what we want is the productivity of that team. Pick the candidate you think makes the biggest contribution to team productivity.
So now we're basically saying, don't think like a typical HR manager or a hiring manager. Think about, "What does good look like?" And here, in this particular case, good is just high productivity. And can we somehow expect that, I don't know, AI picks up on things that we don't pick up? And so the first indication that we have is that it now picks different people from the HR managers.
So it's no longer doing what humans used to do. We don't know. It's too early to say whether whatever it's doing will turn out to be better than what humans can do, but you see now, all of a sudden, before, it was basically the same people, and now there's this interesting difference. And that's maybe the right way to push forward with messy processes where, I mean, we have a process in place, but the process itself is sort of working OK, not that great. Maybe trying to figure out whether AI can actually do a better job, I think, is a good thing to try.
Chris Linnane: All right. So I think you see opportunity there. So you must think it's slightly underhyped right now. There's more to it.
Felix Oberholzer-Gee: So I would say there's probably upside. I can't know because there are so few, there's so few cases where people are really serious about letting AI reinvent. And you also get, so Google created a product that ultimately didn't really find that much appeal with newsrooms, but the idea was every journalist comes to work in the morning, and you spend quite a bit of time just figuring out, "OK, my reporting today, what should I write about?" What if AI already prepared the universe of things that have happened from all the sources that are out there, and then it gave you 10 potential stories? Because we know journalists have standards that are really hard to emulate, we still let you do all the reporting.
But even that tool, again, I was recently in a newsroom conversation, even that tool, the idea that AI would at least prepare the topics for you, it wasn't really all that popular. And so I think this is the other thing that we will see. Where it's going to be applied and how it's going to be applied, as always, doesn't really have to do that much with the technology. It has little to do with what the technology can do and what it cannot do, but it also just has a lot to do with what the human response to, what's the organizational response to the changes that come along with new technology.
Chris Linnane: When AI does improve productivity, what has to be true in order for those gains to stick?
Felix Oberholzer-Gee: And maybe if I can add—
Chris Linnane: Anything you'd like.
Felix Oberholzer-Gee: I can twist your question a little bit. I think the productivity gains will stick. But the big question is, who benefits? Is it the company and the shareholders who paid for the technology, or is it the consumer? And I think this goes back to the earlier conversation around differentiation. If we can't figure out how to use AI in a fashion that is differentiated from everybody else, then it'll be really great for consumers. So many things will be so much cheaper.
I'll give you an incredible example. As you probably know, youth unemployment is quite high in China right now. So in particular, if you're a graduate from an engineering program, it's super, super hard. The government tells us that youth unemployment is somewhere in the 20 percent or so. We don't even really know if that's true or not. So what do all of these engineers do?
Totally amazing. Instead of collecting unemployment insurance, they all start their own companies. And because they're engineers and because robotics is so big in China, they all have studied and learned robotics. The HBS Club of Shanghai organized a conference, and I met some of these entrepreneurs at that conference. Everybody has, like 17 ideas—robots plus AI, what they could do.
And sometimes, it's really, so one guy explained to me when car companies build a new seat, what you want to make sure is that that seat is going to stay comfortable for a long period of time because people spend a lot of time in their cars. So right now, we have these robots that basically mimic people sitting down, like five million times, to really know if the design is good.
With AI, you can do this much more quickly and much more effectively. I don't really know, but his claim was about 10 percent of the time that it took to simulate all the, because you get feedback from the test, and then AI can decide relatively quickly, for this kind of movement, the seat is actually totally OK, but it looks a little dicey when you do different things.
And because there's now this added intelligence to this process, you can actually test new designs of car seats in a fraction of the time. And so that's fantastic, so that lowers the cost of the car manufacturer. And if, in fact, it's only one car manufacturer that now has this technology, really great. If tomorrow, everybody does the same thing, cars will be cheaper. So either way, it's good for someone.
But the big question is, where will the big gains flow? Is the AI story mostly a story where you and I, we'll meet over beer and say, "Do you remember how expensive stuff used to be? We used to pay so much for all of these ridiculous products and services, and now it's not exactly free, but almost free." Or is it that because humans become so productive with AI, that most of the benefits flow to people who work productively with AI? They have a ton of income, but maybe product prices don't really come back because we re-engineer the jobs in ways that make them really valuable for many companies.
So, I think that value creation, I think, is almost a given. The more interesting and more complicated question is, where will it flow? Will value stick with the companies? Will it go to consumers? Will it go to people who work in smart ways with AI? And I think we'll see a big variety in some places where it really goes straight to the consumer, and some places where it's probably good for people who work with the technology.
Chris Linnane: I have this thought. It feels pretty solid in my head, and I might be wrong. I should say that—
Felix Oberholzer-Gee: That's true for about half the thoughts that I have.
Chris Linnane: I feel like that should be a new thing I say before I say any sentence to anybody. But, this idea of differentiation and advantage, it doesn't seem like it could exist going forward. You can be the first to something, it feels like, but anything technology-based, it's just a matter of time, be it an hour or a year, someone else can do the exact same thing. Doesn't it feel like we're on either a slow or a fast track to sameness across the board and no differentiation?
Felix Oberholzer-Gee: So the traditional sources of stickiness or of the difficulty of imitating, they remain as always, so scale, I think, if it comes in the form of fixed costs, creating economies of scale, or network effects, if it comes in the form of the more people use a particular product, the more valuable it becomes. So that I think the technology will not change at all. And then the question is, just on the ideation part, what role will ideation play? And I think you already see pretty good and convincing evidence that in some of the entrepreneurship contests that exist, they now have teams that don't work with AI, teams that work with AI, and then AI freestanding, like no human involvement. And already, it becomes visible that, every now and then, AI has really great and powerful ideas.
And so if we're thinking about going back to this analogy, AI is just a smart person somewhere else, actually, in expectation, that will increase the number of new and novel ideas that we have. And it'll be, sometimes the mistake that humans often make in that thinking, "Oh, but I'm going to do it somehow better." And you already look at it and say, "No, but there's, like, 18 people who already work on this idea."
Chances that you just imitate so that we get these waves of imitation, maybe AI is a way to get less of that because we can say, OK, you'd better think of a better idea that is not exactly what everybody else is doing. And so in that smart, one of the, so rural population will shrink relatively quickly now because people decided not to have kids. And one worry is that in a smaller population, we will have fewer really big ideas because there are just fewer people.
And as a result, the wealth increases over time that we have historically seen in the last 200 years or so, they will become smaller and smaller because we just, say, if you and I are left. Well, OK, so we're probably not going to invent anything really big. And maybe AI is, in part, a way out of that dilemma so that we can substitute for the smaller numbers of people who have ideas by using technology and asking technology to come up with new stuff.
Chris Linnane: That's an interesting way to look at it. And you started so positive, Felix, and then you brought up that we're going to lose population, and now we're going back down again. We talked about that once before. It always makes me sad. Are there ways that AI can look productive, but long-term, it actually is worse for an organization?
Felix Oberholzer-Gee: You can imagine people falling in love with the activity, so interacting with AI endlessly. And maybe you'll have really drawn-out ideation processes that, in the end, don't really lead to anything really big. Because the way we interact right now, that we speak to it or that we type, is not the most amazing way to interact with technology.
So I think that will definitely change so that we can interact in ways that, the way we interact with humans, and so on and so on, going back to the meeting example. But generally speaking, then, of course, if that becomes, say, already now, if you want that, the models can be incredibly funny. The models can be people who fall in love with their bots. The models can feel like your romantic partner.
If that happens, it's a little bit like people wasting time on Facebook, and Instagram, and TikTok when they're at work. And so much of that, I think, is not really different from earlier technologies in that if we can get people to use it in the most productive ways, we'll benefit. If people use it as a really fantastic distraction, then we need to think about how to manage that tension.
Chris Linnane: Now, I've got a story for you, then, that ties this in. In a previous episode, I was talking to Jeff Bussgang, and I pitched him a horrible idea, I'd like to say on purpose. I'll say it was on purpose. But I wanted to test the AI model afterwards, because I pitched this before I thought about how these models work. But to talk about maybe leading you down the wrong road, I pitched an app called Holy Rollers.
And for those of you who've heard this before, I apologize. It's a sermon-sharing app for religious leaders. So a rabbi can write a great sermon, and then they put it up on Holy Rollers. You have to be a member. I think it's probably $9.99 a month. And then a priest somewhere could be like, "That's a pretty good sermon, and I'm just going to take that and hit 'Catholicize,' and it turns into my thing, and we share it." It's a really bad idea.
Felix Oberholzer-Gee: How so? I love it. I think it's a great idea.
Chris Linnane: I think it's a bad idea because I think now you don't even need it. But to do my test, I was like, you know what? I'm going to pitch this to ChatGPT. And it told me it was such a good idea. And I kept saying, "Try to be more critical." And it was critical in certain parts, but it kept giving me workarounds to make it still work. But I could try to make Holy Rollers for the next six months. It would be a huge waste of my time. And I could see companies could be doing that all over the place right now.
Felix Oberholzer-Gee: I mean, so this is not exactly the same, but think about the apps that we have today where teachers share lesson plans, which is not so different from your idea, where basically, it worked somewhere else. And maybe I tweak it a little bit. I don't know that much about religion, but if, I'm guessing you do this all the time, maybe every day—
Chris Linnane: Every week, you've got to have a new sermon, yeah.
Felix Oberholzer-Gee: So 52 ideas. So yes, of course, instead of you're starting from scratch, if you could add a twist to something that was really great, I think that's fantastic.
Chris Linnane: You just changed it. It's not Catholicized. It's Catholic twist. And then it's Protestant twist. That seems more user-friendly and fun that way.
Felix Oberholzer-Gee: Yes, exactly.
Chris Linnane: But—
Felix Oberholzer-Gee: So what's wrong? Why is it a bad idea?
Chris Linnane: I think it's a bad idea, but I think, I honestly do. But—
Felix Oberholzer-Gee: I mean, I wouldn't pay $9.99 because I'm not religious. But if I was a priest—
Chris Linnane: $4.99? But yeah, but I guess my point is that it's hard to, at least from my experience, because I've tried a couple of things where I've tried to just be, "Talk me out of this," and it doesn't quite talk me out of it. It always kind of gives me a workaround to "But you could try this." That feels not dangerous because it's not. You can do whatever you want with it. But it feels like you could waste a lot of time chasing bad things, because it's not critical. At least many of these aren't critical, even when you ask them to be critical.
Felix Oberholzer-Gee: Yeah, yeah. So this is a super interesting question because it goes back a little bit to the economic incentives of the companies that provide the foundational models. You might remember, this is now maybe almost a year ago, where an update to almost all the models left all these models with way too much agreeing with their users. And eventually, the companies pulled it back, and they added more criticism.
But this is a little bit of this old age, well, I shouldn't say, it's not that old, but, say, think about social media. So my job is to entertain and to make connections, but I also want you to spend endless time on my app because it's advertising-financed, and the more time you spend, and so on and so on. That's basically how we created the mental health crisis for teenagers.
And the economic or financial interests of the companies that build the foundational models, of course, they play a huge role. So if the average person likes the fact that the model mostly agrees, or sometimes you can ask for criticism, but say it's friendlier than your boss. Well, if I'm developing one of these models, I'm customer-oriented. I'm going to give you what you want. This, again, I think, is no different from if you work for a boss, and the boss likes to be flattered, people will flatter the boss in ways that he likes.
So I don't know that there's anything unusual or special about it, but it does make us think. Are there applications, are there ways where, say, if we don't really know how the models are—we don't really have a model for how to monetize all the investments that happen right now. And so it could be that it's advertising-financed, in which case I want you to spend endless time on the app. It could be, so OpenAI now seems to be going down the road to have it more linked to e-commerce, which I found completely surprising because e-commerce is a terrible business to be in with really horrible margins. But somehow, that seems to be their new best idea.
And depending on how exactly they will be monetized, we can then start thinking about is that mode of monetization. Does it somehow push us in a direction that is less than perfectly healthy or less than perfectly useful? To the extent that companies buy and implement the AI software, and they have mostly an interest in productivity, I think the LLMs have a healthy interest in producing things that are productive. I would probably worry much more about the B2C market.
B2B, I think it might not be perfect, but we're probably going to be OK. They might create switching costs that are higher than they really need to be because that makes the business more profitable. But I would be much more worried about what is happening in the B2C market, where individuals decide. And you see it a little bit with the bots that become romantic partners. If that's something, maybe I don't have the number of friends I would love to have, and I feel a little lonely. And if that's a good way to make me feel less lonely, companies will definitely cater to that.
And I think it's a super interesting question about all the ed-tech companies that are starting now using AI. Really, we want to experiment on the next generation of kids, what works and what doesn't work. I think there, I would have the biggest hesitation, actually, in the consumer market, not so much in the B2B market.
Chris Linnane: The kid part is very concerning and often used as guinea pigs and things like that. It's unsettling.
Felix Oberholzer-Gee: And we made this huge mistake with social media with really dramatic consequences. And now the question is, now we're getting around, I think France is now the second country that thinks about banning social media for all children. Australia already has a ban. But it's 10 years after we had solid, in a way, it's not totally different from smoking, where we look back, and we say, "Well, actually, these guys knew long before it became public knowledge that what they're doing is really terrible." And it's no different for Meta. Meta knew a long time ago all the negative consequences of social media, in particular for young girls. And, of course, they didn't act on it.
So I think that is a real debate to have: how to use AI, but also how to teach the next generation of kids to use it in a way that is really productive and that is really stimulating. It changes, it will change. So, I had this interesting conversation about, someone said, "Well, what's most important is that anytime anything is touched by AI, that you disclose it so that I know." And I asked, "Do you disclose when you use a spell checker?" And the answer is, "Of course, no." It's like, "Yeah, of course, we're using spell checkers." No one knows how to spell, and it's totally OK.
So, how exactly is AI different from the spell checker? Isn't it just that, it would be such a waste of time to teach anyone how to spell because it's just not necessary, and maybe AI will have a similar effect in that it shifts the boundaries of what you need to be able to do, but it increases the kinds of things you need to do. And it now includes responsible interaction with a really powerful technology.
And so I would love it if we would be really serious in thinking about it, so be careful, I think, one, because we don't really know the consequences, but then be really serious about, can we teach kids? Just like we can teach kids how to read newspapers carefully and skeptically about what they read, can we teach that same skepticism? Can we teach that same responsible handling of AI? I think we can. I don't always see, in the many applications we have for children already, we don't always see that, I don't always see that that's the guiding thought that people have.
Chris Linnane: If we switch back to productivity and AI, is there anything that, yeah, it doesn't sound great, but anything that's super exciting that you think about with AI and how it's going to help with productivity, and the opposite, things that make you think, "Oof," like to steer clear of that?
Felix Oberholzer-Gee: Yeah. I think drug discovery is an amazing opportunity. So I have a good colleague who works in drug discovery. His description is that basically, what you would do in your first year as a PhD student, that has completely gone away.
Chris Linnane: Wow.
Felix Oberholzer-Gee: And so that's super exciting. The part that I find most interesting about that is sometimes we get these really big productivity gains, and we think that's really amazing. And then you always want to think about, doesn't it just shift the bottleneck somewhere else? So right now, the bottleneck is that it takes us forever to discover new drugs. And that's really not great, and that part goes away.
But now, when we test the drugs, when we start with small samples to make sure it's safe for humans, and so on and so on, doesn't the bottleneck just shift from discovery to now, we definitely don't have a system that is based on super-fast testing on large groups of people? So I would always say these two things, think about where AI can really, really make a big, big difference.
And I think drug discovery is definitely one of these things. But then think about the system as a whole, and where are the, if all of a sudden, this thing blows up, is there going to be somewhere else in the system where we get new bottlenecks? And I think sometimes, it's probably going to be not really. That is just a much better system.
Chris Linnane: But just be aware that bottlenecks may be created elsewhere because of this.
Felix Oberholzer-Gee: May be created elsewhere, because we haven't really built a system, because forever, it used to take a very long time to discover drugs. And so the rest of the apparatus, the FDA is not set up for us to discover new drugs in rapid-fire succession.
Chris Linnane: Now, that's great. Do you want to use the bottleneck as the negative? You were talking about the positive-negative. Or you want to think of a different thing?
Felix Oberholzer-Gee: Not a negative, but to say if we want the overall productivity gains, just fixing one issue with the help of AI is probably not going to do it, because you'll just run into other problems somewhere else. Autonomous driving is a good example. So we built the software, and it's getting pretty good. I think eventually, we will get very close to autonomous driving. I was recently in San Francisco. It took me forever to get from A to B. It was so slow. Why? Well, because humans have figured out you can be really nasty to robotaxis.
Chris Linnane: Oh, wow.
Felix Oberholzer-Gee: Everybody cuts in. And when they cut in, of course, you're all, because the software was built on precaution. And we didn't really think, how is it going to interact? We were so focused on making it safe, knowing that any accident, even though, humans are terrible at driving to begin with. The bar is actually fairly low.
Chris Linnane: Sure, I agree.
Felix Oberholzer-Gee: But that, of course, is irrelevant. I think the public reaction to accidents, say, half the frequency of humans would already be completely unacceptable. I'm absolutely sure that would kill off the technology. So there was a good reason to be extra careful. But we haven't really thought about, OK, so now in the real world, in the interaction, if everybody is really nasty because they know you are the most careful driver in human history, that actually makes the technology then much less. So, we get a productivity gain because we don't have to have a driver, and then we get super slow movement because the bottleneck just shifts somewhere else.
Chris Linnane: Yeah, that's it. It moves somewhere else. So you basically had to carry everyone else's bags. You were like the guy who was picked on.
Felix Oberholzer-Gee: Yeah, I'm telling you, it was like, I mean, it was quite a bit of traffic anyway. But then also just, it was quite fascinating to me that maybe in San Francisco, the density is now good enough that people recognize it as a real force. There are many of them. And then if I just cut you off, I don't really have to worry because I know you're going to stop.
Chris Linnane: And you know the people planning that in the conference rooms years ago or whatever were thinking, "Well, this is great," but they weren't thinking about the fact that someone's going to look at another car and maybe be intimidated by the person behind the car, behind the steering wheel. And they're going to look at their thing and be like, "I can step all over this." That would never cross their mind at that point.
Felix Oberholzer-Gee: It's that story when you drive abroad. So there are different cultures depending on where you drive. And part of what you need to learn is how aggressive is everyone. And if you don't match your own level of aggressiveness to the aggressiveness of everyone else, basically, you're very likely to be in an accident because people misjudge you. But also, you're not going to get from A to B very fast.
So, the broader implications of very targeted increases in productivity, how they do or don't spill over into, "Oh my God, now we all can get from A to B much faster," because we don't have the nonsensical human thing, where someone on a freeway brakes because you're an irresponsible driver, you brake, everybody else has to brake, and then before you know it, you have a big traffic jam. Of course, in an only-AI future, that problem will have completely disappeared. But that'll be, I'm not going to be alive by the time that happens.