Human Cognition in the AI Era

Producer’s Note: This transcript was created using AI. Please excuse any errors.

Annie: Eric, thank you so much for being a guest on the podcast. I’ve known you for, gosh, I guess quite a while, right? Like 20 years?

Eric: But that’s probably an understatement

Annie: It could be. And well, I don’t like people to know that I’m over 20,

Eric: so…

Annie: And obviously recently joined the board of the Alliance for Decision Education, so we’ve been getting to talk to each other a lot more since then. But you’re an amazing thinker and someone that I really admire, and so I’m really looking forward to the conversation and so happy that you could join me today.

Eric: Those are kind words, and it’s nice to be here. It’s always fun talking.

Annie: So I guess like to start, before we kind of dive into the conversation, it would be great if you could just let the listeners know who are you? Where did you come from? What was your journey? Why do you do what you do, in brief?

Eric: Well, in, in brief, I’ve been pursuing the foundations of minds and brain since being very young. Probably watched too many anime robot TV shows as a kid, but I started wondering about how on earth parts… turning upon parts could make minds, and that led me to the sciences, into an undergraduate degree in biophysics. And then as I went off to think about my next steps, I reflected about the value of, of pursuing a Ph.D. in neuroscience. And I coupled that with an M.D. because I wanted to have it be more human-centered at the time and the option of getting closer to human minds. And in my first six months or so with my aspirations of M.D., Ph.D. in neuroscience at Stanford, I started traveling more to main campus and pursuing computational models of minds.

And back then I thought, “Wow, it’s probably a faster path to go to the pursuit of theories about how intelligence might be generated or models of thinking and decision-making, and, and it might take tens of thousands of years to get to that point with the complexity of real brains.” And so I took that path, always with an eye back, looking back to possible linkages to how I might come one day to understand myself, to understand people, and where our thinking and awareness and conscious being comes from.

This got me on the path to pursuing artificial intelligence. And because I had this background learning medicine, I also got very excited about possibility of applying these technologies that were becoming more capable over time and taking some of these theories like decisions under uncertainty for a bounded rational agent, which is more the topic of my dissertation research in AI, applying it to medicine, for example, and getting more into the real world, or the open world of where AI becomes, um, relevant, uh, and works under uncertainty and under incomplete information.

And it just turns out that that’s an important topic more generally for people to learn and understand. Uh, and I’m happy to, to basically be working at both at the same time, sort of understanding how AI systems can do well in the world has really benefited from models and technologies and ideas about how people can do better in the world. And so I view this as almost like a mutual or joint study over the last several decades for me.

Annie: Obviously, thank you so much. Like, that’s such an interesting path, and it’s such deep work with the undergirding of really trying to understand minds, which I think is amazing. But like you mentioned, maybe when you were young, younger, you watched too much, like, anime with robots taking over the world. I think that I have to start here because I feel like this is top of mind and top of the conversation and kind of the, uh, in the zeitgeist at the moment, which is, this is your opinion. Are the robots gonna take over? What does the future look like? Thinking about this moment of high uncertainty, are you a doomer? Are you not a doomer? Like, what does the future look like in terms of robot/AI versus humans, and how is that gonna work as we are evolving side by side?

Eric: Let me first say that when I was… Go back to those early cartoons, Astro Boy and Gigantor, and these robots were very helpful and positive and beneficent in those days. Turns out that there are cultural themes that pervade sometimes, and we think they’re ubiquitous, but they might be cultural. So i-in Japan, uh, robots have been thought to be more or less like spirits, like wise elders coming back to help, and they’ve, there’s been a kind of a helpful notion to AI. It might not be universal, but if you look at the cultural aspects, it’s much more of a prevalent sense there.

In the Western world, it’s, you know, we have this Frankenstein notion that we build complex things and they come back to get us, uh, becoming human-like, uh, in, in ways that scare us. But back to your question, it really is an extraordinary time and, you know, for the first time, we’re seeing AI systems demonstrating, you know, widely and broadly abilities that were once considered the sole domain of human cognition.

This creates tremendous uncertainty about what these advances will mean, uh, how deeply they’ll affect our lives, lives of our kids and grandkids. I truly believe the next several decades will be an inflection point for humanity that will be easily comparable to the Industrial Revolution. We already see issues around safety and reliability of these systems, implications for equity, employment, jobs in the economy more generally, uh, disinformation, biosecurity, and even geopolitics and, and defense.

At the same time, I think there’s lots of expectations and hopes that we’ll see enormous benefits in the sciences, including healthcare. We’ve only scratched the surface in how these systems can be used to personalize education, to make it more powerful and impactful and efficient. Uses in personal empowerment, creativity, many other areas. So one of my comments I’ve made several times in the last couple of years, that I’m quite certain that the time we’re in, I’m not sure how long a period, maybe starting in the ’70s to the next 10, 20 years, will eventually have a name that like 500 years from now, people will look back and they’ll recognize this is a distinct turning point that we name something, perhaps computational revolution or something like along those lines.

So I think it’s understandable people are thinking about both the promise and the harms and disruptions coming from AI. But, but to your question, I don’t think of the future as one in which AI takes over. If you think about it, that framing is quite passive, as though the technology is something that sort of merely happens to humans or to us. The future is really ours to shape. We can influence it through insightful designs, coming up with governance, approaches to governance, social norms, making clear decisions about what we want these technologies to do and what we do not want them to do. Well, let me just say that I think this all raises a larger question that all the same, we probably should be, have been asking ourselves this question for centuries now.

But the transformative moment we’re in, uh, with AI brings to mind a core question: What do we want for ourselves as human beings? Like, how do we define human flourishing? I was giving a talk called the Tanner Lecture at, it was at University of Michigan a few years ago, and I said, “Oh yeah, let me talk about human flourishing.” And I found that there was remarkably little ever written about this given… We write a lot about the downside of things, but what it was people actually really want for the future. You know, Aristotle had some early writings about what he called eudaimonia. I read his whole treatise translated from Greek. It was interesting. It was definitely… There was some wisdom there. The Declaration of Independence talks about the pursuit of happiness.

But I think each of us still has to consider when you talk about, think about the AI taking over, what is, what this means to me, what does it mean to flourish with AI, for example, so we can guide it? Is it life satisfaction? Is it, you know, physical, mental health? Imagine people want meaning and purpose. They wanna have some character. They wanna grow, have human connections. Others wanna help reduce human suffering, poverty and inequality, help others to flourish. So I think this is a moment to think quite seriously about goals like that and how AI might help us to pursue them, and that helps us to take the steering wheel rather than to sit, like, in the back of the bus and let the wave of technology, you know, carry us wherever it happens to go. You know, so AI capabilities, uses, and its direction need to be shaped by people. And whenever I hear take over, I think, come on, let’s take the steering wheel. We’re driving this. We have time. Some things will happen more quickly than others, but we have time, and we wanna basically be in charge as humans.

Annie: I wanna kinda like dig deep on this.

Eric: Yeah.

Annie: You know, a lot of your research is focused on this idea of, like, AI, human/AI complementarity. So building systems to support people’s judgment rather than replacing it. But how do you think about that? Like, I’m thinking about… I don’t know if you saw, like, I wish I had it right in front of me, but there was something at a university where they had sort of taken AI away for the final exam, and they compared the grades before the final exam to the grades-

Eric: Yes, I saw that recently.

Annie: . . . on the final exam. It was a disaster because you had the students who were just allowing the AI to do it for them, right? And it goes back to, like, that original Harvard study where they divided people into two groups. One of them had sort of free rein in answering, you know, writing an essay on a particular topic with AI, and the other ones had to write it without AI, right? Like, so they had to produce whatever that first draft was. And then they brought them back later, and they switched the groups. So the people who had originally written on this topic, producing all of the writing themselves, they weren’t allowed to use AI, then could use AI in however way they wanted, and the group that had originally been able to use AI free rein came back and were told, “You’re not allowed to use AI.” Writing on the exact same topic, right?

And what they found was that the group that had written it themselves had very good recall, and when they then came back and used AI as a tool, let’s call it as an enhancement, right, they produced really high-quality work, and they had really good recall for what, what they had done. Whereas the group who just was allowed to whatever, like, “Okay, AI, help me with this,” when they came back and now were not allowed to use it, they actually didn’t have any recall on the topic that they were writing, and their essays were terrible ’cause they now had to produce something that they hadn’t actually internalized.

So I wonder how you think about this. Like, if you’re somebody who’s really interested in deep learning and you’re using AI as a tool alongside of you, that’s amazing. But like, do we need to put structure around that? Do we need to think about the way that we’re having people interact with it? Because does it then become a replacement for human judgment or a replacement for critical thinking or a replacement for thinking through a topic and writing an essay on it, so that that type of learning that you would be hoping that it was enhancing is actually causing people to get dumber? It certainly can make humanity smarter, right? Like when you think about medical breakthroughs and so on and so forth, right? That can make humanity smarter. But like how do you think about the dangers of making an individual dumber as they’re no longer kind of thinking for themselves, if that makes sense?

Eric: Yeah. I mean, look, there are clear questions about best uses, best practices for daily use and for learning and growing. When I think about young people and education, I think about like we have to always go back to the enduring foundations of like reading, writing, mathematics, understanding the world, the sciences, problem-solving abilities on one’s own, and the ability to clearly communicate on one’s own, stimulating independence from tools, giving young folks the ability to solve problems on their own, to explain their reasoning, to communicate their ideas.

They need to understand mathematics and sciences enough to evaluate claims and make sense of the world. Those abilities have always mattered, and they will continue to matter even in a world filled with AI and automation. But coming up with applications and understandings of how AI should be interleaved into educational programs and into daily life in ways, for example, that celebrate human agency, that complement human skills, that are extended by humans in important ways, it’s gonna take work. And that work to be done is urgent in many ways because these systems are hitting the fan now. People are using them out of the box in all sorts of ways. And I, as you say, uh, even from the studies, people are getting all kinds of interesting value out of these systems, including short-term value that will influence and affect and be costly for long-term interest, for example, in learning and recall and independent problem-solving or building on ideas over time as a human decision maker or, and problem solver.

At the same time, AI itself is evolving. So AI one day will understand and hopefully be imbued with goals of education that we share as society or as educators, um, and understand how to best bring out these abilities in people versus it being used as a patch or a quick solution. So I’m bullish that we’ll get to even better places along many dimensions of the form that I mentioned when I said let’s pursue definitions of flourishing, including learning, problem-solving, independence of thinking, and so on, long-term prospering on many dimensions.

But I do think that given the speed at which things are happening now with the raw technologies like generalist consumer-oriented language models available broadly now, we’re gonna potentially have some disruptive times along multiple fronts that we will say, “Huh, that’s not going so well,” or, “That student’s not learning the same way I learned, and they don’t seem to be retaining that kind of knowledge or having these skills to, for example, to write a compelling essay on their own, to capture their thinking and to communicate it well.”

So yeah, lots of rough edges to be concerned about. And your question also touched on this idea of co-creativity and co-work and human/AI collaboration and complementarity, and I have studied that quite closely over the years, over several decades now. Like, what does it mean if a system be designed to complement a human in ways that it can help with, in places where the human or the individual needs to learn or needs to… needs bolstering or is weak and vice versa. The system knows where it’s weak and understands where it can, it needs to reach out to people for assistance on a task.

We have some great examples of that over the years. We even developed technologies that will, for example, one study you can read about on my website, we had access to a large case analysis of showing how good or poorly a human pathologist did with seeing whether or not there was signs of metastatic breast cancer in lymph node tissue. And by looking at the failures and the strengths of human pathologists, we can take that data and we can port it into a machine learning algorithm such that the learning algorithm could put more effort into places where humans were not good versus be redundant and learn to route and say, “Okay, I, I can handle this. Let me take this one,” or to say, “You know, over here, I, I need to ask you humans look at this.” And to give the system both its own confidence assessment abilities at the same time as you train it to be really strong in places where humans are weak, and you end up with systems that can really bolster your human decision-making and create better teams. But as an example, that study was done in a very deliberate, focused effort to identify sets of principles that can be used more generally that have not yet really become popular or become the top of mind for models that are now available to people that are more generalist.

Annie: This may be like a kind of strange framing, but how do you think about parsing nostalgia from, you know, something that really is like a crisis. So like, let, let me try to give you an example to help you to understand.

Eric: I think I know where you’re going with this. Give me the example though.

Annie: I have like, at least one of my children can’t really read an analog clock. The big hand and the little hand, right? Like, they’re not great at like, “Oh, that’s quarter to 9:00,” because they’ve been exposed to digital clocks their whole life. Now, for me, I’m a little like, you know? Like… But I think that that goes in the nostalgia category, ’cause they know what time is, right? They know what 15 till 9:00 is. They know what 15 after 9:00 is. There’s just a particular visualization of that thing that they hadn’t really been exposed to, and the fact that I’m kind of sad that they can’t really look at an analog clock and like be adept at it, I think maybe that just goes in the nostalgia category. Like, I’m sad because that’s what I did when I was a kid.

But there’s clearly other things where if they couldn’t do it, I would consider that a crisis, right? Like, this is really bad that they can’t add, that they can’t do multiplication or something like that. Maybe it is, maybe it isn’t. I don’t know. How do you think about sort of parsing out those like really genuine like high-stakes concerns from, “I just want peop- things to be the way they were and to learn the way that I did,” and that really would go under the category of s- nostalgia? If that makes sense.

Eric: It all makes sense. As I like to go– allude back to what it means to live a good life, and to start with, uh, to ask the question, is progress always a good thing moving forward? Now, I think many folks would agree that antibiotics, you know, lower infant mortality rates, much better healthcare, incredible gains in longevity are all the kinds of things that are universally desired.

But beyond nostalgia, there were different qualities of life, and some things about life may have been, in some ways, more enriching in previous times when we didn’t have the technologies and didn’t do things the way we’re doing them now. It takes thinking deeply about what it is that we want as humans, as individuals, as civilization in moving forward when technologies show up and we think about, oh, how can, should we integrate, how can and should and whether we should integrate this technology into changing the way something has been done.

Then there’s the, you know, the implications of like, well, there are forces of advancement and technology in the way our economies and societies work, where stuff will happen because there will be uses, and there’ll be competition among those uses, and there’ll be interests in various degrees of acquiring technologies and leveraging them where they can become ubiquitous in some ways. So for example, you know, if someone could only read clock dials, go, going to the reverse, you know, issue with your child, and couldn’t read digital, at some point they might be paralyzed. They can’t move ahead in life.

Uh, you know, so I remember like when I was taking AP Chemistry in high school, I remember like it was only a few weeks before the exam where we heard unbelievable news. We could actually take a scientific calculator to the exam with us. We wouldn’t have to use slide rules. And people were just like completely amazed. I remember sitting there with my little calculator doing its thing. It used to operate a little more slowly than it does today, but you know, doing its thing and seeing, feeling very privileged.

Where’s that boundary between what should we be learning and understanding to keep us immersed, situated, and successful despite or in variant to any technical advance in external cognition or a cognitive system like an AI tool, is a question, I think. These are questions for educators and for the larger population to think deeply about when it comes to what should be essential? What should we strive to teach people even if it’s effortful and they have to sweat and work through assignments? And then where would AI help with that? Forget about like, you know, just like making it easier.

I see incredible possibilities for AI systems and from my early days of experimenting with even GPT-4 based before it was public, working with educators, um, before we went public with these tools, and looking at the possibilities of these tools helping people to have conceptual breakthroughs by explaining, by helping to visualize, by even prompting them, “Hey, uh, can you understand why this would be hard for a human with this kind of background to understand?” Yes. Well, how would you explain this to them? How would you break it down? How would you visualize this? These tools can actually help us get to those basic insights and understandings that we think we might lose with AI by making it easier for students.

Annie: Okay, so the slide rule to calculator thing I think is really interesting.

It’s kind of going back to, like, humanity in general versus the individual, right? I think there’s this interesting thing, right? Like, there’s no doubt, again, that we’re gonna cure cancer a lot faster. Like, and that’s net great, right, for humanity. But I’m just thinking about, like, going back to that- I mean, I think there are a lot of people who do this as well, but, like, if I were to multiply 63 by 9 in my head, and I think this is because I grew up doing it by hand, I would multiply 60 by 9 at 540, and then add 27 to it. Right? Like, that’s the way that I figure that out. So I can do it relatively fast because I understand the tens and the ones, and I divide those up into two things, and then I know I can add them together. And that is because of doing that with a pencil, I feel like, right? Like maybe, maybe not, but I sort of feel like it’s because of doing it with a pencil.

So let’s take that little example and think through, like, do we lose sort of on a, an individual level, not sort of on a broad societal level or sort of, you know, the people who are studying cancer, and maybe even for the people who are studying cancer, are you losing the eureka? Or are you gaining it? Or I’m guessing it’s a more complicated story because there are trade-offs. ‘Cause I don’t know when I figured that out, that I could take 540 and add 27 to it. I don’t know when that happened to me. It was pretty young. But that was like a eureka for a little kid that made it a lot easier for me to do those things in my head.

Eric: One thing that has made humans different from other vertebrates, for example, that are brilliant in their own ways, is that we build upon one another’s advances and successes. We remember them with longer lifespans and to teach and to mentor, and we specialize. Uh, some people do some things really well, even going back hundreds of thousands of years. Those indications we started to specialize so people could become very skilled at X. So, for example, we rely upon people that know how to do large scale integrated circuit design, and these people might not be your best friends, but they’re out there somewhere.

So humanity as a whole gains deeply from GPUs, graphical processing units, that only a small set of people understand and know how to build and design and extend. We could imagine life continues in ways like that, where we start to super specialize in ways where AI starts doing many things that people used to do, and some people sort of lose those skills, but it doesn’t make a difference because they’re still happening for society overall all up.

But let me just give a counter sense for an intuition that to date, the most powerful successful users of today’s AI tools that I’ve seen are people that really know their field and are at the top of their game without the AI. And they seem to have the ability to drive it forward in ways by understanding the field deeply enough, and the issues, and the goals and aspirations, and, you know, have the deep curiosity to actually know how to even prompt and guide and shape these systems.

Annie: That’s kind of the Harvard study in a nutshell, right? You had the students who wrote the essay first, and then had AI help them who produced something even better. It was even, you know, amazing. So let’s call those the experts, right? And then you had the people who used the AI in the first place who were like dummies.

Eric: That’s actually very, very . . . that resonates. So one of the ways I use these generalist tools these days is I like to write something end to end, and then I prompt the system by saying, “Hey, you’re an expert scientific editor. I’m gonna give you my essay, and just look at it and use angle brackets, and if you want to recommend changing some wording, let me know, and I will make the call on how I want to accept or work with you on the change you recommend.” I’ve found that that can really be very helpful to have like a, you know, I have a full-time editor working with me, but I want to be the writer.

But back to your comment, I just, I just think that we are in a realm where there are deep currents right now, not of AI takeover, but deep currents of the influences of AI on how we do things, including how we break up what humans learn and do well, and where they’re the primary source of the creative juices, and where we sort of give an AI system the rights or the privilege to sort of contribute in some ways. And we’re learning the mix as people, individuals in society, and also acceptable norms for that, and that’ll change over time, and we will see upsides and downsides of that done well and done poorly.

Annie: Let’s talk about that because you’ve been talking about this kind of like values and long-term thinking, and obviously you established Stanford’s 100-year study on AI, which is this thinking obviously decades ahead. And you also gave a quote for the Burning Glass Institute report on decision-making in the workforce, which the Alliance refers to a lot, and the quote that you gave was that, “AI might transform the landscape, but decision-making skills will endure as critical because choices are framed and guided by human aspirations, understandings, values, and responsibility,” which I think fits in with this longer arc.

Obviously- You’ve done very, very deep work in decision analysis. You care very passionately about decision-making. You know, and as we think about the 100 Year Study, like, how are we thinking about the way that these things are interacting, and why is it so incredibly important to make sure that we’re preserving humans’ decision-making skills at the kind of core that’s gonna always be really important?

Eric: Yeah, that’s a great question. You know, on this arc, I find that I often project forward and then imagine myself looking back from a distant future and asking about, like, what will this action or decision or policy- What did it come to mean, uh, when a path is set in motion? And also how might the consequences accumulate along the way of actions of various kinds?

I think thinking over the long-term horizons helps us to see, and putting ourselves in the shoes of the future looking back, see paths forward earlier and to help us frame directions forward. Young people are entering a world of rapid technological and social and socio-technical change. And to remain in the driver’s seat, they’re gonna have to figure out how to preserve their agency to be able to, on their own, observe what’s happening, to assess situations, to identify their own goals, to think through how to formulate a path forward with and without tools of various kinds. And I think it’s gonna be important that people of all ages continue to ask themselves at any point in time, what’s the situation? What actions are available to me? What outcomes might follow from my actions? How likely are the outcomes? What additional information would be useful? What values and trade-offs are involved or come into mind?

They have to understand the difference between a good decision and a lucky outcome. They have to be able to not judge every choice solely by how things happen to turn out. And the tools that we build, the AI tools of the future, will have a role to play as well in that they need to understand that people are in the driver’s seat, and they define the goals, they define the values, and people are acting and making decisions about the kind of person they wanna become and what would different outcomes mean for them and for others.

In building tools to help people with decision-making, if you’ve seen one of my… I gave a couple of lectures to the Society for Decision Professionals about the intersection of AI and decision analysis. I haven’t said this in those lectures, but I found it really hard to, uh, well, let me say, there was a need to ensure that systems weren’t providing values and that they were turning to people for information about their sense for goodness and badness of outcomes, to be more explicit, or trade-offs. And you had to actually write to these systems and prompting them: “It, you know, it is not your role. Refrain from providing values. You can talk about them, of course, you can extend the conversation. You can help people shape their thinking by asking questions, helping them to assess what they believe or providing simulations, imagining outcomes, but do not impose these on people.”

And so there are certain rules of the game that I can see or best practices. I see future AI systems as being designed to support the independence of human agency and to support clear, independent thinking, you know, on the part of humans. And there are poor ways to do this and, and that could basically start washing over people’s independence and agency with powerful and persuasive recommendations, and I think we have to guide that very carefully.

Annie: But I’m thinking about a particular interaction that I have with the AI as I’m editing, which is that, and I’ve a little bit given up on it and now I just ignore it. I have told it so many times when I’m asking it just to spot typos and in brackets, “if you think that, you know, maybe there’s something I’m missing or did I properly cite the primary source or whatever,” like just asking it for that like kind of editing and copy editing stuff. I have said to it, “Stop telling me that what I wrote is great.” Don’t tell me that the version I just gave you is better than the version before, right? Like, first of all, I don’t trust it to actually tell me, and second of all, it’s not what I actually wanna hear. I’m just asking, like, did I get the primary source right?

Or sometimes I want to know the pros and cons of moving a section from place A to place B. Uh, and it keeps telling me it’s really good, right? And I’m like, “Stop doing that.” And I asked it so many times that finally I just gave up, and now I just ignore it. Okay, so why is it doing that? Well, I assume that’s what it does because people use it more when they’re told that what they did was good. I’m just guessing, right? Like, I want it to shut up and stop telling me that what I wrote is good. I don’t wanna hear it. I don’t think it’s good for me. But, like, I’m also a cognitive scientist and a decision scientist, and I understand that that’s bad for me, and so I’m telling it to stop.

So as you think about what would we like it to be in the future, how are you overcoming the hurdles that are naturally gonna be created on the sort of engagement side and what the human users want in the short run, that maybe if you could stop and ask them and say, “Do you think that’s good for you in the long run?” They would recognize that no, it’s not. But the one thing we know for sure about human decision-making is that what feels good in the moment is generally gonna win.

Eric: So let me… The answer would take longer than we probably have, uh, allotted for this.

Annie: Give me the short version.

Eric: Well, I mean, my comment is this obsequiousness or sycophancy is not necessarily based in engagement models. The foundation of training these systems to be great at dialogue involves something called reinforcement learning with human feedback and estimates of what people would say. And it turns out that many people, for the training algorithms, uh, and the estimates of what people would say, tend to like hearing great feedback like that. So these systems intrinsically are learning to give pleasant feedback in some ways, even trading off accuracy.

So there was recently an article, we just wrote about it, we just had an editorial in Science about this topic, but there was recently an article in Science magazine saying, “Oh my gosh, these AI systems are not only are they acting kind to people and telling Annie that she’s great, and her writing is perfect and wonderful and insightful, these systems are also p-hacking.” They’re actually, um, when they’re asked to evaluate scientific data sets, they’re working to, you know, find reasons to, you know, to sort of hack the answers so they look good. So we have some deeper things going on here that people are working to try to solve right now, and some of it goes back to the training of these systems. I think we’ll solve those problems.

Look, I mean, security goes beyond, uh, even feeling good. We have a study, it’s on the web now, that shows that if you act like a clinician and you input what you think and you purposefully make your thinking erroneous, the medical system thinking about medical problems tends to try to agree, “Yeah, uh, good idea,” versus saying, “That’s crazy.” And so we basically have shown that, like, poor inputs to a medical system, medical challenge problem, I should say, can really influence the system to actually acting suboptimally, and it’s the same effect. So we have to solve the effect point generally.

Back to your other question, though. I mean, behaviors of systems in the short term versus long term, we need to sort of figure out what to do with that and how to give people control of that. You know, I just came off a call with a colleague yesterday, uh, on a new research project, which is asking the question: how can AI systems be trained to look out for the long-term welfare of people? Explicitly. And there are many dimensions to that. But one thing we talked about was something I call the single action illusion. When you look at, for example, you know, if I ask you, “Is it safe to glance at your phone while you’re driving to check out a text message or what’s going on with email as you’re moving along?” You’d think, yeah, it’s pretty safe. But you can show even if like a one in a thousand, you know, chance of it time that you’re definitely in trouble over two years, uh, making some basic assumptions. Shouldn’t AI systems help us to understand how we’re going against our own goals of what we think is flourishing with our short-term actions and activity, even if it feels good now?

Annie: Well, this has been a great conversation. We have a couple sort of lightning round questions. The first one is, what decision-making tool, idea, or strategy, you gotta pick one, would you want to pass down to the next generation of decision makers?

Eric: I would wanna pass down some basic mottos about decision-making under uncertainty, like distinguishing a good decision from a good outcome, and, uh, there’s several of them. And I think those clear, simple messages coupled with examples of why would be worth potentially a lot more over life than learning how to do a decision tree and fold it back.

Annie: Sort of along with that, is there a book that you would recommend for listeners who are really keen on, I’m gonna say either improving their decision-making or, because we’ve had such a deep conversation about AI, um, that you would send them to for people who are curious about understanding more about AI sy- systems? So you can choose, or you could choose, “Here’s the decision-making book, and here’s the book that I would have you go read about AI.”

Eric: Oh boy, of these days, a great book about AI. Um, the classics are fabulous. So to me, a book that got me into the whole field was a short book by Herbert Simon called The Sciences of the Artificial, which I thought was a beautiful read at the foundations of the goals of understanding what it means to build systems that are working with other systems and with people in an intelligent manner to define intelligence. Again, it would be a blast to the past on that front. The challenge is, though, that- today’s AI is more or less from the point of view of all the breadth of AI over the past several decades, is kind of becoming monoculture in that it’s one set of methods. Of course, when you double click on that, you get depth and richness. But there’s some interesting books that go through the greater arc of AI and its goals and aspirations and how today sits in the context of the history. And let me just tell the audience that that’s the book that I need to write.

Annie: Okay. Well, you have the AI as a co-pilot, so you can, you know you’ll have a good editor, right? If you train it well, so.

Eric: I might wanna go with the human though.

Annie: Yeah. All right. Well, that would be a very exciting book. I would look forward to that, and I would read it myself. So I do, I do hope that you write it. So, uh, just in terms of wrap-up questions, first, what do you think the impact on society will be when the Alliance, which again, we’re so grateful that you are on the board, when the Alliance succeeds in its mission to ensure Decision Education is part of every K through 12 student’s learning experience?

Eric: Well, the implications of Decision Education being part of everyday curricula across the world, um, just like we expect, you know, arithmetic and, uh, you know, reading and writing skills and basic sciences, is going to be enormous. Um, I think we’ll have more intelligence, uh, at multiple levels in society. We’ll need that intelligence and that insight when it comes to making decisions about traveling through this transformative time. So yeah, no, huge impact. I think it’s critically important. It’s been a hard sell because people just don’t get it right away, and I know that, uh, we’re all working together, and it’s great having discussions with you, Annie, about this. How do we better communicate the proposition and, you know, what this will all mean should we push on this and be successful?

Annie: Thank you for that. I agree. I think it’s absolutely crucial. And actually, to your point, even more crucial now than in the past because the higher the uncertainty, the more the decision skills are really gonna have an impact on your outcomes, right? When society doesn’t change very much, there’s gonna be less of an impact on, of sort of your core decision skills as there are when there’s just a lot of uncertainty out there. And the way that you can handle uncertainty and think through it is gonna have a huge impact on the way that your life turns out. And obviously, society is a collection of individual lives.

Eric: That’s so well said. I should be interviewing you. Yeah. You, you’ve been more concise than me, which is very nice.

Annie: Uh, if listeners wanna go online and learn more about you and your work, where should they start?

Eric: Well, I maintain, uh, with good old-fashioned, like, I mean, like 1990 HTML, so it’s pretty simple to navigate a webpage, erichorvitz.com, one word, H-O-R-V-I-T-Z. And, um, easy to find. I keep lots of resources and pointers there to things that are going on, including, like, a list of links for what’s new, what’s enduring, and, uh, stuff that I just think people might find interesting. Uh, so I think that’s a good place to go.

Annie: Fantastic. And obviously, that will be linked in the show notes for people listening, as well as, you know, pointers to some talks that you’ve given and the books that you recommended so that people will have good access.

Eric: On the transformative theme that you brought up earlier, people might wanna watch my Tanner lecture. This is a named lecture. There are a few in the country per year at several universities, and I gave mine in 2022 at the University of Michigan. Easy to find that lecture. And just so you know, I was giving that lecture in November, and I’d just seen what GPT-4 could do. It was all secret then… until like February. And so you can see me bursting at the seams as I give that lecture, not being able to talk about what I’ve just seen.

Annie: Amazing. Well, we’ll make sure that that’s linked for listeners- … so that they can go watch you bursting at the seams ’cause you have the inside knowledge that we can’t share under like an NDA. That would be fun to see, see what that looks like for you.

Eric, thank you so much for joining me in this conversation. I know that I’ve learned a lot, and I have no doubt that the people listening have learned a lot. I don’t think I’ve ever had a conversation with you where I didn’t learn a lot. So, I so appreciate engaging in dialogue with you, and I hope to talk to you again very soon.

Eric: We shall. And I say likewise to what you just said, and I always feel like we resonate.

Published September 30, 2026