Trade-Offs and Truth-Seeking

Producer’s Note: Since the recording of this conversion, Andrew Berry has joined the Board of the Alliance for Decision Education.

Annie: I’m excited to welcome my guest today, Andrew Berry. Andrew is the General Manager and Executive Vice President of Football Operations for the Cleveland Browns. His appointment at the age of 32 made Andrew the youngest person to hold the title of general manager in NFL history. And in his first year as GM, the Browns made it to the playoffs for the first time in 18 years.

Andrew studied economics and computer science at Harvard, and in 2009 he turned down a job offer with Goldman Sachs to take an entry-level position for the Indianapolis Colts. Over his career to date, he has worked with the Colts, the Cleveland Browns and the Philadelphia Eagles, finally returning back to the Browns in 2020 as GM. In June 2020, Berry started the Be The Solution campaign to emphasize education and encourage staff, fans, players, and the media to implement action for ending social and racial injustice practices.

Hi Andrew! I’m so excited to have you here.

Andrew: I’m excited to be here, Annie. I appreciate you for having me.

Annie: Well, we’ve known each other now for two years. So, I got to know a little bit about you in Philadelphia, and now you’re back at the Browns. Boo-hoo for Philadelphia. But obviously we have a little bit of history, but what I would love for people who don’t have our history is for you just to start off by telling us a little bit about what you do and how you got to this path of at the age of 32 becoming GM of an NFL team, which is a harder thing to get than Senator of the United States.

Andrew: Well, I don’t know about that. And I definitely got a little bit lucky along the way. I guess I would be remiss if I didn’t start to say that I think anybody who gets into one of these positions—and particularly in sports—if you asked 10 people, there are probably 10 different stories in terms of their path. And, for me, I consider myself very blessed and very fortunate. So I grew up in a household where education and football were at the forefront. You know, both my parents were the first people in their families to graduate college.

My dad was a huge die-hard Cowboys fan, and raised me and my brother to be Cowboy fans. My mom was a chemical engineer in college and had played hoops, but grew up as a Falcons fan. So sports and education were really big in our household. So I grew up playing  all sorts of sports, and football was of course my favorite. I went to college at Harvard and was a corner on the football team.

Like all other college athletes probably, I had hoped to have a shot to play in the NFL. I was a marginal NFL prospect. I made it to Rookie Minicamp coming out, but didn’t make it that far  past that. And, at the time after I was graduating, I had interned and then accepted a job to do sales and trading for Goldman Sachs.

Well, I wasn’t super excited about going there, if I’m being honest, because it wasn’t necessarily my passion. And my first boss in the NFL was a gentleman named Tom Telesco. He was a scout and director of the Colts at the time. He’s currently the General Manager for the LA Chargers. He had been on a scouting trip at Harvard and became familiar with my background story, and reached out and said, “Hey, have you ever thought about working in football or an NFL front office, because we think your background would be a nice fit.” And I really hadn’t. But he said, “Hey, we have this entry level position open as a scouting assistant. We’d like you to come interview for it.” So I really went to the interview more as a courtesy because I had accepted this job in New York. I didn’t know much about the industry, and in preparation for the interview, I was like, “You know what? This actually feels like what I’m searching for, like something that I would really enjoy doing professionally.”

I was fortunate enough to go through the interview process and receive an offer. And then, at that point, meeting everybody in Indianapolis, I just did an about-face. I was just like, “You know what? I think this is more ‘me’ than what I had intended to do, a few weeks earlier.” And so from there, I was very lucky that I started my career under one of two general managers who were in the Hall of Fame, Bill Polian. I spent my first seven years in Indianapolis and really did everything at junior levels within football operations: pro scouting, college scouting, operations, and learning contract management. And ultimately, before I left, I was overseeing the pro department and the number two in contract management, helping us get our first research and strategy group off the ground. From there, I was fortunate to go to Cleveland under Sashi Brown—who was the General Manager at the time—was running an open search for a Personnel Director because he had been newly named the General Manager. Sashi had a legal background, had been their general counsel, had run contract management, you know, very, very smart guy, and was taking a very progressive approach to team building.

I didn’t know Sashi the first time, I was fortunate. Most general managers, when they gain those positions, they hire people that they know. Sashi took a gamble on a 28 year old Personnel Director. And it was one of the best environments for me because of everything that I learned. I spent three years there, then left for a year and worked under Howie Roseman, who I think is one of the best General Managers in our sport.

[Howie is] another guy who had a bit of a non-traditional background in the industry. And then when Cleveland turned over their management group—I guess a little bit over a year ago—I was fortunate to come back and interview with the ownership and receive the offer. And it’s been a really nice 14 or so months to start.

Annie: Yeah! Oh gosh. Okay, so there’s so many things that are coming to mind for me right now. So I don’t even know where to start, but let’s just start with something from my perspective, thinking about my life. I had the benefit of accidentally finding a particular area in which the things that I had been studying could express themselves in a very unusual way. So, I was studying cognitive science, which also includes understanding data, statistics and probability theory. And then, I accidentally ended up in poker, and I’m hearing that in your thinking. And I think for me, the collision of those two backgrounds really ended up informing the way that I think about the world.

So I know for you, obviously, you were thinking [about] why you accepted a job at Goldman Sachs. And when you were at Harvard, you studied computer science, which is a very particular analytical way of thinking. And then when I hear you saying, okay, so you land with the scouts. The thing that I always hear about in organizations is that there’s kind of turf fights.

If I think about West Side Story with the Jets versus the Sharks. And one of those turf fights, I feel, is between the analytics department and the scouts, and that there’s this kind of fight that happens. So, I’d love to hear your thinking about having come from a world that was more in the analytics side—and in fact that seemed to be the path that you were going to end up going into Goldman Sachs—but you now end up in the scouting department. So how did you think about the way that that merged? What was the scouting department at the Colts like? Was there this turf fight, and how does that inform your thinking as you go forward?

Andrew: Yeah, it’s a good and interesting question, Annie, because I think you’re capturing—and not just in sports, but across many organizations—where there is maybe a little bit of this natural tension between some more subjective measures and some more data-driven measures.
For me—you’ll probably hear me say this a thousand times in this podcast—I was lucky with my background and the time in our industry when I came in. I mean that for a couple of reasons. So I got my master’s in computer science and my bachelor’s in economics at Harvard, but I was also at the time a I-AA football player who had gotten far enough to be in Rookie Minicamp for an NFL team.

So for me, when I came into an organization to really cut my teeth in scouting, I didn’t necessarily have the same stigmas as some of my peers around the league that had similar academic backgrounds. It was like, okay, well he’s, for lack of a better term, a “football guy,” because not only did he just play at this, even though it’s not a Power Five conference, well he actually was a pretty good player, he was actually in our scouting database. So, it gave me … right, wrong, or indifferent—and probably more wrong than anything—I didn’t have to tear down some of those stereotypes initially, when I got in.

The second piece of it is, I was getting into the NFL at the time, although data has been a huge part of baseball and basketball decision-making for a number of years now,  football is still relatively new from the perspective of using some of the more advanced methodologies to incorporate into their decision-making. Right? You’ll see predictive analysis and regression analysis in baseball and basketball, but really it’s only taken off in the past couple of years, at least in a widespread manner in the [football] industry. So, when I got in, the path was, “Hey, if you want to run a team one day, you really have to know personnel.” Scouting really was the place to start. And it probably wasn’t until year five or six in Indianapolis when we really dipped our toe in the water in terms of building out research and strategy groups.

So I didn’t necessarily have the same uphill battle, probably at least until later in my career. Because, you know, really then the narrative shifted after my first stop in Cleveland and I was further removed from my playing career. But at least initially, it probably helped that I was, “one of the guys,” or “one of the scouts,” coming up, that I never really had to tear apart any of those stigmas to get a point across or propose a different way of looking at things.

Annie: So when you first went to the Colts, did they have an analytics department?

Andrew: They did not. And I would say most teams in the NFL did not. Which is crazy to say, because that was like 10 or 11 years ago. It’s not like this was 30 years ago. But really, it was considered [to be] a thing that works in baseball and basketball. That was not a football thing, at this point in time. So there wasn’t really a heavy presence.

Annie: So, you know, one of the things that I think about in decision-making is you have “what’s true,” which is essentially: what are the things that are the facts on the ground? And then you have subjective judgments that you make on top of that. And I’d love to hear your thoughts on how people then, and today, are thinking about the interaction between those two things.

Because one of the things that I find is that when you get into a world where things clearly are more subjective, and this is true in any type of decision-making, some decision-making is going to be just, if you know the data, you’ve got the answer. And then you have other decision-making—like hiring decisions, for example—where you can have some data that’s informing it, but obviously the answer is going to be quite subjective in the end.

I find that what people end up doing is they sort of take sides, and they’re either: “the data’s giving me the truth and stop with your judgment,” or “my gut is telling me what to do, and stop with your data!”

So I’d love to hear your thoughts, because you’ve been there during the evolution. First of all, what, what was that like in the beginning? And where is the NFL today on that issue?

Andrew: Yeah. So early on, it was very much like your West Side Story. I mean, just two competing camps, basically shouting at one another. That’s how I would describe it more industry-wide, and largely because I think: 1) the data perspective was different, 2) probably how some of that information was communicated to people who maybe haven’t had as much experience with it was a challenge, and 3) because in football, relative to hoops or baseball, there’s a lot more gray in terms of what are hardcore facts, so there was a fair amount of football wisdom that maybe have been too casually dismissed early on, and it led to a natural tension between what I’ll call the two competing camps.

And then I think in some places, it got to a point where it’s like, “if we’re going to make a decision, what percentage should we rely on ‘subjective,’ and what percentage should we rely on ‘data’?” And I think some organizations still operate in that mindset, in that manner, when oftentimes that’s not the right question because, number one, it’s not like one side’s always right [and] one side’s always wrong. And just to even think of them as “sides”, I think is the wrong mentality. And then the second piece is, it really depends on the question that you’re asking, how you’re going to weigh different inputs that go into decision-making.

I think the league, at this point, has evolved into having a greater understanding in terms of how to use all pieces of data. But I think there still are gaps for a couple of reasons. I think the first area where the NFL has seen growth—relative to the other sports—is just on the analysis side, I think we’ve got to the point where you have the quality of analysis, whether it’s in terms of methodologies or people, where you have those experts in the sport who add real value.

I think the two areas where the league as a whole is still either catching up or, you know, some teams may be a little bit further ahead than others. Number one, is information quality, because the analysis is only as good as the information that you’re able to gather. And I would argue that probably baseball is furthest along relative to all the major sports.

And it’s maybe a little bit easier [for baseball] because it’s really a game of discrete events, whereas [with] hoops and the NFL, it’s not quite the case. You know, hoops is able to discretize their game because of player tracking data. And, you know, you only have 10 players on the court. Football is still making inroads there just because there’s a lot more complexity, but the teams are “advancing the ball,” so to speak, in that area.

Annie: Just to be clear for people who maybe don’t know the difference, if you think about baseball, it’s basically nine independent people on the field.

Andrew: Right.

Annie: You know, one person is hitting the ball. Another person is pitching. So there’s not as much interaction between the players. So that’s where each play is very discreet. There’s nothing … it’s not as continuous so that if you’re thinking about a particular player you can isolate them so much better from the rest of the team, in terms of understanding how you’re supposed to think about that player, in a way that it’s very hard to [do] in football with 22 people on the field, all interacting with each other. Right?

---

“You can have the best data, the best analysts, but if it doesn’t inform decision-making,
it doesn’t amount to anything.” — Andrew Berry

---

Andrew: Exactly. So I think that’s a frontier that teams are making a lot of progress on. We’re not all the way there yet, but a lot of progress on … But then I think actually the major hurdle, and I think this is largely, call it, “alignment relationship buy-in driven” is from a decision-making standpoint.

So let’s assume we’re in a world where we have ideal data, we have world renowned analysts, and we’re able to combine all the different pieces of information: scouting report, medical information, objective data, performance data, you name it. And that information and analysis gets to any of the decision makers in the respective places, whether it’s a coach, a general manager, you name it, and it doesn’t actually inform the decision-making of the organization, then it doesn’t amount to anything. You can have the best data, the best analysts, but if it doesn’t inform decision-making, it doesn’t amount to anything. And so I think those two areas: the actual data quality, and informing decisions, that’s probably the areas that have grown a lot over the past decade. But there’s probably still some room to grow within the sport.

Annie: And do you find big differences among teams? I mean, it feels like, from a fan standpoint, there’s a lot of pressure on teams to now have really robust analytics departments. But, to your point, I’m thinking back to you saying one of the advantages you had coming into the scouting department, despite the fact that you’re secretly an egghead.

Andrew: That’s fair. That’s fair, Annie.

Annie: I mean, I am too, so consider it a compliment. You had football cred, right? They were like, “Oh, he’s a football guy.”

Right. Obviously the NFL is still rife with, you know, the old school who are like, “Stop with your analytics.” So, the thing that you always hear is [when] scouts are like, “He looks like a football player.” Right? “He looks like the position.” They feel like they can sort of see that. So while there’s been a lot of pressure from fans to make sure that every team has to have a robust analytics department. Really, what do you see as the tension? Is there still tension between analytics and scouting, but then also between different styles of coaches, who are more or less open to that. And what are you seeing around the league?

Andrew: I don’t think that tension is quite as pronounced as maybe it’s perceived to be externally. I think largely because at the more junior levels of your organization or your scouting departments, your coaching staff, those are usually people who are a little bit younger.

I think you really are seeing a wave of people that are coming into the sport that—while they bought an individual may not necessarily have been a “quant” at any point in his or her life—there is an appreciation for data, because now people are consuming the sport or growing up in the sport with more data, and more good, publicly available data than I certainly did, whether I was in high school, or college, or even early in my career.

And so I think there’s a greater appreciation for using that to make decisions, or evaluate players, or something along those lines. I think the level of tension is probably different from organization to organization. I think a big part of it is maybe how you incorporate the different disciplines. Ideally, from my perspective, you want to integrate them from the ground up, [so] that they’re partnering together and there’s perspective and wisdom shared on both sides. Because really the goal is the same, right? Whether you’re building a draft board or free agency board, whether you’re trying to build a set of rules for in-game decision-making, that’s going to take input from all the respective areas in your football operation. Rather than saying, “Hey, what do you recommend?” [and] “What do you recommend?” and trying to merge it at the top, I think it makes sense to really grow it together. But not everybody necessarily will take that approach.

So, in areas perhaps where it is a little bit more siloed, I think that maybe [the] tension that I certainly observed seven or eight years ago probably still exists, but I think a lot of operations are probably a little further along than people give it credit for. I do think that there may be a little bit of a gap still, as you go from the more junior or mid levels of the organization to  moving some of that content up to the decision-making realm. Part of it is because, generally speaking, the more senior people in the organization have been in the business longer and may not have quite as strong of a feel for some of the new information or integration of data with maybe traditional methods. But, that being said, I do think the industry is evolving.

Annie: So, it’s interesting, when you think about, across your career, you’re seeing the evolution of the integration of thinking about what are best practices in terms of decision-making, how do we incorporate data and analytics into our decision-making? So that’s one kind of tension that it sounds like is starting to get resolved.

But I think about this other type of tension, which is between the external pressures: the fans, the media, and in some organizations that would also include the owner. And the people who are bought into this type of thinking, who do say that we can really structure great decisions and we can think about it, we can create repeatable processes, we can reduce the effect of cognitive bias, we can make our decisions less noisy … But I’ve famously opened a book with one of these ponderings, which obviously was back in 2015, so maybe less buy-in back then, but people are still mad about it …

[In situations] where the analytics [and] the way that you think about it from a decision analysis standpoint would bring you to one solution, but it’s really “out of the box,” and if it doesn’t work out, there’s just a lot of career risks that go along with that. So now you’re in a position where you’re dealing with that career risk.

I mean, I think about the evolution of your career and as you’ve been learning along the way, you’ve also had a lot of protection, right? Like if something happened and things didn’t go well at the Eagles, it was going to be Howie Roseman who was going to get the flack for it, or Doug Pederson. But you got to sort of sit behind and integrate all of these things in the way that you wanted to. But now you’re the one who’s sitting out there and you’re the one who’s going to catch the flack. So I’m wondering: how do you deal with that [tension between] internal to the organization where you may have really good alignment, versus how you think about the decision with what the external risks are?

Andrew: That’s a great question. So I’ll talk about it from [let’s] call it an “outside the box” perspective where if it doesn’t work it ends up being a scarlet letter, so to speak. And then I’ll also talk about it from a “time horizon” perspective.

So I’ll start with the first. So that doesn’t bother me anymore. And it used to, and a big reason is because when I was here the first time, that was essentially what happened. Right? We had a stretch of 1-31. And although I wasn’t the general manager, I was part of the senior decision-making group and going through that experience … my boss was let go. I was demoted. And that was when the narrative changed. Like I told you, I didn’t have to deal with those stigmas coming up in scouting, but then immediately once I was hired, or even before then, it was, “Hey, here’s that analytics guy that was part of that 1-31 experience.”

Annie: Who was trying to Hinkie us.

Andrew: Right. And so the industry perspective changed for me personally, and going through that stretch was really hard at times. Certainly for the last year and a half when I was in Cleveland, and it even followed me to Philadelphia. So when I got back here in the decision-making role, I had been through enough adversity and personal career adversity where I wasn’t really afraid of the worst case outcome. It’s like, I just lived through 1-15, 0-16, a really difficult experience of demotion, a period where you’re a little bit of a pariah from an industry standpoint. So once I get through that, I’m not really worried about making a decision that’s controversial if I think it’s the right thing for the organization.

So that part is not hard for me. I think the other challenge, though, is that a lot of what we talk about from a decision-making standpoint is decision-making over time. A lot of what we talk about as being long-run focused. It’s kind of like investing, you know, no fund manager has an indefinite time horizon, and neither do we.

I think that because in sports in general, it’s emotional, right? It’s emotional for fans. It’s emotional for players. It’s emotional for owners. It’s emotional for the coaching staff, front office staff. I think generally speaking sports can be a little bit impulsive. At the end of the day, people look at one lost record and outcome. And because of that, that can really drive decision-making. Because of that pressure that you alluded to that you may not get, “Hey, this is a good strategy and it’ll work over the course of 10 years.” Unfortunately, if you get unlucky in the first three, or if it doesn’t work out in the first three, that can realistically still be a problem.

So I think [the balance is] fighting against that natural inclination, because that is real pressure, but also understanding that your decision-making strategies should adjust based on time-horizon. That should be an input in terms of how you would behave. Now that doesn’t mean that everything should be so shortsighted and outcome-based, but there is a little bit of trade off depending on where your team members are, or when you’re expected to deliver a certain set of results.

---

“Your decision-making strategies should adjust based on time-horizon … Now that doesn’t mean that everything should be so shortsighted and outcome-based, but there is a little bit of trade off depending on … when you’re expected to deliver a certain set of results.” — Andrew Berry

---

Annie: Yeah. I mean, it’s just interesting trying to think about how you strike the balance. I made a joke about Hinkie, but it’s a real problem, right? Tanking the 76ers, he wasn’t around to see what the results of that were, although it’s very easy to track the lineage of those results. And obviously having a 10 year time horizon, well, “We’re not going to win for 10 years.” Nobody’s going to accept that.

So how do you think about what the right balance is between the short and long term time horizon, as you’re going into the draft, for example?

Andrew: So I think for me coming off an 11-5 season, we’ll wait a little bit more, generally with some of our decisions in terms of short term results, relative to when I came into the organization in 2016, when we had a roster that was older, very expensive, and not very good. All of that’s going to be really shifted towards a longer term focus to build this foundation. But when we’re in a time where we think that we’re in a window of contention, we probably will make a few more short-term trade-offs relative to the long term.

Now, from a general manager’s seat, let’s talk about it from a roster-building perspective, that’ll depend on what mode of acquisition we are discussing. So you mentioned the draft. To me, the draft is all about more longer-term asset management. And that’s the mode of acquisition that allows you to build a sustainable foundation.

So probably with very few exceptions, we will be a lot more long-term focused as opposed to, “Hey, look, we need X now, because we think we can be really competitive this year.”  That’s what allows you to be competitive year over year. Now, [with] free agency, even from the trade market, there may be certain bets like, “Hey, we’ve been a competitive team in the past. We really wouldn’t have acquired players on shorter term deals who are older or towards the tail end of their career. Now, there may be a little bit of leniency depending on the role or depending on the player,” or “Hey, we wouldn’t be major players in this market because we just haven’t built up our foundation. Now, we may be able to polish a few different sides of the ball because we are in a window where we do think that we’re able to contend.”

Annie: That’s such an interesting way to think about it, the different modes of acquisition with what the time horizon is and how that interacts. So this makes me think about something. So, you obviously came in, and like magic …

Andrew: I wish it was like magic!

Annie: I know! For the first time in 18 years, the Browns made the playoffs, you had an 11-5 season. So you mentioned Sashi Brown, and obviously there was some of that tanking strategy. For people who don’t know, it doesn’t mean losing on purpose, it means stripping your team down in order to build it for the longer run. It’s getting some focus on those longer term decisions. So, but those have to have time to play out. So, if you’re thinking about the success that you had coming in this year, how much would you say is the influence of what was happening [due to] Sashi Brown in terms of, as you come in the experience that you had this year with the success?

Andrew: It’s a great question, Annie, because I think oftentimes, whether it’s fans or teams, [they] would like to suggest that any outcome that happens in a given year is solely because of the decisions in that year. And that’s not the case. So one thing I’ll push back on … I know Sashi is not on here to talk about it, but the strategy really wasn’t tanking in 2016. We were certainly disappointed we didn’t win more games, but you’re right that part of the focus was on making sure that we could acquire and draft and have enough resources from a cap perspective that we could have a long-term sustainable build and keep a young core together. And I think the resources at the disposal of the team, the quantity and quality of picks, and the cap space that was accumulated during Sashi’s time has a huge impact on the success that the team is having today.

And likewise, not just Sashi, you had the general manager after him, John Dorsey, deploying some of those resources. All of those play a role in terms of the team’s success. And I think it would be disingenuous to suggest that there wasn’t some good done during that time period by people who were in the decision-making seat before I was, and I think that’s true for every franchise.

Annie: Yeah, it’s interesting just thinking about: how do you balance having that core built up? How do you balance those time horizons in terms of the decisions that you’re making? How do you make sure that your approach is flexible so that you don’t become someone who’s just really focused on young talent or someone who’s really focused on the proven entity, but you have a mixed strategy when it comes to that?

Andrew: That’s another great question, Annie. I think part of it is just making sure you keep the ultimate goal in mind. The goal at the end of the day is to win a Super Bowl. And  depending on where you think you are on the win curve, so to speak, will dictate some mix of short-term and long-term trade-off.

---

“The goal at the end of the day is to win a Super Bowl. And depending on where you think you are on the win curve, so to speak, will dictate some mix of short-term and long-term trade-off.”
— Andrew Berry

---

Because I think it’s easy to think about “sustainability, sustainability, sustainability,” or “win now, win now, win now,” when what makes the general managers’ jobs both hard, but also very interesting and fun, is that you have a foot in the present and a foot in the future. And hopefully you’re never standing on one foot, but perhaps how much weight you’re putting on one or the other will depend on really where you are in your team’s life cycle. Because as you can probably guess, I probably am … probably general managers in general are a little bit more longer-term focused because many of us grew up thinking about the draft, and young players, and things along those lines.

I have to catch myself … especially if you have a good team … the goal is to win the Super Bowl. Let’s not forget that the goal is to win the Super Bowl. Not at all costs. You don’t want to mortgage the sustainability of the team, but the goal is to win the Super Bowl. And I think if you don’t lose sight of that, that at least helps.

---

“What makes general managers’ jobs hard, but also very interesting and fun, is that you have a foot in the present and a foot in the future. And hopefully you’re never standing on one foot, but how much weight you’re putting on one or the other will depend on where you are in your team’s life cycle.”
— Andrew Berry

---

Annie: Oh, I love that. I love the distilling of that particular problem. So one of the things that’s really interesting is back in the seventies, I don’t know if you’re familiar with him, but there’s a guy named Barry Staw who has done a whole bunch of really interesting work on something called escalation of commitment, also known as escalation traps. And there’s a broader psychological phenomenon that would encompass the sunk cost fallacy.

So for people who don’t know, sunk cost is: you have resources that you’ve spent on something, and the question is, do you want to continue in that endeavor? And when you’re already in the losses, when you’ve already spent, you’re just more likely to continue in the endeavor, even if that endeavor is failing. And obviously there’s problems with that because whatever dollar you spend to continue, or time that you spend to continue, you could be spending on other things that might be more profitable.

So this is a big football problem because every single time you choose to spend on a player, and you have a salary cap, so you don’t have unlimited resources, and there are times that you can trade people, but not always, you have this problem. And so he noticed first in the NBA this very interesting phenomenon, which is that if you completely normalized for skill, so you took two players who were exactly identical in skill, and one was a very high draft pick and the other wasn’t, what would happen is that the high draft pick would get much more playing time and actually be much less likely to be traded. So they would stay on the team longer. Now, again, I’m saying these are totally equally skilled players, and obviously you can see even bigger things like that. Like let’s say they’re a high draft pick and you’ve traded away a bunch of draft picks to get them. So you’ve traded your way up to get them. And this has since been replicated in the NFL. So we see this problem in the NFL and this obviously creates a problem in terms of trading, and how you are thinking about play time … and maybe Carson Wentz’s in the back of my head here. I don’t know. I’m from Philly. I can’t help it!

So I’m interested in: how do you think about those kinds of problems? Because obviously this type of problem of a sunk cost and endowment and escalation of commitment has such a big effect on the way you’re managing your salary cap and the way you’re managing free agency and trading. And I’m wondering just how have you been thinking in this particular area about how to get more rational about what the value of a player is, regardless of whether they are presently on your roster or not?

---

“[There’s a] psychological phenomenon that would encompass the sunk cost fallacy … when you’ve already spent, you’re more likely to continue in the endeavor, even if that endeavor is failing…
So we see this problem in the NFL and this creates a problem in terms of trading,
and how you are thinking about play time ” — Annie Duke

---

Andrew: It’s a very hard challenge for teams and decision makers, because you’re right. It’s not even just what you invest financially, or in terms of draft capital, but what you invest emotionally or mentally into the decision. And you want to give it every chance to work out. So before I answer your question directly—and this may be just the scout in me—I think certainly part of the reason is this dynamic we’re talking about, the sunk cost dynamic, but I also do think that there a part of it is like you can have two equally skilled players and with the higher draft pick, usually, maybe they’re the better physical talent. So there is a higher, or at least perceived higher, upside as that player develops over time, that may not be tied to the cost of acquiring. Now, I think it’s a fair question whether we, as an industry, measure upside accurately or well, but just to defend us emotional decision-makers and us as scouts, I would be remiss if I didn’t at least say that.

Annie: Well, let me just say though, this isn’t an emotional decision problem … There is not a single human being who doesn’t have this issue, right? You know, you’ve invested in something and you refuse to sell, because you’re behind. I saw this in poker all the time. You know, people would be in a poker game and they’d be losing and someone will come up and say, “Hey, it’s our anniversary. We had a dinner date tonight.” And they’d be like, “No, this is the best game I ever played in. I’m sorry, we’re going to have to postpone our anniversary until tomorrow night.” So it’s a similar kind of problem. Right? So these kinds of issues of: how do you think rationally about when you’re supposed to let go of something? They’re just really hard. And I am thinking about the NFL that this problem must come up repeatedly because you have free agency, you know, and so on and so forth, you’re trading draft picks up and down.

Andrew: It certainly does. So there are a couple of things that we try to do, and by no means am I perfect in this area.

So I think, number one, let’s talk about just the emotion piece. One is trying to do the hardcore analysis decision-making removed from the emotion of the season, the emotion of the player, and the emotion of the decision. And that may sound simple, but the off season comes quickly in the NFL and it can be easy to just think of the last thing that happened in the season, or something along those lines, and just not have enough distance from a moment that will always stick in your mind regarding a particular player.

I think the second part is getting a number of independent perspectives and letting those independent perspectives state their case, speak their mind. Because it’s easy as a decision maker, just to be so much in your own cocoon that you’re thinking about only your viewpoint on the player. So whether you’re making a free agency decision, individual evaluations from your pro scouting group, or the individual evaluations from your research and strategy group, or the individual writing evaluations from your coaching staff, because usually that gives you a fair analysis, with at least taking out some of the bias. Now there will always probably be some organizational bias because it’s easy for people, when you’re living with a player day-to-day, to be clouded by what everyone’s saying in the building. But usually that helps out a little bit.

The other thing is that we do have a lot of good objective data internally that as much as I may like dislike what I’ve seen or what has happened recently with an individual player on the roster, it’s a good anchor where it’s like, “all right, well, this really isn’t bias,” or “this is largely unbiased.”
And then really the last thing that we try to do is to have a third-party perspective. So there are a couple things that we use that aren’t in our building, because as much as you want to try and weed out every type of bias with all of your internal evaluation methods, that’s probably to some degree impossible, or at least a bit challenging.

And so it’s really taking into account all those perspectives and maybe digging in where those perspectives disagree to make sure that you actually get to the truth. But the only way that I can do that is if I’m removed from the emotion of the decision. So if it tells me something that I really don’t want to hear, I can take it in and ultimately make the best decision for the organization, as opposed to something that’s going to just satisfy me or satisfy my desires.

---

“[By] getting a number of independent perspectives [and] objective data … taking into account all those perspectives, and digging in where those perspectives disagree … you actually get to the truth … [Even] if it tells me something that I really don’t want to hear, I can take it in and
ultimately make the best decision for the organization” — Andrew Berry

---

Annie: Yes. I love that. So you said two things in there that I think are so great. So one of the things is that Barry Staw, who I mentioned before, mentioned that if you have a different person make the decision about quitting or stopping whatever you’re doing, then you, the person who made the decision to start, you do better. It sounds like you do that by getting a third-party perspective, someone who doesn’t have ownership over the original decision, like it’s not their baby, you know, that decision.

So I love that you’re doing that. But the other thing that you mentioned is just getting a lot of eyes on it. You know, what are the different perspectives … and going with the campaign that you launched called Be The Solution. I’ve heard you talk so passionately about the relationship between diversity from a social justice perspective and diversity from a decision-making perspective and the way those things play together.

And I’d love to understand, first of all, what Be The Solution is, and then get your thoughts on the way that those play together, and why we should really care about that.

Andrew: Yeah. So, Be The Solution is a campaign that we launched in the wake of George Floyd’s death, when the entire country was dealing with social unrest. And what I found is that it had a very strong effect on me, and it had a very strong effect on a number of people within our organization. But the thing that I actually found is that a lot of people wanted to either do something or truly understand why so many people across the country were so passionate, but they didn’t really know where to start or how to get involved.

So where we were fortunate, with the platform that we have as an NFL organization and the support that we have from our ownership group—the Haslam family—is that we were able to really use our platform to provide people a way to get educated, to provide them the resources, whether it was with books, podcasts, videos, documentaries. But then [we] also have specific ways that they could get actively involved socially across our city of Cleveland.

I’m really proud because our organization has really been at the forefront of this. People have gotten engaged and it’s been something that’s been healthy for our city.  And just really in hopes of using our platform to do something good.

So your second question Annie, about just really the value of diversity. I don’t think that people often stop enough to think about why that’s important. And then also what diversity really means.

Let’s take out the ethical considerations, take it from just a pure utilitarian … business perspective. I think [diversity] helps teams make better decisions, like no different than we talked about the different perspectives from our scouts or coaches or analysts. Because diversity … I think people oftentimes just focus on race. And demographic diversity is important, whether it’s race or gender or age, you name it, because that oftentimes shapes our experiences and perspectives, and that’s really what you’re trying to grab. But experiential diversity, whether it’s someone who’s been in the business for a long time or it’s been short, or they’ve just had different experiences within the business. And then cognitive diversity—how people actually think—all three of those are important.

And that’s really what we strive to do when we hire organizations. And ultimately when we bring people to the decision-making table, because that’s what allows us to—you’ll probably never eliminate all blind spots as a decision maker—but we at least try to eliminate as many as possible, and that’s at least been an effective way to do that, for us.

---

“[Diversity] helps teams make better decisions … Demographic diversity is important, whether it’s race, gender or age, because that oftentimes shapes our experiences and perspectives. But experiential diversity, different experiences within the business, and cognitive diversity, how people think, are [also] important. And that’s really what we strive to do when we bring people to  the decision-making table, because … you’ll probably never eliminate all blind spots as a decision maker,
but we at least try to eliminate as many as possible.”— Andrew Berry

---

Annie: Yeah. I love that perspective. The way that you can think about marrying the social justice issue with the cognitive diversity issue, and think about diversity from many different angles, I guess from many diverse angles. And the way that completes a picture of great decision-making.

So just one last thing before I get to our closing questions. I heard you on Huddle and Flow, which was amazing, everybody should go listen to that podcast. You were so great on it. And  you said something that really caught my eye—or my ear, I guess, because I was listening—that I thought was so great. They asked you how you dealt with this particular season. Which I think was so difficult in so many different ways because you know, you’d wake up in the morning and wouldn’t necessarily know what your roster was going to be because of COVID. And now, thinking further about the way you take that uncertainty into, for example, this year’s draft, where obviously the college season itself (where you would have been scouting from) has been quite strange.

And I thought your answer was so interesting and so insightful. I’m going to paraphrase it really shortly, and then I’d love for you to elaborate, which was, “well, it’s not really different than any decisions that I’m having to make, even when it’s not COVID.” I found that incredibly deep. So I would love to hear your thoughts on that and why my ears were perking up there.

---

“At some point decision-makers in sports, they just have to get comfortable with being uncomfortable … If it’s not COVID, it’s going to be something else. Whether it’s an injury situation, a staffing situation, some other weather-related emergency, a drought. You just don’t know what it’s going to be, but a lot of the ways that you handle crises and decision-making under uncertainty in [COVID] will apply to a year where we didn’t know what the salary cap was going to be until a couple of days before the league year … The lack of full information that you had is similar and you still have to do your
best to make the right decision with it anyway.”— Andrew Berry

---

Andrew: I guess part of it is, if I could define the general manager’s job into two things, it would be: crisis management and decision-making under uncertainty. Just the forms that those take differ from year-to-year, or week-to-week, or quite unfortunately, day-to-day.  But it’s the same principles. Like we talk about computer science, it’s kind of like the idea of abstraction. It’s the same process, or same situations, or same general principles, but the situation is different. And so, one thing that our chief strategy officer … I remember he said this years ago, he goes “At some point decision-makers in sports, they just have to get comfortable with being uncomfortable.” And it’s so true. And so, people always assume that COVID is gone and your life’s going to be so much easier, everything’s going to be downhill. And it’s like, no, that’s not going to be the case. If it’s not COVID, it’s going to be something else. Whether it’s an injury situation, a staffing situation, some other weather-related emergency, a drought. You just don’t know what it’s going to be, but a lot of the ways that you handle crises and decision-making under uncertainty in a year where the primary thing was COVID, and the roster, and something along those lines, will apply to a year where we didn’t know what the salary cap was going to be until a couple of days before the league year. And although those are two different problems, the lack of full information that you had is similar and you still have to do your best to make the right decision with it anyway.

---

“I could define the general manager’s job into two things, it would be: crisis management
and decision-making under uncertainty.”— Andrew Berry

---

Annie: Yeah, so my ears really perked up to that, because one of the things that I’ve been saying is basically when you look across a lot of the cognitive biases that really frustrate our decision-making, a lot of them have to do with being overly certain, right? More certain than the world would actually tell you to be, whether it’s overconfidence or the illusion of control, we think we have more control over our own destinies, that the world is more predictable than it actually is. What COVID has done is essentially forced you to stop pretending. [At least] to some degree, I mean, we’re still overconfident, but stop pretending. It’s like you had blinders on to the uncertainty and then COVID came along and just took them off for a second. And now, as soon as we’re all vaccinated, we’re back out, everyone’s going to put those blinders on. And so that’s why I loved what you said. Your point was, it’s not really much different, it’s just that you might be seeing it more, but it’s always there. And that’s the whole point is that you have to get comfortable with the world as it actually is. And take a lesson from this moment where you couldn’t pretend.

---

“We think we have more control over our own destinies, that the world is more predictable than it actually is. What COVID has done is essentially forced you to stop pretending …  It’s like you had blinders on to the uncertainty and then COVID came along and just took them off for a second … The whole point is that you have to get comfortable with the world as it actually is. And take a lesson from this moment where you couldn’t pretend.”— Annie Duke

---

Andrew: That’s a hundred percent accurate, Annie. A hundred percent accurate.

Annie: Well, so I mean, I heard something, I went, “Oh, that’s so good, I love that.” You know, full disclosure, partly because I’ve sort of been saying that. So I was very excited to hear my point of view parroted back at me, which is something all human beings really enjoy.
Okay, so first of all, let me just say, this has been incredible. This is such a great conversation. It’s very clear why you’re the youngest GM in football history. And just in terms of being able to get into your mind and the way you think about these decisions and how thoughtful you are in framing and thinking about different time horizons and the interaction between heart and head. And so anyway, I just want to just tell you how grateful I am for this conversation.

So just pivoting a little bit to the Alliance [for Decision Education] itself. You know, if you were to think about the mission of the Alliance, which is to empower every student with decision skills. So in the same way that we think about empowering them with social, emotional skills, and also apparently trigonometry skills. But let’s say that we were to succeed and empower them with decision skills. What would you expect? How would you expect society to look different?

Andrew: I actually, I think the biggest challenge … because we bring kids up, we really tell them what to do or what to think as opposed to how to think and how to make decisions. And I think that ends up bleeding into society where it’s … a little bit like what we were talking about earlier with the idea of certainty. You’re like, “Hey, this is how things are done.” Or we almost operate in a world where we think it’s a deterministic environment, when it’s completely the opposite of that.

I think you’d probably have a more fast changing society. From the perspective of: I don’t think people will be as indoctrinated into tradition or ways things have been done in the past. I think they’d evolve more quickly with the pace of the world. I also think you’d have more respectable dialogue back and forth because people would be aware of their own biases and  the inherent uncertainty with a belief, a decision or, or just a general problem. Because I think overconfidence is a huge challenge. It’s a huge challenge in the business world. It’s a huge challenge in society. And I think a big part of better decision-making is understanding how uncertain most environments are. And I think that that would lead to better discourse overall.

---

“[If we empowered every student with decision-making skills] I think you’d have more respectable dialogue back and forth, because people would be aware of their own biases and the inherent uncertainty with a belief, a decision or, or just a general problem. Because I think overconfidence is a huge challenge. It’s a huge challenge in the business world. It’s a huge challenge in society. And I think a big part of better decision-making is understanding how uncertain most environments are.
And I think that that would lead to better discourse overall.”— Andrew Berry

---

Annie: Yeah. I love what you said about that. Because I think we have this tendency to think that if somebody holds a different point of view than we do, that they’re evil. And then you have this very strange Hanlon’s razor, which is: never attribute to malice what you can attribute to stupidity. And I’m like, well, I don’t know that that’s any better! Why not just [think that] someone can hold different points of view than you for totally rational reasons or a mix, and we should be open-minded to listening to what they say. And obviously there are boundaries to that, no question, but I hear what you’re saying about how we might be more respectful in the way that we interact with each other in a way that would be more healthy for society.

As you’re thinking about your job now, as GM, and you think about the players coming onto your team, and then obviously there’s so many different roles that you’re having to fill across the organization, can you think about: what’s the one decision-making tool, idea, or strategy that, if you could magically have out there in the world, would improve your ability to have really great quality decision makers in your organization?

Andrew: Is this something that actually has to exist from a process standpoint or my dream?

Annie: You get to dream it up.

Andrew: If we had a machine or a pill where you could eliminate confirmation bias, that would be the number one thing that I wanted. And I say that largely because I know I have a problem with that. But I think it’s one of the things I see all the time in football. I go into every draft process knowing that I have to actively fight against that for myself. As you get more information, you try to apply it to the view that you’ve taken. If there was a way to completely eliminate that bias, I would sign up for it and spread it all across the organization.

---

“If we … could eliminate confirmation bias, that would be the number one thing that I wanted … It’s one of the things I see all the time in football. I go into every draft process knowing that I have to actively fight against that for myself. As you get more information, you try to apply it to the view that you’ve taken. If there was a way to completely eliminate that bias, I would sign up for it
and spread it all across the organization.”— Andrew Berry

---

Annie: Is there a particular book you might recommend to listeners that you think would help improve their decision-making?

Andrew: That’s a good one. Probably Thinking, Fast and Slow, that was good. I listened to that one on audio, probably my last year in Indianapolis. And it just was rich in content, it just made me try to be more conscious as far as using the different systems in my brain, just trying to be a little bit more thoughtful and deliberate in my own thought processes.

Annie: Well, we’ll make sure that Thinking, Fast and Slow is linked in the show notes for everybody to go read the Kahneman book, which is so amazing. Okay. Last question. And then I’d love for listeners to be able to know where they can go find you.

So you know, way back … you probably didn’t think I was listening, but it was … about your parents. And I’m just curious, was there some sort of family explosion, like a huge fight that occurred between your Cowboys-loving father and you in Philadelphia all of a sudden? I would just like to say, I know from being in Philadelphia, we’re talking about a bitter rivalry here.

Andrew: No, and that was the bitterest of all bitter rivals in the NFC East for my dad, because he had actually worked in Philadelphia when we were young, as a Cowboys fan, and he was in the news business so it was tough. At that point, he was a fan of his son. I’m sure he won’t admit it. I’m sure a little piece of him died. But he was all Eagles gear the entire year. Now I will say this though with the Browns, he’s back to his cowboy stuff, but you know. It’s not Browns related, but he was a great fan and supporter of his son the entire time.

Annie: Yeah. I mean, I’m pretty sure that for either Eagles or Cowboys fans, if you were to make a bet … and the bet with an Eagles fan is to go wear Cowboys gear to a game or vice versa, that would be so much worse than betting like $2,000. “No, I’ll pay the money!”

Andrew: That’s a hundred percent right. So I’ll tell you the funny thing, as much as my dad was all Eagles all the time, I did have some relatives on his side of the family that were like, “Love my grandson, love my nephew, but we’ll root for him for every game but the two that they play the Cowboys during the year.”

Annie: Ha! There you go, see!

Andrew: They say blood’s thicker than water, but I’m not quite sure … that’s false for sports.

Annie: It’s not thicker than football, apparently in some cases. Yeah. That’s what I was curious about.

So, if listeners want to go online and find out more about you, more about the Browns, particularly more about Be The Solution. Can you let them know where they might find you follow you on social media, et cetera?

Andrew: Unfortunately, I’m not a social media guy, so I don’t know.

Annie: That’s probably fortunate for you, not for us, but for you.

Andrew: So I’m not on, you know, Facebook, Instagram, and you know, anything along those lines.

Annie: You can go to the social media sites where you won’t find Andrew, but you can find #BeTheSolution. And obviously we’ll link to all of that in the show notes for people who are really interested in  getting involved with that so that they can feel like they’re doing something and contributing.
That’s amazing. Well, listen, I am so excited to have gotten the time to speak with you. All the stuff that we talked about, again, it’s going to be linked in the show notes. You’ll be able to get a transcript of today. And Andrew, every conversation that I have with you, I feel like I learned so much from you.

Andrew: The feeling is mutual. I learn a lot more from you than you learned from me Annie.

Annie: Well, I don’t know about that, but I so appreciate you taking the time out. I know this is a really busy time. So thank you so much for having this conversation with us.

Andrew:  No, I appreciate you Annie, and thank you for having me on.