So to Speak podcast transcript: Thinking about free speech in bets with Annie Duke

Note: This is an unedited rush transcript. Please check any quotations against the audio recording.

Nico Perrino: All right, so Annie Duke, welcome on the show.

Annie Duke: Well, thanks for having me. I’m so excited because this is one of the rare ones where I could just come to the studio instead of be on the phone.

Nico: I know, I know, it’s nice. So, I want to start by cribbing from the economist, Friedrich Hayek’s essay... famous 1945 essay, The Use of Knowledge in Society because I was reading your book, I was reflecting on many years ago when I first read that essay and I was like, there are some parallels here; specifically as your title suggests, the use of knowledge when the knowledge is incomplete and you need to make a decision.

You argue that the way we make decisions and think about those decisions can be compared to playing poker and is rife with errors and in Frederich Hayek’s essay, he draws parallels between a free market economy and a planned economy and in a free market economy, you have a lot of people with different knowledge in different places. But in a planned economy, you have one person and that knowledge isn’t all there in that one… one person.

So, I… I… I just wanna lay that out there, I was thinking about that essay in the context of your book, but in thinking about an individual, not planned economies or free market economies, what sort of errors do we make in thinking about our decisions?

Annie: Oh my gosh, that’s such a broad question. So, let me… let me actually try to answer it in a broad way and then maybe we can get down into the details.

Nico: Yeah.

Annie: So, here is the general problem, there are two sources of uncertainty for us. Source number one is luck, those are things that are outside of our control and you can broadly think about those as… as things that are… the… are happening in the world that you can’t make any difference. So, for example, I can’t control the actions of somebody else, just as an example so if I’m going through… if I’m following the rules of the road and someone doesn’t follow the rules of the road, I can’t… I don’t have any control over that so if they hit me, there’s… that’s –

[Crosstalk]

Nico: So, if you’re going through a green light and they go… they run a red light, you can’t control that?

Annie: Right, so you can think about that in poker that I don’t control the cards that are getting dealt. That’s… that’s random and I don’t have any control over that. So, there’s luck, but then there’s also hidden information, you can also think about it as information asymmetry. There are things that I know and then there’s lots of information that I don’t know. So, you can think in poker there’s an asymmetry, I know my cards, but I don’t know yours and for you the same thing, you know your cards, you don’t know mine.

And this is generally true of any kind of decision that we have to make, we just don’t have perfect knowledge of the world. Now, what that means is that when we make decisions, any given decision can have a broad set of possible outcomes, even though only one outcome will actually occur. So, there’s this huge set of possible outcomes because of the intervention of luck and because we don’t have all the information so we’re making a decision that’s gonna be a little bit noisy and what that means is that there’s lots of ways things can happen.

So, now the problem for us is that what that creates is a very loose connection between the thing that actually happens and the decision quality because we can make a very good decision, we can have it turn out well; we can make a very good decision, we can have it turn out poorly; we can make a bad decision, we can have that turn out well; we can have… make a good decision and we could have that turn out well, as well.

Nico: Yeah, well in your book, you talk the situation… a… a CEO or someone is on the board of some company and their president wasn’t doing all that great of a job; wasn’t doing well compared with other people in the market; other companies in the market and they decided to fire the CEO or was it the president? I forget exactly.

Annie: You know, it’s about… you have the contours essentially.

[Crosstalk]

Nico: The contours essentially right, but then they hire a new CEO or a new president and it doesn’t work out well for them; the person’s not performing well and the company continues to slide and you talked to this person about the decision process that went into firing that first CEO or president, and to you, it seemed pretty sound, but you could never predict what will happen as a result of that decision because of the intervention of luck or incomplete information in hiring this new person, or other market forces that come into it. So, you say… you tell… you tell us to separate the decision from the result.

Annie: Right and this… this is… this is generally where all of our decision problems come in so because there’s this noise, because there’s this looseness in this connection, it allows for all these biases in the way that we sort of try to make order of the world because we’re not really good at luck and hidden information. So, the example that… I’ll get to the CEO example so let me give a simple example so in chess, there isn’t a lot of places for us to go wrong in terms of our interpretation of why things turn out the way they do because in chess, if I lose a game of chess to you, what do we know about my decision-making in comparison to yours?

Nico: That it’s poor.

Annie: It’s worse, right.

Nico: Yeah.

Annie: And that’s because we’re taking away this very strong luck element, like the pieces sort of stay where they are until someone moves them through an act of skill and I know that those pieces are gonna stay where they are. So, we’re taking away that very strong element, like you roll the dice and off comes your bishop. And, we don’t have the hidden information, there isn’t the same information asymmetry, there’s… and there’s a little bit, but not a lot so I can see your whole position; you can see my whole position. That means that I can see every possible move that you could make; you can see every possible move that I can make.

What that means is that the connection between the way the game turns out and the quality of the decisions that you make is incredibly tight, which actually elements a lot of the errors that we can make in terms of trying to work backwards, from outcome quality to decision quality. And also –

Nico: Is this is why, AI for example, almost always beats a human in chess?

Annie: Yes, because –

Nico: Because they know all the possible moves that could come.

Annie: they could brute force it till the end of the game.

Nico: Yeah, I even heard about how that plays out when you’re playing poker, but there is more opportunity for luck. Like no matter how smart the artificial intelligent is, it can’t predict the next cards that are gonna.. it can predict the likelihood, but not what exact cards are gonna come.

Annie: Yeah, so AI’s doing a little bit better in poker, but it’s been flow, it really lags behind chess for that reason. So, it… it seems to be doing… doing pretty well in one… one-on-one situations where you’re sort of limiting the problem, you’re putting some limitations on the problem, we’ll see how it does as you start to expand that set, but that’s exactly why. So... so, we have because of this loose connection, there’s all sorts of ways that we can go wrong so one of the ways that we can go wrong, as you pointed out, is this resulting problem.

So, if we think about chess, if I loose a game, I know that I played poorly. And if I win a game, I know that I played well, in relation to my opponent, not in an absolute sense. So, that’s fine, but what we do is we try to act as if chess applies to everything. So, in the case of poker, it would be like saying if I win a hand, that necessarily means I must have played it well and if I lose a hand, that necessarily must mean I played it poorly, but we know that that’s not so because there’s too loose a connection so it’s actually a really big decision error we make.

In the case of the CEO, you could see this, I think I wanna pretend like we’re playing chess because people are really uncomfortable with this uncertainty so what happened with the CEO was he had a division president, the division president was under-preforming compared to their competitors. So, he went… I just had asked him… when I come in and do trainings, very often I’ll have people come in prepared with an answer to this… the following two questions; what was the best decision you made in the last year? What was the worst decision you made in the last year?

Now, I’m still waiting for one day someone to come in and say, I made this terrible decision and it worked out better than anything I’ve ever seen; nobody ever does that, they always tell me the worst outcome they’ve had in the last year and the wor… and the best outcome they’ve had in the last year, which is very… so they’re acting like it’s chess. Like oh, I’ve gotta answer this question about what’s the worst decision I made? This thing turned out horribly, clear… clearly that’s it so that’s what this guy did, he came in and he said, well I fired my division president; I now spent the last year on a search and I haven’t been able to find anybody to replace him.

And I said, oh, okay, that sounds like a terrible outcome, but what about the decision process and, as you said, when we actually dug down into that, it was very good. They had really gone through an incredibly thoughtful process, but they… they had actually even considered splitting the job into two in order to sort of play to that person’s strengths and then having like a co-president. And they had actually thought about past searches and things like that.

So… so, it turned out that –

Nico: It was a very thorough decision.

Annie: It was a very thorough decision, very well thought out given the information they had and what they could control, it was actually a good decision, it just turned out poorly. So, that… I mean this is where we really have trouble so if we think about it, we can think about it prospectively and retrospectively; retrospectively as we have these experiences, the feedback of how things turn out, that’s supposed to be what… be what’s supposed to help us learn, but you can see that when it’s not a really clear signal for how good the decision was, we take all the wrong lessons from it and it actually prevents us from learning.

And then prospectively it’s really hard because we just aren’t comfortable with luck and we aren’t good at knowing what we don’t know and until we get really comfortable with that, we can’t actually create a really good map of the future, which makes it actually really hard to make a decision prospectively. So, we get it coming and going basically.

Nico: Yeah, when I… so I just recently purchased a… Aaron’s gonna overhear, our audio engineer’s gonna be tired of hearing me talk about this, but I recently purchased a condo and you actually talk in your book about people’s decisions in purchasing homes; your leased reference, and I just purchased a condo; I loved it, I fell in love with it, I moved in a couple of weeks ago and I arrived, there was about a month and a half before… between when I purchased it and when I moved in. I arrived, boxes in hand, to find that the water heater had ruptured.

Annie: Oh no.

Nico: And there was standing water in three rooms and thousands of dollars worth of damage and I had actually known that the water heater was old when I purchased it. I brought an inspector in, we inspected it, but I thought… I even bought a home warranty to ensure myself against the broken water heater, but it had been running fine and it had been running fine for years and I thought, I’ll just monitor this, I’ll keep an eye on it. The water heater had ruptured during the time that I purchased it and when I moved in.

And one of the pieces of knowledge or information that I didn’t have, was that if you leave a water heater on and you’re not using any o your appliances, like your dishwasher, your washer and dryer or your faucet, water has thermal expansion and expands and it puts pressure on the water heater and… that over time can cause it to rupture. So, I thought I had made a good decision with all the information I had, but I didn’t have enough information and… and it seems to me what you’re arguing is that you can make a good decision based on the knowledge that you have, but there might be other knowledge in there that might result in a poor outcome.

But that doesn’t mean you made a poor decision, necessarily, based on the information that you had so when you’re playing a hand of poke for example, you can make all the right bets based on all the odds you know, guessing what the cards are in other people’s hands, but for example, if you’ve never played this opponent before, you don’t know how they bluff or if they bluff and you can lose that hand, as a result.

Annie: Yeah, so I actually… this… that water heater example is actually a really good way to talk about this so… so I appreciate the example. One of the things that we need to think about, as outcomes occur, is almost always there’s new information that reveals itself after the fact. So, the question that you want to ask yourself is, is that something reasonably that I could have known beforehand? Or is this something that would have only revealed itself afterwards? So, if the answer is, no it’s… this isn’t something reasonably that… that –

Nico: I guess I could have known that.

[Crosstalk]

Annie: No, but like I’m not… I’m not like a plumber –

Nico: General contractor.

Annie: I’m not… I… I’m also not a physicist, like I… or whatever, I don’t… this is not something that was reasonably within what I could have known; the inspector could have known it, like they could have maybe conveyed it to me, but I wouldn’t have even known to know… ask the question of them so if they weren’t gonna volunteer that to me, which is in the luck category, it’s not actually kind of reasonable thing for you to know about; that if you don’t use the water heater, the chances of something like this happen or you know.

So, now what we wanna do, is really ask ourselves that question because then we can decide basically between was this something that I actually made a mistake, not knowing; or is this is just a new info… piece of information now that I can learn from so that going forward, I now can add this into the types of things that I would put under consideration?

Nico: That’s how I’m analyzing it right now, but at the time, I was like, I should never… I should have never purchased this condo, I should… I was catastrophizing is what I was doing, I, you know I had a happy life as a renter, I made terrible decision here –

Annie: Yeah, exactly so we get that global stuff, but then also kind of on the micro level, what will be… happen is we’ll say, I should have seen that coming, I should have known; how could I have been so stupid? That was so dumb and it’s like no, it wasn’t; it wasn’t a reasonable thing for you to know. Now, one of the ways that we can help ourselves to sort of put a pin in that so that we don’t fall for that trap. It’s called hindsight bias, thinking that something was inevitable to happen, is at the time of the decision, we can say to ourselves the following question, is there some piece of information that I think would radically change the decision that I’m about to make?

And if there is, yeah, go find it. Like… and it has to be findable within the time period so if I’m trying to decision say between two cars and I’m trying to decide which car to buy and I’m like 60% to buy car A and 40% to buy car B and I need a new car, I can say, well okay, I’ve got two weeks to make this decision; I’m 60/40, that seems pretty good, it seems like I should just go ahead and buy car A, but before I actually bank that decision, I can just ask myself the question; is there’s somebody’s opinion or some piece of information or something that I could know right now that’s… that would actually really radically change this decision for me?

If the answer is yes, go find it. Oh, wait, my cousin’s a mechanic and he really… let me go talk to him. If the answer is no, I’m already 60/40, there’s nothing that’s gonna really radically change it; then just go ahead and make the decision. Then if it turns out that your particular car is a lemon, you can’t be upset, you don’t… you… there’s no way for you to know that that particular car, that specific car that you bought was gonna have mechanical problems because it happened to be a lemon.

Nico: Yeah.

Annie: So, it’s the same thing with you, it’s like how could you possibly know that water expands and that that –

Nico: Yeah.

Annie: and then, not only that, but that’s only gonna happen a certain, low percentage of the time and you did the right thing, you got an insurance policy against some crazy thing happening anyway.

Nico: Yeah.

Annie: Although it must have been an annoying process.

Nico: Yeah, it… well, it derailed me for like three weeks just because yeah… I’d… I had never owned before so I didn’t know who I was even supposed to call so I… it turns out you call your insurance company when that sort of things happen, but then you also need to find a general contractor to do all the work and I can’t move anything and I was supposed to put flooring in, you couldn’t put the flooring in until the drywall was fixed so it was… it was just a big headache.

But theoretically I mean I… in a perfect world, I would have had all the knowledge and information to know exactly how I was supposed to treat that water heater, but I thought I was doing the right thing by getting the insurance policy; bringing in an inspector, asking them, but there’s –

Annie: And here… yeah –

Nico: just one piece information that I was missing.

Annie: but here’s this very interesting thing, it still might not have changed your decision even if you knew.

Nico: Yes.

Annie: Because if you really looked at that and you said what percentage of the time is this actually gonna burst on me? The percentage of the time is likely pretty low.

Nico: Well, I would have turned off the water –

Annie: Well, right –

Nico: is probably what I would have done.

Annie: so you… you may have, yeah you would have turned off the water. But… but you may not have, you may… you may have said I’ve got an insurance policy and this is really… this is really low probability and –

[Crosstalk]

Nico: Well, there’s… there’s another… there’s another wrinkle in it too, I was emptying the heater water that night that it happened and I was bringing the hose out from the water heater into the drain and a general contractor just so happened to be working in a unit next to me and I said, hey, can you take a quick look at this? Turns out that if you drain your water heater, you’re also supposed to shut down the fuse that goes to it because if the water heater is empty and the fuse… and the electricity is still on to it, it can start a house fire. And –

Annie: See.

Nico: but I would have never known that.

Annie: Right, that’s what I’m saying.

Nico: I would have never known that had that a general contractor not so happened to have been working in the unit next to me and I just so happened to ask him for his opinion. So, it’s like I… I… life is a knowledge problem. Life is a knowledge… and this is what Hayek is talking about in his use of knowledge in a society, is like central planning do… can’t aggregate enough knowledge to distribute resources efficiently. The way that knowledge works, it is very defuse and the free market is best a garnering that knowledge in a way that a central planning commission just can’t.

Annie: Yeah, so I would… I… I would go back to this idea of liquidity, like you have an information market and there’s lots of different ways that you can create a market that’s illiquid so I can create a market that’s illiquid for myself by not going out and looking for information so now I’m not… I don’t have free flowing information that I can go and try to find. But you can think about this in… in… in like the difference when you open up the market of ideas to people.

So, if you’re in a little, tiny town in 1850 and you have your one doctor, this is really an illiquid market in terms of medical information.

Nico: Yes.

Annie: You get to go to that one person and they tell you whatever they’re gonna tell you and this is all you need to know. And now when we think about it, even if you live in that tiny little town, where there’s only that one doctor, and there’s no doctor within hundreds of miles, you have a computer and you can go look at all these different sites where you can start to research what your symptoms are; research… in fact there are doctors who you can access and actually speak to through the Internet so that creates liquidity in the market.

So, this is what we’re trying to think about is the more viewpoints; the more people that we can bring in; the more people that have different perspectives; the more information sources that we have, the more liquid we can create, not just the… the market in general, but also we wanna think about our information market. How are we creating liquidity in our information market? The way that we’re doing that is through making sure that we have access to lots and lots of information, but part of that is not just access, we have to be seeking it; that’s number one.

And number two, and this is really, really important, is that we have to have like a liquid mindset, which means we have to decide that we’re gonna be open-minded to the information that’s coming across our path because otherwise, we’re creating illiquidity in a different way; where it’s basically we just have a shield and information that doesn’t agree with us is just bouncing off of the shield and so then, even though we have access to lots of information and we could actually really reduce the information asymmetries and the hidden information that we have, we’re just not paying attention to it.

Nico: Yeah, well, I mean this is a good segue into my next point, this is so to speak, the free speech podcast, a lot of this book when I read it, struck me as an argument for free speech, in so far as free speech… a free speech regime allows for more information to enter the marketplace. So, there are a lot of different justifications you can have for free speech; you can have it’s good for democracy, why is it good for democracy? Because it allows different perspectives to come in so you don’t get the confirmation bias, which Cass Sunstein talks about in his… in his study on judges.

It also defends individual autonomy, which may not have a strong connection with this information concern; it allows for freedom, but it also allows for a marketplace of idea; the production and dissemination of new knowledge; our discovery of truth and viewpoint diversity and so I’d… I’d say like 75% of those justifications, and I’m sure I’m leaving some out, have some connection with your argument that what we need in order to make good decisions, is more knowledge and access to more knowledge.

And what makes a good decision is our ability to go and seek it out or a desire to go and seek it out and look it… look at it in a sort of unbiased way, putting together… putting to side… aside those cognitive biases that we often engage in, such as motivation… motivated reasoning, which you… which you talk in your book. Did you realize, and I know you read John Stuart Mill because you reference him in your book, did you realize sort of the intersection between those two things as you were writing it? Or just when I invited you on the podcast and I suggested there might be an intersection?

Annie: No, I was… I was thinking about that in advance. You know, I mean I’m… I’m… I’m really… I’m… I’m just… I think it’s really, really important that we approach the world, not asking why are we right, but why are we wrong? And I… I don’t think that that’s the natural way that our brainware is installed.

Nico: Yeah.

Annie: I think that our brainware is installed to say, why our right? Why are we right? Why are we right? And I… I talk about it as approaching the world through this reasoning to be right as opposed to accurate. And this is where, sort of, the title comes in, this idea of thinking in bets, that what happens is that when you ask somebody to bet on something –

Nico: You call it, wanna bet?

Annie: wanna bet.

Nico: Is the title of our chapter or whatever.

Annie: Exactly. It causes the uncertainty to bubble up to the surface, remember I said, we sort of like to… we want the world to be a game of chess. We want there to be these really strong connections between what happens and why; that we should be able to say for sure why. We don’t like randomness, actually there’s a lot of work in sort of the conspiracy theory world, which says that part of the reasons why… why we’re very attracted to conspiracy theories is they sort of solve for randomness. We don’t like the idea that something just randomly bad can happen to us, as an example.

So, when you start framing things as, wanna bet, what happens is it causes you to actually shine a light on the uncertainty. So, to… as an example, one of the things that people are saying right now is, the Democrats are gonna take the House in November. And if you… when somebody says that, notice they’re expressing that with certainty.

Nico: It causes… it costs them nothing to say that.

Annie: It costs them nothing to say that, but if I now said to that person, do you wanna bet on that? You get the same response every time, well I didn’t say for sure.

Nico: Yeah.

Annie: That’s what they say, I didn’t mean 100%. Well, okay, but you said it as if you meant 100% and your brain was acting as if it was 100% so as soon as I say, do you wanna bet? They start to say, well wait a minute, I didn’t mean 100% and what that causes is to not say, am I sure, but how sure am I, which is a really different question. It sounds –

Nico: Which is the question you argue, we should always be asking ourselves.

Annie: Right. And it doesn’t sound like that much of a difference, but it’s a huge difference. Am I sure is as… is saying that there’s some sort of bar, which is certainty, that I can get to, which is completely unrealistic. How sure am I is saying, I know I live in the middle. So, let me try to figure out how sure I am and then what I can do is try to narrow down the uncertainty. Now, in order to do that, in order to make a good bet, what now happens? I become hungry for the marketplace of ideas. I wanna hear other people, what they have to say.

I wanna hear their speech; I wanna hear people who disagree with me in particular because I already know why I’m right, I already know why I believe that the Democrats are gonna take the House in November. What’s really important now for me to decide, whether I wanna make that bet with you, is to go figure out why they might not take the House in December because that allows me to see how sure am I in terms of what kind of bet would I be willing to make here?

So, it forces you to acknowledge; number one that there’s a lot of luck involved, now to November, that’s a lot of time. There’s a lot of stuff, this is… we’re living in very highly volatile political times, there’s a lot of things that can happen between now and then. So, that’s the luck element, I can’t do anything about that, but then there’s the hidden information element. I’m not an expert in districting. So, I can look at the generic vote, but that doesn’t necessarily tell me how is that gonna play out in a highly gerrymandered state, like say Wisconsin?

Nico: Yeah.

Annie: So, now I’m gonna start ask… I’m gonna start going and looking at that, I’m gonna become really information hungry and go start to try to narrow down my own uncertainty. How can we do that without creating that open-mindedness to every, single point of view? And the minute that you approach the world saying, well why might I be wrong? Now, free speech becomes really, really important because I don’t want someone to decide what I get to hear and what I don’t get to hear; what is useful for me or what is not useful for me.

And the other thing that I think it does is it really makes you focus in on that thing of, well, it’s fine when you agree with it, but what happens when someone’s making those decisions where you don’t agree and now that’s cut off from you?

Nico: Yeah, so in this sort of situation, I… I should have been seeking out people… so after I made my… I made my decision about the water heater, I should have been seeking out people who would disagree with my decision about the water heater and trying to figure –

Annie: Yes.

Nico: out why that might potentially be a bad decision. In your book, you talk about the Hillary Clinton-Donald Trump election and how the polls and everyone thought that Hillary Clinton was gonna win because I… I forget what the exact percentage was, 75%… she had a 75% chance… likelihood, but for most lay people they think, oh wow, she’s a shoe thing, but she still had a 1 in 4 or 1… or 1 in 3 chance of losing the election, it just so happened that, that’s what happened. But for us, we look at that and we say the pollsters were wrong.

Annie: Yeah, so I have all sorts of… my head keeps exploding about this polling versus betting markets, versus forecasts and people really kind of mash them up number one and then because people are uncomfortable with uncertainty, when you express something probabilistically, they have these sort of defaults to yes, no and maybe. So, this is what kind of happens, so when you look at say, Nate Silver’s forecast, so… and then I’ll just talk quickly about the difference between a betting marketing and forecasting a poll, but when you look at his forecast, which was based on lots and lots of polls, what you saw in the last week was, some were 60/40; 65/35.

Now, there is nobody who understands probabilities who would say Hillary Clinton is gonna win the election. They would say, she’s… she’s more likely than Donal Trump t win the election, but you know, that’s what they would say, she’s more likely. But the question that I have for people who then say, well they were wrong is okay, so I have a coin; 65% of the time, it’s gonna land heads; 35% of the time, it’s gonna land tails. How much of your net worth would you like to bet on a single flip that it lands heads?

Nico: Yeah, zero.

Annie: Are you… are you betting all of it? I don’t know.

Nico: I’d probably bet a little bit more than zero, but –

Annie: Yeah, you should, well you’re a favorite right so –

Nico: yeah you should, yeah you should.

Annie: I mean you’re… 65% of the time, you’re gonna win the bet, so you should… you should certainly be willing to take that bet, even money. But you’re not betting your net worth.

Nico: Your whole net worth, yeah.

Annie: No, like you’re… I’m… like all right, I’ve got a gun to your head, let’s go, you’re gonna be like no, thank you, take away the gun, but yet people acted as if that was the case, that they would bet their life on it because afterwards, they were so incredibly shocked. And this has to do with this idea of, remember I said the prospective problem, not accepting the uncertainty so that you can’t make good decisions and then when you don’t view the future in the right way, as there’s lots of ways it can turn out and they each have some probability, then after the fact, you’re much more likely to say you were wrong.

And then, we have this further problem with polls, forecasts and betting markets. So, betting markets are like a free market economy where we’re just sort of betting against each other and you saw this in like Brexit where there was a betting market in London and people were just sort of betting so that there was equal action on both sides. Now, remain looked like a big favorite there, of course it’s London, where everybody thinks remain is gonna win the day so that’s just people betting against each other.

Then, there are polls... well then there are forecasts, which is some sort of model, which says given the data that I have and given what I know about past experiences and I have this model of how I think it’s gonna turn out, that would be Nate Silver, it’s 65/35 Clinton/Trump. And then you have a poll; so all a poll is, and this is what people don’t understand, polls cannot be wrong or right; they can have bad methodology.

Nico: Yeah.

Annie: That’s it, but it’s like I’m taking a mock vote that day so all I’m doing, it’s like August and I take a mock vote and the mock vote says it’s 58% Democrats and 42% Republicans and this is the way that people would vote, if we voted that day.

Nico: And the methodology could be bad because it could not… it might not consider third party candidates –

Annie: Correct.

Nico: or people might change their votes or –

Annie: It could be a bad sample size; it could be –

Nico: Or people might not turn out to vote –

Annie: Exactly.

Nico: the people you polled. Yeah.

Annie: Unrepresentative sample; you could… it matters if you’re calling people on the phone or doing it on the Internet or whatever, there’s all sorts of things and Nate Silver’s pretty good at rating the quality of the polls. So, this is what we have to remember, a poll can’t be wrong or right, it’s just a mock vote of a certain set of people and the poll is an input into a forecast or a betting market.

So, when people say the polls were wrong, it’s… it’s like a completely absurd saying… I actually saw something the other day where there was a poll that came out showing Cruz plus six against Beto O’Rourke and then the next day, there was a poll that came out which showed O’Rourke plus two and somebody… there was all this stuff on the Internet, these polls contradict each other. And it’s like, well no they don’t, there… there’s two mock votes, that doesn’t make any sense.

Nico: So, if I’m… if I’m someone who cares about the outcome of this election and I want to get the best possible information to determine what’s gonna happen, what should I look at forecasting models, betting or prediction markets or… or polls? What do you think is the best way to look at this? And then I wanna ask you about prediction markets.

Annie: Yeah, so I think in general, most people can kind of ignore the polls because you want somebody smart to help you with this problem, like okay how good is the poll? So how much should I weight this poll? And then how much should I… so that’s the methodology thing so that’s an in the moment thing, how good is the methodology of this poll? But then also, how predictive have… have polls been in this particular type of election? So, it matters for example, polls are not as predictive in a small, homogeneous state like New Hampshire versus a large, diverse state like… actually Pennsylvania’s a pretty good example –

Nico: Yeah a pretty good example.

Annie: of a large, diverse state. So, and so that could –

Nico: Or Ohio.

Annie: right or Ohio, polls tend to be pretty good in those states which are large and diverse versus small and homogeneous and then… and then you don’t know that particular polling company, what their sampling methods are. And then the other thing that you don’t know is, how predictive are they given how far out from the… from the mark… the vote they are? So, if you take a poll two months beforehand, it’s much less predictive than one minute beforehand; one minute beforehand it’s very predictive.. So, –

Nico: Yeah, because two moths beforehand, you might not know about the… the Access Hollywood tape and the Donald Trump –

Annie: Right.

Nico: you know, it’s just like –

Annie: So, this is go find… go find an expert so you have to… so I’m not an expert in all of this stuff and gosh I don’t wanna take the time to become one. So, I recognize that and so I go try to find someone who can actually sort of vet this for me so Nate Silver, I think is very good at vetting this information and you can go look at his forecast to get a good sense. Now, betting markets, it’s kind of the same thing as a polling problem, it depends on how liquid the betting market is.

So, this is kind of back to Hayek. The more liquid the betting market is… in other words, the more diverse the group of people betting against each other, generally the betting market is gonna give you a pretty good answer –

Nico: Because you have more people bringing more knowledge to bear on the question and there’s skin in the game in the way there’s not in a –

Annie: Exactly.

Nico: a forecasting model –

Annie: Exactly so –

Nico: because they’re putting money on it.

Annie: Right so the reason why the betting markets looked weird in remain was because all the action was coming from London. So, that creates illiquidity in terms of England in general. So, what you’ve got is people who sort of are getting a particular slice of the information, but they’re… you’re not gonna get a really liquid market there. So, –

Nico: So, you need a liquid market that spans the… the whole geography of the country?

Annie: Correct.

Nico: If it’s just London, then you have a skewed –

Annie: Right and, but that’s okay because the betting market isn’t supposed to tell you the answer there. Remember that what the… the bookies want is half and half… they want people on both sides –

Nico: They wanna break even and take the fees.

Annie: They want… right so they don’t really care. So, in terms of a consumer, if you wanna know whether the betting market is telling you something useful or not, there’s kind of two things; one is, is it a liquid market? And two is, is it something that’s like really, really highly specialized where crowd… crowdsourcing wouldn’t be great? So, the example I can give to you of that is, if I have a group of people trying to decide on something and I ask a whole bunch of people, what’s the population of England? This is pretty common knowledge for people in America; that… that taking the average of the guesses is gonna be pretty good.

But if I ask them what’s the population of Malaysia? Now, I’m not gonna get as good an answer because Americans don’t have as good information about that because again, it all goes back to information so it’s probably better to ask just one expert there. So, again, you wanna sort of think about that in terms of what is the question that’s being asked of the betting market? Is it something that generally people are gonna know about? Or is it something that’s like some sort of super highly specialized thing? At which point it’s better to just go back and ask an expert.

Nico: So, that’s a connection to Hayek too. I mean if the central planner is, for example, a very successful hog farmer, they might have very good information on how to run hog farms, but not how to run maybe perhaps, like a soy bean market –

Annie: Right, exactly, exactly. So, we don’t wanna… we need to understand when is crowd gonna be pretty good versus when do you really need to go to an expert?

Nico: So, let’s say you have a very liquid market… prediction market or betting market, would that be more informative than a forecast… like a Nate Silver forecasting just because there’s skin in the game?

Annie: They’re gonna… they’re gonna tend to agree probably. I mean if you have a really good forecaster –

Nico: Does the forecasting model incorporate the betting markets typically? Or –

Annie: Well, it incorporates the same information –

Nico: Information that betting markets are, yeah.

Annie: that the betting market is… is getting so that’s the thing, just the… what is the betting market looking at? They’re looking at all of the polling.

Nico: Yeah.

Annie: And they’re actually kind of looking at Nate Silver, as well. They’re looking at the forecast, his inputs into that, so… so generally, you’re gonna get very similar inputs into those two… into those two things. The nice thing about the betting market is that when you have a really liquid betting market, you have people betting against each other where one person, all their friends are saying, remain is gonna win and another person, all their friends are saying leave is gonna win and so what happens is that a lot of the bias kind of washes out. When it’s liquid.

Nico: Yeah. So, in the United States, prediction markets are illegal right? If you play… if you play them with money.

Annie: They have them, there’s like Predictit, which is not –

Nico: It needs to be off-shore, right? The money needs –

Annie: well predicted isn’t… isn’t for money, that’s the point. Like they do have ones that aren’t… aren’t where you not betting money.

Nico: But that’s a problem. That’s because… that’s a problem because it there’s no… as Nassim Taleb would talk about in his most recent book, Skin in the Game, there’s no skin in the game so you… you’re not… you don’t have that incentive to put aside your biases.

Annie: Well, skin… skin doesn’t have to be money. I mean… and I think this is a really beautiful –

Nico: Well, I mean I guess you could… it could be reputation too, but the reputation –

Annie: It could be reputation; it could be leaderboard; it could be badges.

Nico: It could be gamifying the –

Annie: It could be gamifying it, which is would Predictit does. So, you can have one where… where the skin in the game is the money; you can have things where you’ve gamified it in some way where there are other things that can create skin in the game. Some –

Nico: But presumably, some of the incentives are stronger than the others. I’d imagine money would be more –

Annie: It depends, I mean those badges… like what… I watch my kids play these video game and there’s no money involved whatsoever and they’re very motivated by leaderboards and badges. So, I… I think it kind of depends on what motivates you. For some people, as an example, health creates a lot of skin in the game and for other people it doesn’t. And I think that this tends to be what… what… it just needs to be something that you really value.

Nico: Yeah.

Annie: For some people, reputation is enough; that’s enough skin in the game, I’m putting my reputation on the line with what I’m saying and that… for some people that’s enough.

Nico: So, because I was… I was thinking, I mean we talked about those justifications for free speech earlier; democracy, individual autonomy, freedom, but if knowledge is your most important justification for free speech, then legalizing betting markets or prediction markets could –

Annie: I’d love it.

Nico: it could… I mean you can make a First Amendment argument for it –

Annie: Yes.

Nico: because it would provide people with more information about everything from politics to who’s gonna win this sports game to the likelihood that there’s gonna be a I… I don’t know, a house fire. It could be anything –

Annie: Yeah.

Nico: I mean anything in which knowledge would be useful in making the decision.

Annie: I… I think it would be incredibly useful and people can go look at… and this is actually a different kinds of… kind of useful in terms of, again if we think about information that’s available to us, we can look at what the world’s betting markets, like Betfair are saying about, for example, the American election. That’s really interesting because then you get to see what somebody… what… what the European market is thinking about America. That’s a really nice way to get an alternative perspective because we live in America, we see… we consume American media; we talk to Americans.

It’s really good for us to go and look at what the betting markets in Europe are saying in order to get another perspective –

Nico: Because they are legal in… they are legal –

Annie: They are legal there. And most of the arguments… I think there’s a huge information argument; a huge free speech argument for allowing these betting markets, it’s a way for people to exercise their speech in a way that actually gives people pretty good information. Most of the arguments that I’ve seen against legalizing betting markets, go in sort of the… the moral category, like it’s immoral, it’s bad, or… or a real sort of concept creep on direct harm, where it’s like well, I knew a guy and he was gambling and he lost his house and his children were very sad because of it, which is… that’s a very sad story.

I don’t… I don’t particularly think that that’s an argument against preventing people from gambling.

Nico: Yeah. I mean well, if that’s your argument against prediction argument… or markets, that would… you would have to argue against legalized… making all –

Annie: Alcohol.

Nico: Yeah, alcohol, almost anything in which there could be an –

Annie: Exactly.

Nico: adversed… adverse effect.

Annie: Well, look I can take that, I know a guy and he drove too fast and then he got in accident and is… he got hurt and then he went bankrupt and his children were sad. We can… you can make those –

Nico: Well, that’s the risk of freedom, is that people have the… have personal responsibility and the option to make adverse choices and when you decide whether you’re gonna speed, you weigh the… the joy you get from driving down that road really fast and seeing the trees go by you quickly versus the… the increased likelihood that you might get in an accident and hurt yourself. And people in a free society are free to make those decisions or get a ticket. I mean we place certain guardrails about it, but we don’t outlaw driving for example because –

Annie: Well, that’s what I’m saying, it’s sort of… it’s sort of the idea of outlawing driving, as opposed to having consequences for the behavior. So, I think that the problem is that for almost any activity that you can engage in, I can construct an argument that brings us to, and then your children were sad.

Nico: Yeah.

Annie: I mean… and I think that that’s a problem so I think that we really wanna constrain what we mean by direct harm. So, that… that’s just my opinion on betting markets and I think that the… the info… I agree that the information that you can get from that in terms of having… being able to make good choices about the information is really… I think it’s really important and I know that for myself, very often, because we all live in bubbles. As much as we try not to, we all live in bubbles; I’m always quite interested in when the market disagrees with me.

I mean I think that if you look at some of the most successful investors ever, they always start with that first question. When they have a really strong apartment, sorry apartments, yeah, I’m stuck on your… I’m stuck on your water heater –

Nico: They… the probably have a strong apartment too if they’re successful betters.

Annie: Right. When they have a –

Nico: Stable foundation.

Annie: when they have a really strong opinion, where they’re going against the market, before they… if they’re very successful, before they make that investment, they always check themselves and say, well wait a minute, why does the market disagree with me so much here? So, it’s such valuable information to have the market disagree with you. For you to think that something is gonna… supposed to have a particular input… impact and then it doesn’t, that’s really, really telling for you to understand that other people think differently than you do and go find out why because that’s gonna help you.

Nico: Well, that’s what they did in, what was the movie, The Big Short?

Annie: Yes.

Nico: That… I mean –

Annie: Exactly.

Nico: what… what they did is these guys were trying to figure out why this other investor was shorting the subprime mortgages so what they did is they went out to Las Vegas and started talking to people who were giving out these mortgages that then these investment companies and banking companies would package into these subprime mortgage things and… and they found that wow, there’s information that we weren’t aware of just sitting at our desks here in New York.

Annie: Right, it came from why… why does this person disagree with me?

Nico: Yeah.

Annie: And then you go find out why. Now, what most of our reactions to somebody disagreeing with us, is that guy’s an idiot. Now, if you say well, that... ha ha ha, how dumb, he’s doing this silly thing, you don’t ever go and explore it and then you miss the opportunity. And it could be an opportunity like they had, but mostly at the core of it, what it is, it’s an opportunity to… to calibrate your belief. It’s an opportunity to change your model of what the objective truth is because we all live in our heads, with our own experiences; we interact with the world in a particular way, the world interacts with us in a particular way; we’ve been exposed to certain information; we all have these beliefs really strongly lodged in our heads.

And what we’re looking at is the opportunity to be able to calibrate those beliefs; to make adjustments as to what we think is true and what we not so that we can develop the most accurate model of the objective truth. When we swat away that… that person who’s shorting the… the mortgage… the subprime mortgages, because that just sort of disagrees with us, we lose that opportunity in all sorts of different ways.

It… now that’s gonna realize in terms of what our net worth is, but at the base of it, is that we lose the opportunity to calibrate a belief and at the base of every single decision that we make, is a belief that informs it so if we aren’t paying really strong attention and us… saying that’s what really matters to us, not the money because the money is the result. But at the base of it; at the core of it, is what we believe.

Nico: Are we getting better at recognizing these cognitive distortions at trying to collect more information, not swatting away the ideas of people with whom we disagree? Because you talk in your book about the State Department’s Dissent Channel; you talk about CIA’s Red Team, which I guess was implicated in the Osama bin Laden raid, they helped kind of push that forward. We’re seeing a lot more research on it from people like yourself and Daniel Kahneman and what was… Gilbert, what’s his first name?

Annie: Dan Gilbert.

Nico: Dan Gilbert. Are we getting better at recognizing our –

Annie: Gary Klein does a lot of this stuff –

Nico: Yeah, and like creating these pods of people who help us –

Annie: Yeah, Phil Tetlock.

Nico: Yeah, –

Annie: I’m just gonna name all of them, yeah.

Nico: there’s… there’s just… there’s just so many more people who are saying... Cass Sunstein, who are saying… creating echo chambers, not going out and seeking additional –

Annie: Jay Van Bavel.

Nico: Yeah, it’s just… you can just… and yet you have a… you have a recommended reading at the end –

Annie: Yeah, I do.

Nico: of your book with a… with a lot of these people in there. Are we getting better at least, recognizing the benefits of it if… even if we as individuals, through evolutionary psychology, which I mean this is… this is a result of our evolution that we… we kind of seek confirmation bias; we are motivated reasoners as… as Jonathan Haidt talks about it; we… where the elephant is driving, we’re the man on top of the elephant. How… how… how is this all sorting out moving forward?

Annie: Oh gosh, I feel like it’s kind of like sort of good news, bad news. So, –

Nico: At least our institutions seem to be recognizing it; at least some of them… some of them.

Annie: Our institutions… our institutions do seem to be recognizing, but if we think about it, I mean John Stuart Mill was a really long time ago. He –

Nico: 1859, yeah.

Annie: he was… he was writing about this very issue. I mean one of the things that… I mean he’s… there’s a couple of things that I would pull and I’m gonna loosely quote; one… one is he who knows only his side of the argument, knows little of that, which I think is a… I think that is a quote that everybody should know because I think that that’s the thing that’s really true. But… but what that’s really getting down to and he has a much longer bit on this, and… and I would recommend people go read it –

Nico: And he also… I mean he –

Annie: on liberty.

Nico: he also… there’s so many arguments that he makes that can have some sort of connection as he best said, “Both teachers and learners go to sleep and there’s no enemy in the field.” I mean not having people out there that force you to justify your own position –

Annie: Exactly.

Nico: is to your own detriment.

Annie: And he makes –

Nico: Even though it feels better.

Annie: I think that this is… I think this is one of the most compelling things that he says is, even if you are certain, certain of the truth that you believe, that you still must engage with people who honestly believe the other side because otherwise it becomes stale. It just becomes a thing you believe, you have no idea why and in engaging with people who honestly believe the other side, one of two things happens. Either you at least understand why you believe what you do so that the truth is not atrophied. Or you adjust your position, isn’t that great?

I think that people are afraid of adjusting their position because in the moment they have to say, ooh I was wrong and that feels really horrible. But if we go back to this idea that our beliefs are at the base of every single decision that we make, isn’t that a great thing when you figure out that you’re supposed to calibrate a belief; when you figure out that you’re supposed to change your mind, even just a little bit, because that means that your decisions are gonna be better for it because the beliefs that are informing them are more accurate.

So, yeah, you might take a little pain in the moment, but think about the quality… the increase to the quality of all your decisions and we can think about, in terms of humanity, beliefs that we were 100% sure of. In terms of humanity; here’s one, the sun revolves around the earth. Isn’t it good that people genuinely believed something else and that that argument was created.

So… so, I do think that these ideas have been around for a long time, I think that when we look at our framers, when we look at the Federalist Papers, I think that a lot of these ideas are flow… woven into this idea of dissent and how do we create destruct… constructive dissent? How do we make sure that there isn’t one decider? That there is a marketplace of ideas that are clashing with each other and I think it’s built into the foundations of… of this country.

So, I… I do think these ideas have been around for a long time, I do think they’re being implemented in a more kind of thoughtful and structural way in some places like Red Teams in the CIA; the Dissent Channel in the State Department and I think that’s a good thing. I do see that the tribalism though, is the real bad news.

Nico: Yeah, you see that in more of the… more political institutions and even institutions that are just like one standard deviation away from politics are being infiltrated by politics.

Annie: Exactly so… so the problem with the tribal… the tribalism problem is that if you hear information that disagrees with you when you’re not in an open-minded stance –

Nico: Well, the definition of a tribe is almost by nature the fact that they exclude people from the other tribe –

Annie: Right, exactly.

Nico: and presumably the information that comes from the other tribe.

Annie: Well, it’s not just that they exclude the information that comes from the other side, it’s that if they are exposed to it, it’ll backfire and cause them to actually in… increase their own position, to entrench in their own position because a tribe kind of gives you four things. And I really recommend people go look at, for example, Jay Van Bavel’s work on this, which is… he’s out of NYU, he… he’s done some really great work on this.

But basically tribe is giving you four things. One is belongingnees, that’s obvious, I belong to something; two is distinctiveness, that’s what you said, I’m different than those other people. But here are the two really bad things that it’s giving you; one is it’s telling you what is moral and what isn’t. So, that’s sort of like when you think about what Jon Haidt talks with like the sacrament and then you can’t –

Nico: Sacred values.

Annie: yes, exactly so you can’t violate the sacred values, that’s the… the morality piece. And then it also gives you epistemic closure, which is certain knowledge. Now, you can imagine, I’m like no, you should have a tribe that’s about epistemic openness, that… that… it’s like that strong convictions, loosely held, like we sort of wanna be those people, but the epistemic closure is like we… again, we’re not comfortable with uncertainty and one of the things the tribe is telling you is what is right and what is wrong.

Now, within the tribe… so what that means is there are trusted sources within your tribe who tell you these things; who say, these are the sacred values, this is what is moral and what is not; and this is what is true and what is not. And… and take this as a matter of faith, do not question; these are certain truths. So, if I see another –

Nico: And evolutionary, we’re plying for that.

Annie: Right, because we did better when we banded together as tribes because we don’t have big claws and huge teeth and the ability to fight off predators one-on-one.

Nico: Well, that’s why I think sort of our not being engaged in any big wars at this moment, is making us focus more on politics because it’s where we can go and find our tribe and the bowling alone phenomenon, we don’t have these civic institutions any more; religion is on the… the… the downward spiral, it’s just we don’t have anything outside of politics it seems, anymore to… to cling ourselves to and our… evolutionary we need sort of belonging in something and right now people are gravitating –

Annie: In something and it’s toward politics, exactly.

Nico: Yes, and politics… if politics poisons everything that it touches, boy –

Annie: And I… and I think if you view political tribes through the lens of religion, as… as you just said, I think that you get a good view into what that looks like. I think it is becoming very much like a religion, where there is the sacrament and there is things that are true and you take things on faith. And it’s us versus them and I think that there’s a lot of kind of overlap in the… in the way that you… in the way that those things are sort of playing into our tribal nature and our belief systems. I mean a lot of it is that the universe is a really big unknown and… and there‘s all sorts of stuff that we don’t know about it.

We don’t really understand our origins; we don’t understand what happens after we’re gone and looking for closure on that… looking for us to tell… people to tell us what’s true and what’s not. And the other thing is that we don’t really know what’s right and wrong because it’s hard; there’s a lot of uncertainty –

Nico: Well, many of us think we know what’s right and wrong.

Annie: What’s right and wrong.

Nico: I mean that’s… that’s one of the arguments I make about free speech that under girds censorship, it’s what John Stuart Mill talked about, is the idea that the censor thinks they are infallible, it’s that their certainly is the same thing as absolute certainly, it’s the… we’re very… it’s like I… I heard someone say this once, it’s like I always think I’m right, but I don’t think I’m always right.

Annie: Right, which is… that’s… that’s again, that’s like strong convictions, loosely held.

Nico: Yes, exactly.

Annie: That’s a nice way to put it. So, yeah I mean because look I can get into… in terms of what’s right and wrong, is it right to kill somebody? Everybody says, no, definitively. Well, what if you’re in a war?

Nico: Yeah.

Annie: What if someone is trying to kill you? Is it right to steal? No. Well, what if you’re grandmother’s dying and the only way for you to get a medicine to save her is to go steal it from a pharmacy? Then is it okay? So, –

Nico: Yeah, it’s the classic philosophical question from –

Annie: Right, there’s the –

Nico: philosophy 101.

Annie: trolley, do you pull the lever on the trolley? Are you… who are you saving, who are you not saving? These are activity incredibly hard questions. We don’t deal with this kind of uncertainty well, we want closure on this issues and this is what tribe offers us. So… so, the answer is yes, I think that we understand a lot more about it, I think that there are places where you’re seeing it implemented a lot better. But what I see is, through this tribal problem, that there’s a lot of slippage now. I think in a lot of ways that we’re going backwards.

And unless we can somehow find our way to a definition of tribe that I think we used to have as Americans, then I think that we’re gonna be in trouble. It’s… it’s kind of interesting because I think that one of the things the the Cold War did for us –

Nico: It brought us together.

Annie: Yeah.

Nico: It brought us together, I was… I was having this very kind of cynical conversation with my parents recently, and I said I think the only thing that can bring us together is a common enemy or even a war. And you don’t even… you don’t wanna hope for that and I’m certainly not hoping for that, but I very strongly recall what happened after 9/11 and how my, at least my community in suburban Chicago, banded together in a way that I still remember to this day.

Annie: Because you… you all of a sudden defined yourself as American. Not as Republican, Democrat, whatever; it’s like, you were American. So, I’m… look I’m not wishing for the Cold War, that was a scary time, but you can sort of see how it redefined the tribe that you belonged to so now what happens is that we view a common enemy as people who live within the same country as we do. So, Republicans now have a common enemy, which is Democrats; Democrats have a common enemy, which is Republicans.

Even within those parties, you see this sort of common enemy kind of happening where –

Nico: And this is where Jon Haidt and Greg Lukianoff in their most recent book; this is one of their three great untruths, that the world is divided into good and evil people.

Annie: Right, exactly and… and when you start to make your decisions based on common enemy, as opposed to common good, I think that that’s where we can really get into some big trouble. So, unless we can somehow get some sort or realignment or shifting, I’m not… I… I have… again, I’m… I’m optimistic in some ways, I’m really pessimistic in other ways.

Nico: Well, we need… you… you had mentioned the founding of America; it was the first nation that was ever really founded on ideas. And there is the idea of the individual and… and I think at least –

Annie: As opposed to like the divine power of somebody.

Nico: Yeah. And I think you can, it is possible, it’s probably very difficult to create a culture in which we are skeptical of ourselves. You talk about this… the psychology here in which it feels twice as bad to make a bad decision, than it does to make a good decision. Like it’s more powerful emotionally to make a bad decision than a good decision so we –

Annie: Or to feel like you made a bad decision –

Nico: Yeah.

Annie: than a good decision, yeah.

Nico: But we need to create a culture in which when we recognize that we make bad decisions, we don’t take that to heart as much so that it allows us to analyze our decisions more analytically or rationally. I… I think you could potentially create a culture on that and I –

Annie: I… I think that you can, I think it’s about what are you rewarding?

Nico: I mean it is… it is kinda self-help actually when you talk about what we’re –

Annie: Well, we can go back to your –

Nico: we’re rewarding ourselves when we do it that way.

Annie: Yeah, so let’s talk about your water heater.

Nico: Yes, let’s –

Annie: Right.

Nico: that’s all I’ve been talking –

Annie: Well, it’s… it’s the most important thing that I heard today.

Nico: Yeah, yeah seriously, at least financially.

Annie: But we can take… you… we can take your water heater; it’s like you can… and… and it depends on what you’re being rewarded for. You can take this thing that happened and you can sit there and you can beat yourself up about it and you can ruminate about it and you can say why was I so stupid and then you can sort of like catastro… why did I even buy this house, this was a stupid decision. You can make it a much bigger thing and generalize and all this stuff, or you can say –

Nico: Or I could learn from it.

Annie: oh my gosh, I’m so lucky that this happened to me because now I know that this is a thing that I have to worry about for the rest of my life and now, not only do I know this, but when my friends are buying a place, I can warn them if you’re not moving in for a while, not only make sure that you drain the water heater, but I was lucky –

Nico: Turn that fuse box –

Annie: turn that fuse box off. So, now you’re gonna be able to spread that knowledge to other people because you now have it and isn’t that gonna create so much good for the world?

Nico: Yeah.

Annie: So, it’s about how you frame it and this is one of the things –

Nico: And it eliminates risks for many people in my lie… life.

Annie: Exactly.

Nico: That’s why free speech… I mean we’re… not in the water heater situation, that would never be controversial, no one would ever ask for censorship, but bigger questions, having that information out there. Does the… is the emperor wearing clothes or not wearing clothes?

Annie: Exactly. And so… so I’m a big fan of saying, look we have the mindware that we have and I’m not gonna be able to say, well, I don’t do confirmation bias and I’m completely open-minded and I have no tribe and I… because I mean, these are things that are really built into us so I think it’s a matter of… of kind of, with real intention saying, how can I sort of take what these tendencies are and use them in order to create a reward system that actually rewards things that… that align with sort of reaching my long-term goals?

So, if I understand that my long-term goals are to have pretty good outcomes in my life and to increase the probability of those and have beliefs that are generally true and to be open-minded and all of these things? I can now band together with some other people and say we’re gonna make a different type of agreement. So, –

Nico: And that’s what you did in poker?

Annie: Right. When you come to me and you say, hey, I had this really bad reaction to my water heater breaking and I’m really beating myself up about it and I think that maybe it was my fault, let me tell you the decision, let’s talk about it. Now, I’m offering up my opinion to you and if one of my opinions is, whoa, actually you really could have known about that if you’d looked on any site about whatever; then you’re not gonna be sad, you’re gonna say thank you… thank you for letting me know that this is something that I could have found out beforehand because now I know that it’s available and I can go look at these things.

Or I can say, no you don’t know, but isn’t it great –

Nico: Yeah, that you do know –

Annie: that you know now because… because now you can go change. And what we’re doing is we’re changing what the interaction is. If we think about echo chambers, it’s like –

Nico: This is cognitive behavioral therapy. I’m thinking about that a lot with Greg and Jon’s book, –

Annie: Right, exactly.

Nico: Coddling of the American Mind, it’s like looking at our decisions rationally, not engaging in black and white thinking; engaging in catastrophizing; engaging in… there’s… I mean there’s a number of cognitive distortions –

Annie: Exactly.

Nico: that you could engage in.

Annie: Because here… yeah –

Nico: And are very easy for us to engage in.

Annie: here’s how the normal interaction goes, I read this opinion piece in The New York Times, I think it’s awesome, what do you think? And you’re like, it’s awesome and we’re like great, we love each other. But if you said, actually no, I actually really deeply disagreed with that. I’m like, oh, different tribe so that’s kind of how it normally goes and so what happens is, we tend to form these groups that are all like, we agree, we agree, we’re all awesome, we’re so great. And the problem with that is that even if within that group, where there’s all this natural agreement, you think that you’re sort of like, well what would the other side say?

It’s like you’re really… you tend to be just creating straw mans left and right and then what you’re really doing is patting yourselves on the back for being so good at arguing against the straw man. What you wanna do is create a group that naturally creates sort of steel man arguments that you must defend against. Sort of to that point of Stu… John Stuart Mill, you’ve gotta be arguing against people who generally believe something that is different than what you believe.

You have to be seeking that information out and you can create a group that rewards that, where when you come to me and say, you know what, I really believe this thing, but now I read this other piece from someone who I think is really an expert and I think I’ve changed my mind, let’s talk about it. And then I can be like, that’s amazing and you can say I think I made a really big mistake; that’s so great that you recognize that; that’s awesome, what was it so I can learn from it.

And notice that what’s happening is you’re getting all the same social reinforcement. You’re getting the, I like you, I’m getting to interact with people whose opinion I respect and they’re… they’re reinforcing me in a way that actually is creating that kind of open-mindedness; it’s like we’re redefining tribe. Our tribe is now like the mistake admitors, like the belief calibrators, the opposite opinion seekers, the we wanna find out everything we know about why we’re wrong people. And that’s now what’s being reinforced and if you came to me and you said, oh, I was… I read this thing and that person’s just an idiot, I’d be like really, what?

And now you’re not getting reinforced for that kind of behavior. So, I do think that with intention, we can change at least locally within our lives; we can change what the social contract is. Now, what’s great is that if you have a lot of people doing that in a local way, guess what happens? It spreads.

Nico: It spreads, yeah. It has that contagion effect. They –

Annie: Hopefully. A good kind of contagion.

Nico: So, before… there was… there’s the… the free speech discussion that we could continue to have, it could go forever, I mean I… I was also… I wasn’t familiar with this book before I read it; you mentioned it in your book, which is Samuel Arbesman’s –

Annie: Arbesman –

Nico: the Half-Life of Facts.

Annie: Half-Life of Facts. It’s so great. So, –

Nico: It’s… he argues that practically every fact has been subject to revision or re… or reversal or –

Annie: Yeah, so he… here’s some –

Nico: or rehearsal or something.

Annie: examples, like we… there was a very long time that we thought that Coelacanths were extinct and they’re not. And the number of spices that have become unextinct… there’s so huge and you can think about simple things, like the sun revolves around the earth; the earth is flat, although that… some people still believe that, but –

Nico: Or that fats are just the enemy.

Annie: Fats are the enemy; sugar who cares?

Nico: Although that could presumably be subject to reversal as well.

Annie: That… that might be subject to reversal –

Nico: But that’s why we need to keep flooding the information in.

Annie: like right now, there’s… there’s stuff going around about statins, like are statins… the idea was that because people have heart disease, have high cholesterol, people were… there was an assumption that high cholesterol causes heart disease. So, if you lower the cholesterol, then clearly you would lower the heart disease. But it may be that the high cholesterol and the heart disease are outcomes of something else so we don’t know. I mean the thing is that we’re revising these things all the time and he’s really arguing that you have to hold these things loosely.

And you can think about this in your own life, this is question that I ask people all the time, is there something you believed like super strongly; that you were like totally sure of when you were 20, that you don’t believe now?

Nico: Oh, I’m sure.

Annie: Yes.

Nico: I don’t wanna sit here for like five minutes and think about it, but oh yeah, I’m sure.

Annie: Yeah, the list would be long so… so that’s what he’s saying about The Half-Life of Facts, it’s like what’s happening sort of in terms of humanity, and understanding that these… there’s this ever evolving knowledge as we start to understand that. So, think about all the… the information that we had before we could see an atom and then think how you could see things that were... before you could see things that were subatomic, for example.

In our own lives, that’s true as well; there’s so many things I believed when I was 20, like all of them, that are just… I look back on that, I’m like what was I thinking? So, why do I think that the things that I believe today are any different? That’s what you have to understand; it’s not different today just because some time has passed.

Nico: Yeah, we shouldn’t… to echo Francis Fukuyama, believe that we’ve reached the end of history –

Annie: No.

Nico: that… with our knowledge –

Annie: Right, or our own history.

Nico: it’s… yeah.

Annie: I mean we have to think about that in terms of humanity, but then also individually as well.

Nico: All right, well, Annie, thank you so much for coming on the show and I hope to have you on again sometime.

Annie: Oh, well thank you, this was a lot of fun.