Freakonomics Radio - 687. Are Prediction Markets the Best Forecasting Tool Ever — or Just Another Casino?

0:06Hey there, it's Stephen Dovner. Before we start today's episode, I want to ask for your help with a future episode about psychotherapy. The episode includes what is called a misery index, and we want to hear some of your stories. I realize this isn't for everyone, but if you are interested, use your phone to record a voice memo and send it to us at radioatfreconomics.com. I'd like you to identify your most intense and persistent form of misery and include some specifics.

0:35What exactly was the emotion you were feeling? What's the worst it ever got? How did it affect you? And how did you deal with it? If you only want to include your first name, that's fine. Again, the address is radio at Freakonomics.com. Many thanks. And here now is today's episode. Most of us are not very comfortable with uncertainty. And that's a problem since so many things are uncertain, like the future.

1:09Since the beginning of time, we've been trying to predict the future. The ancient Greeks were famous for their oracles. They would also cut open a sheep and read its liver for signs about whether it was a good time to start a war, for instance. For centuries, potato farmers in the Andes have looked skyward to the Pleiades to help determine when to plant their potatoes. 25 years ago, scientists were finally able to explain why this actually worked. Pleiades visibility is related to the climate pattern known as El Nino. Today, there are prediction markets on El Nino and the markets indicate this year's El Nino may be the most extreme on record. So are prediction markets an upgrade or maybe a disaster waiting to happen? Will they improve our collective decision making or are they just another casino in an economy full of casinos? And aren't prediction markets susceptible to insider trading?

2:07The biggest prediction market in the US, Kalshi, recently imposed its first-ever lifetime ban against an individual trading on their site. This was over bets on whether George Santos, the disgraced New York congressman, would attend President Trump's State of the Union address. Santos had posted on social media that he was going to attend, but in the end, he didn't. On Kalshi, someone made a profit of $17,000 by first betting that Santos would attend and then betting that he wouldn't.

2:36who was at someone, you guessed it, George Santos. So how much credit should Kalshee get for figuring that out and banning Santos? Today on Freakinomics Radio, we will hear from the CEO of Kalshee. Kalshee is really the most accurate way to predict the future. As well as their chief contract writer. These are legal contracts that people are signing on to when they're trading those markets. We'll also hear from an intellectual godfather of prediction markets. The hope is that we could use betting markets as a general information institution all across society. And next week, in part two of this series, we will hear from a regulator. What I think is the cost and a very real serious cause is trust in the markets. Our two-part series, The Price of Prediction, starts now. This is Freakonomics Radio, the podcast that explores the hidden side of everything with your hosts, Stephen Dubner.

3:54two big prediction markets at the moment are together worth over $40 billion. Shane Copeland, who founded Polymarket in his early 20s, was for a time the world's youngest self-made billionaire. One Kalshi founder, Luana Lopez Lara, is the youngest self-made female billionaire. The other founder and current CEO, Tarik Mansour, joined the billionaire list at age 29. But that is not what Mansour wants to talk about.

4:23this valuation of the company or my net worth or others. Now, just don't think that's that important. It's kind of paper money. Okay. So what is important to talk about when it comes to Calci and other prediction markets? If you believe in markets and their ability to aggregate information and price something, in the case of prediction markets, they're essentially aggregating information about a question about the future. If you believe in the crowd wisdom and people kind of putting skin in the game on something, it tends to be more accurate than other places. For someone who doesn't know, Calci, just describe what the firm is. The easiest most compelling way to describe Calci, it's essentially a financial market that captures a much broader universe of things. Historically, financial markets have been sort of limited to an elite audience. Whether it's the stock market or commodities or interest rates, what we got excited about was broadening that universe to just no questions about whether an event is going to happen or not. It's about things that people care about.

5:18or relate to, they understand whether it's in politics, the economy, climate, weather, culture, you name it. Mansour was born in Bakersfield, California to Lebanese parents, but they moved back to Lebanon when he was young and they divorced when he was six. I grew up outside the system a bit. I mean, I was a math nerd, single mom. My dad is alive, but he was never really present. And then there was a lot of turbulence in Lebanon. A lot. Growing up, there's a mix of two things. There was one, my mom really had high expectations of us, like you should do something big in life, make it count, make it worth it. And then the second thing is we had a deep level of frustration with just the way that Lebanon worked, like the corruption. This is just like a bad country. What drives me is I just want to have some sort of legacy. Like I want to build something that people point to and respect and have sort of validation. You were born in a good generation. One generation earlier would have been okay, but like nerds won the war in a way. It's kind of amazing, honestly. The nerds are cool right now. It wasn't the case when we were growing up.

6:16It's like a 10-15 year phenomenon now. But 20 years ago, I don't think that was consensus yet. In 2014, Mansour left Lebanon to attend MIT. He studied math and computer science, and he soon landed internships and jobs with elite firms, Goldman Sachs, Palantir, and Citadel. When I got the idea, you probably know the story, but when I was in 2016 at Goldman, a lot of the trades was like, hey, we want to go long Trump or hedge against Trump winning the election. But they're very sloppy trades, yeah?

6:45There was very sloppy. We said, like, sure the S&P, that's the Trump trade. Bad trade. They were right about Trump winning, but then they lost money because the S&P actually rallied. So the use case is very clear. I just think it's opening up access for people with significantly broader or more diverse set of interests to have a shot that they don't have in traditional financial markets. It wasn't long before Mansour and his MIT classmate Lopez Lara started Kalshi. Where does the name come from? Kalshi means everything in Arabic.

7:14Back when we first started the company towards the end of 2018, we got into the startup accelerator, Y Combinator, and they needed like a name for the application. We were looking for like a cheap domain name and I kind of like names with K. I was like, what if we call it Kalshi? And then we're like, we'll change it later because it's like a bad name. It's hard to pronounce. But now it's a big brand, so it's harder to change. Many tech startups embrace the Facebook mantra of moving fast and breaking things. Let's not Kalshi. What we did.

7:44is the exact opposite. What Luan and I decided is we're going to abide by core principle in the company's regulatory first. And we spent four years getting regulated before we launched a single market. We went to the federal government and said, hey, how do you regulate this? And what does that mean? Two pillars. How do you build a market that has market integrity where fraud is police? There's no insider trading or your police. Number two is you enable the right set of customer protections.

8:06Kalshi opened to the public in 2021. Some of their first contracts were on mainstream current events. For instance, in a given week, how many Americans would get the COVID vaccine? Would the Tokyo Olympics be canceled? The hard part came when Kalshi wanted to offer election contracts. The CFTC, the Commodity Futures Trading Commission, said they couldn't. Kalshi sued.

8:30And the case was decided by a federal appeals court in favor of Kalshi in the fall of 2024, just in time for the presidential election. It had taken Kalshi a while to get there. Imagine the first four years of my career. I mean, it came at a great personal sacrifice. It really was very tough because regulation is not fun. It's not sexy. It's super boring. It's like exhausting sometimes. Did you come close to quitting a few times? I mean, daily, pretty much.

8:55But the thing that I just really wanted to see exist in the world, I just felt like the cost of regret would be too great. There's a little bit of sunk cost fallacy when you're like two years deep. You're like, I'll try another few months and then it keeps going. What were your advisors saying? People always trusted our approach because we were pretty dogmatic, but it's an anti-pattern. The pattern in Silicon Valley is you build and you move fast and you build product that customers love.

9:15Did you have potential funders tell you like, Tarik, that's a really nice idea. But there are 50 people who are capable of doing something like this, and they're not going to go the slow, legit route. Yeah. I mean, Polymarket was launched at the time. That's why people know about the brand of Folly first. That's not because they started the company first. We started first. They started after us, but they took the idea and they were like, hey, we'll launch it offshore. Why do we wait? That distinction is Really a large part for why we're 90% market share today, like why we've grown so much is because we stayed committed to that approach and I think regulation has given us a huge edge because people trust it more and I think we can go mainstream, institutions are onboarding at a higher rate because it's harder to do it from outside the system. You have to really change the system, which is harder.

9:57I do feel like many of us, even in the political realm or maybe especially in the political realm, but even in the financial realm, a lot of decisions are made with some insight and some information, but an awful lot of guesswork that we then convince ourselves is empirical somehow. Totally. I'm just curious whether you think that Kalshi ultimately or even now serves a bigger purpose of improving decision-making. It's not like we're making bad judgment calls based on the information we have. It's just that we have a very limited set of information. That's the key thing. During World War II, in the comics, Frederick Hayek dug about the knowledge problem. It was this very basic idea, which is that a lot of decisions are centralized. It's centralized authority figures, whether it's governments or heads of households or leaders and companies and so on and so forth. But the information that is relevant to that decision is actually pretty distributed. If you think about this as that tree, a lot of decisions are made at the root of the tree,

10:53But the information is lying in the nodes of the tree. Distributed makes it sound to me like distributed among many people. The word I would think of is more like siloed or hidden. Are we talking about the same thing or no? It's all of them. It's actually distributed. It's siloed. It's hidden. It's fragmented even. So sometimes you may have a piece of information that standalone doesn't make much sense. You have to combine it with someone else's piece of information and all of a sudden it could click. And then it's also dynamic. These nodes.

11:23in the network or the tree, the information is updating in real time. You could probe it, you could ask the node yesterday, but today it might have a different answer. At the time, Hayek didn't call them prediction market, but he's like, well, the best way to solve that problem is probably some version of market prices, because that's what they do. Market prices in traditional financial markets, they sort of aggregate information that could be distributed. This was the idea of this information market. Use market prices to disseminate and propagate information.

11:51What form did that take for high activity? Try to do something like this? It was very theoretical. In my opinion, the first test of that theory was in the 80s with the University of Iowa. Are you familiar with that experiment? Yes. These are the Iowa electronic markets. Exactly. And they ran a small scale prediction market. A few hundred people doing very limited sums. Real money? Real money. Yes. Yes. Real money is absolutely key. There needs to be skin in the game. You have to be punished if you lose and you have to be rewarded if you win. There's no better reward.

12:19than making money and no better punishment than losing money. The Iowa electronic markets, originally called the Iowa political stock market, was focused on elections. The CFTC had allowed it on the condition that it remained an academic experiment and traders were capped at $500 each. So how did it do when it came to predicting? One study compared its predictions to a large group of national polls and found that the Iowa markets beat the polls 74% of the time. And by now, there were other people starting to think hard about prediction markets. I had this idea of a much wider application of betting markets in the late 1980s. That is Robin Hansen. He is an economics professor at George Mason University. And it's always been in the background as something I was willing and eager to do if there were people to do it with. But there have been long stretches where

13:19There hasn't been that much interest. Do you feel you've been sort of wandering in the wilderness and all of a sudden there is a city on the hill and you're invited and everybody cares? Well, there's a path to the city. We're not at the city yet. But maybe I can see a route that might go up the mountain and we're starting up the route. What does the city look like? The hope is that we could use betting markets as a general information institution all across society. The main reason for that hope is when we do head-to-head pairwise comparisons of betting markets and some other institution at the same time, same topic, similar resources, the markets just do about as well or substantially better in terms of accuracy and similar cost. That's just a remarkable fact. We don't actually use speculative markets very much in our world in academia or business or journalism or nonprofits. It just seems like there's this huge opening to do much better.

14:16So many basic questions I have for you based on just that good statement. First of all, why do they do better? I'm an economics professor. So we have a lot of things we think we understand about this. But then most people aren't that inclined to believe economics professors about such things. So I'm mostly going to rely on the data and just say, look, the data says it does work better. But if you want reasons, I can give them to you. Yes, please. Let's start with the comparison of you, a reporter interviewing me a.

14:45non-reporter about something. I don't necessarily have the incentive to tell you the truth or to work hard to tell you the truth. I have an incentive to be engaging, entertaining, dramatic, you know, tell you what you want to hear. Maybe embellish your own stature. Not you, but others might. Right. So a prediction market in contrast just gives you a very clear, sharp incentive to get it right.

15:08But secondly, if you ask me about something I don't actually know that much about, I'll still give you answers, because I want to talk to you, the reporter, and get in your piece. Whereas the speculative markets give you an incentive to just shut up and don't speak about things you don't know very much about. You're enticed to go look at all the markets and ask, which of these markets do you know more about and only speak up about those? There's this old saying in poker, when you sit down to a poker table, look around and find the fool.

15:36That's who you'll be making your money off of. If you don't see the fool, it's you walk away. And most markets, you'd be the fool. So you should not trade most markets. You should just leave them alone. That incentive to select is a second powerful thing. In addition to having an incentive, you just only have the feedback from people who think they're world-class about something. Number three is that if you ask me a question, I would just give you the direct answer of what I think on the subject. But With the markets, there's always an existing set of market prices on all these different topics. You're invited to not have a direct opinion on these topics, but go look for biases, look for patterns, look for errors in these prices. And if you can find any pattern that looks like it's a mistake, you are paid to fix that. Robin Hansen sees the true potential of prediction markets in their ability to inform decisions at scale. When most people get into this topic,

16:40The thing they think they want the markets to be about are the usual topics in the media and public conversations. And that's what I thought initially too. And then after a while, I learned decision theory as a grad student and realized that according to our standard decision theory, information is valuable because it advises decisions. There's a world of people making decisions out there and that's an enormous potential demand for information. That's where I have since put my hopes.

17:08Hansen laid out this vision in a 1999 paper. He proposed a new form of governance for organizations and the broader political system using a form of prediction markets he calls decision markets. So this is my grand vision. Advice all around for what to do, for example, a firm could have a market on the stock price of the firm conditional on the CEO leaving or the CEO staying by the end of the quarter.

17:37which would be advice about whether the CEO should leave or stay. Those prices would say which scenario is worth more to the company. You could also do other major decisions of a firm, restructuring mergers, acquisitions, introduction of new products, nonprofits could do this, political parties, governments, and you're even personal, like a student deciding which college to go to or what major.

18:02could have a market in the consequences for their life. You could have a market and if you dated someone, how long would that relationship last? In a case like dating, where is the information coming from though? Who's betting in that? And how do they know what they're talking about? The wonderful thing about these markets is you don't have to decide who knows best. You just have to make sure that whoever might know is invited to participate. There's a lot of people around you who have seen you date for a bit and they have some opinions.

18:29friends, family, maybe friends of the person you're dating and so on. Yeah. Or people you've dated before. Well, have good advice there. Right. Okay. But if you're talking about a group of people with perhaps warped incentives, I would think the people you formerly dated, maybe your own family, et cetera, on what dimensions would those predictions be valuable? Well, those people do have information. And the question is, can we elicit their information without it being distorted by their other interests involved? And the Answer is actually yes. All information institutions, including journalism, have this problem that people who have access to grind might try to distort your output. If you interview someone who wants the world to think a certain way about a topic, they may give you distorted testimony, right? But betting markets are remarkably resistant to that sort of influence. That's one of their strengths. In fact, on average,

19:23When you add traders to a market who are trying to manipulate it who are trying to distort the price If other people expect those traders to show up there overall the price gets more accurate That's a remarkable fact about these when I've heard you speak in the past about the value of let's say an internal prediction market in a firm this was I don't know, 10, 15, 20 years ago, I thought, oh my goodness, A, great idea. B, there's no way this idea will not take over the world. Eli Lilly ran an internal prediction market in the early 2000s that correctly identified drug compounds that would survive phase three. There was a Google project called Google Profit, P-R-O-P-H-I-T. And from what I could see, they were both successful, but they didn't last. And indeed,

20:12firms around the world didn't rush to create internal prediction markets. Can you explain why? That city on the hill we were talking about, that's past the jungle or the forest on the hill where we try to get past corporate politics. If we think about the example of a deadline, deadlines have been one of the earliest applications in corporations because they're just really simple. You have a date when you're supposed to deliver something and the question is, do you? And obviously, We often fail to make deadlines, so this is a live realistic question. Will we make this deadline? Now, these markets we've created on deadlines, they have consistently been accurate. However, people who run projects don't want the markets. If you run a project, you want to know if you'll make the deadline, but you more want to have a good excuse if you fail. And everyone's favorite excuse if they fail is the following. We were going along just fine. Everybody thought we were going to make it.

21:09And then at the last minute, something weird came out of left field, knocked the project flat, and that's why we didn't make the deadline. Tarik Mansour, the CEO of Kalshee, calls Robin Hansen's work on prediction markets foundational, but he especially likes to cite the work of Philip Tetlock, a University of Pennsylvania psychologist who you may have heard on this show in the past talking about the folly of prediction. Tetlock argued in his book, Super Forecasting, that when it comes to predicting the future, even an expert can be beat by a thoughtful amateur. Here's Mansour again.

22:12So in your view, how do you read that research? Why did the experts underperform? There's variety of theories, but like people that have domain expertise and something over time, they become a little too dogmatic about that thing. Whereas the people that like can take a step back and look at it more dispassionately, tend to self-calibrate better. And, you know, they have less bias. This is the key thing. It kind of means that domain expertise is not one of the most important things for being a good predictor of the future. That's a fairly heretical thing to say in some circles.

22:40Imagine saying that to politicians or researchers, financial firms, analysts. This notion that actually, if you want to get the right answer, you should get a very diverse set of participants who pride themselves in self-calibration, thinking critically about the word, reading beyond the headlines, having this idea of being able to have a clean filter on everything they read. That's more important than actually the expertise in the specific domain. That's foundational to what we do. Coming up after the break, how does a Kelsey contract get written?

23:12I'm Stephen Dubner. This is Freakonomics Radio. We will be right back. Now I would like you to meet Nicole Kagan. We have the most robust data set that has ever existed on transaction level data in the US on a federally regulated exchange. What Kagan is talking about is at least the most robust, publicly available set of transaction level data. And how does she know this? I'm the head of research at Calci, and I also write the contracts and self-certify them with the CFTC.

23:51She joined Calci in April of 2025. At the time that I joined the exchange was transacting, I think about $300 million of volume a month. We're now doing about $14 billion of volume a month. Five years ago, the job that Kagan does today did not exist. So how did she get here? After I graduated from college, I worked for a hedge fund for a year. I went to graduate school at Oxford. Every person that I went to college with wanted to be a professor of economics, so I wanted to be a professor of economics. Are you serious? What were they thinking? I have no idea.

24:21But I decided after doing all the PhD coursework and none of the dissertation that is actually probably the more interesting piece of doing a PhD that I wasn't so interested because I think that there's a lot of real world data that you don't have access to when you're an academic, that you can actually just play around with when you're in industry. How do you see what you're doing now filling that gap? It's massively useful because we're making the data available to researchers for the first time. Drill down the what kinds of data?

24:47So we have data on, for example, how people are trading, how users trade through time and move through different markets, how they enter the platform, where they exit the platform. We have micro data on individuals because we have KYC. KYC stands for Know Your Customer. That's an anti-fraud safeguard used by banks and other financial firms. So we have data on where people are based.

25:08For example, where the state of registration is in the lead up to the next presidential election will be really interesting to see if we're seeing different trading activity in swing states on swing state related markets. Your main function is to write a contract, correct? Exactly right. We create contracts that are robust, legally robust, and then we offer a platform for the exchange of the capital on either side. It would be incredibly useful to hear how a contract gets.

25:36written from idea to getting live on the Calci market. The first stage is having the idea. We then think about what the economic justification for listing a market on that idea is. Let's take a really simple contract on economic statistics. For instance, what will the Federal Reserve set interest rates at next month? That's a pretty important thing. At that point, we consider whether we already have a contract certification for it or whether we need to write a new contract.

26:05We have many submissions that we've already filed, in fact, thousands with the CFTC that allow us to list certain markets. They're written pretty generically. They allow us to switch out certain words and then list those markets. If we don't have that contract, we'll create a new contract. A new contract will take the form of something like, will econ stat be value in time period? Well, we have these three variables that we're setting.

26:29Do you ever have good ideas for a contract but decide not to go forward because you think it'll get hung up for six months or something getting CFTC approval? Generally speaking, when we're talking about the self-certification process, we get these submitted to the CFTC in the morning. The CFTC holds them until the afternoon, and then we're allowed to list them unless they've intervened to tell us that we can't. What are some markets or contracts that you don't allow and why?

26:56Under the Commodities Exchange Act section, I think 4011, there are six categories that are subject to restriction. These are markets that we do not offer. These are markets on terrorism, assassination, war, gaming, any activity that's unlawful under state or federal law, and anything that is contrary to the public interest as defined by and interpreted by the CFTC. If 4011 did not exist, would you offer all those contracts? No, I don't think we would. Because why not?

27:26We think a lot about the incentives surrounding the markets that we're creating. So we would never, for example, want to induce an actor to adversely impact some outcome or some other person or to cause harm because we have a market that is live. That's something actually we take very seriously. We have specific rulebook provisions that carve out what we do in cases of death. We might have markets on what somebody might say in a speech or whether somebody might attend an event or what somebody might say at that event.

27:53There are certain cases in which we might want to invoke rules around what we want to do if that person is subject to violence or death that prevent people from cashing out at a dollar or zero. And in that case, you just refund all money. As an exchange, it's difficult for us to void transactions. What we'll do is we might resolve to say the last fair price or refund if it's a possibility for us. The last time we were actually asked to delist a market could have been the elections.

28:20when we were going back and forth in the regulatory process. This was prior to the 2024 authorization to list them. This is when you were asking forgiveness rather than permission phase. We were very much still asking permission, but I think the permission was a little bit piecemeal. That was a time that we did actively delist markets. Usually it is not the CFTC that will ask us to delist a market. It'll be us internally finding a market that we might not want to list after the assassination of Charlie Kirk.

28:48any market related to Charlie Kirk had to be altered or taken down or settled. There are times where we might intervene to deal with markets where there are events that have happened that would compromise the integrity of those markets. Writing the rules is one of the most interesting jobs at Cal State and I don't just say that because that's my job. But there are a number of things that you need to think about. You need to think about what is the underlying.

29:07that you want to refer to, what is the actual topic that you're going to resolve this on the basis of? For something like the federal funds rate or for something like inflation, for example, what is the actual number that you're going to resolve that market off of? Then you have to think of who is going to provide you with that number. The gold tier there is going to be the original statistical agency that produces that number, ideally a government or federal agency that produces that number.

29:31That's, I assume, changed a good bit during the second Trump administration. Yes, there's just been different reporting from government agencies. It depends on the market. By and large, our contracts haven't changed because they tend to have many source agencies listed. Then we have to think about what the variables are and how we define them. Let's say we're talking about the Fed funds rate or we're talking about inflation. We need to make it generic enough that we're able to use that variable to include either of those things. So we might say something like, it's an economic statistic as specified by the exchange.

30:01It can be as broad as that. And then we'll get to the actual payout criterion. The payout criterion defines when we would resolve a market to yes, when we would resolve a market to no, and when we might resolve a market to neither yes nor no. In these cases, we'll write something that says if the print is value or if econ statistic is value, then the market will resolve to yes. That's a crude simplification of the way that we might write the first line of these payout criterion.

30:26But then after that, we have to think about edge cases that might happen. What happens if you're talking about the Fed funds rate and the meeting is cancelled? Or it's delayed past a certain time period? Or something happens that interrupts or disrupts that meeting? Or what happens if in fact there are three numbers that are produced instead of one or two? This actually became really relevant last year because when the government shut down, there was no publication of inflation for the month of October.

30:52And that is effectively unheard of in the macro space. This sent hedge funds into a bit of a frenzy at the time, but we had a contract on what will the rate be in October. And there was no rate in October because it was never published. What happened in that case? We had our own formula that interpolated from past months what the figure would be. We made that clear. That was a clear part of the full rules. And then we added a green box to the page to clarify to users how we were going to be resolving those markets.

31:21I'm guessing some users disputed that reckoning. I'm sure that there were people that were more or less happy with that conceptually, but we do need to weigh what we do in these kinds of extreme situations because we can't just throw up our hands and say, well, we don't know what the rate is. We need to create clear path to resolution in every edge case that we can imagine exists.

31:40This is true of this economic contract that we're discussing, but it's also true of things like elections where you might have questions about what happens if it's challenged, what happens if it's overturned, what happens if there's a coup, what happens if they're never inaugurated, what happens if something happens to them, what happens if they change their name? Is it still the same person that we're talking about? What happens if both candidates have the same name? That's pretty confusing.

31:59How long does that process take for you and your team to create a contract that is able to include all those variables and potential edge cases? It sounds like it might take a year to write one contract, but plainly that's not the way you want your business to run. No. And in fact, it's not the way the business runs. It depends on the complexity of the contract. If it's an area that we understand well, it could probably take as little as a few hours to a day because we're pretty used to it. What's the most complicated or longest contract you've...

32:25We have markets on what people will say in a given speech. We recently rewrote the rules for those. Because why? There were lots of questions that came up about different grammatical features that might come up. What if there's an apostrophe and what they say, what if they misspeak? What if they pronounce a word incorrectly? Is that still the same word? Is that a different word? What happens if it's in a proper noun? What happens if it's in a live stream and then the live stream goes dark and then it's uploaded somewhere else later? It took us over a month.

32:53to write this new set of rules for our mentions markets. They're seven or eight pages long. They are unbelievably detailed. There was an article that came out, I think Bloomberg published it. They were trying to get at why we wrote rules that were so detailed. The conclusion that they came to was it's for automated processing and it must be AI. It's actually got nothing to do with that at all. We wrote them for user clarity and didn't even consider the fact that if it's clear to a person, it's probably clear to a computer too. Do you at least run your contracts through an AI?

33:21As a stage of the process, we will usually run them through some kind of LLM to see if there's anything that we've missed explicitly or anything that is a logical contradiction that we haven't caught. But it's not the primary thing that we rely on when we're writing these contracts. Let me ask you about one more edge study. Let's say who will perform during a Super Bowl halftime show, especially if it's maybe Cardi B, who's not the headline performer, but maybe a performer, maybe not. And you're going to ask this question.

33:49The rules for this contract were not particularly unclear. These contracts basically said, if the person is dancing and singing, then they're performing. What you saw in the video in the probably seven second clip in which she appeared is that she's definitely dancing. But you absolutely cannot tell whether or not she's singing. And she wasn't maked. Exactly. But it kind of looks like her lips are moving. From our team's perspective, we said, well, okay, she's clearly dancing. We can't tell if she's singing.

34:14We don't feel comfortable resolving this to yes or no, because we can't tell. We're epistemically a little bit uncertain about this. What we're just going to say is we're going to resolve this to a last fair price. We do have the power to do that in cases where it is genuinely unclear whether or not an event met the resolution criteria that we had set. What does that mean to resolve it to the last fair price? Usually, our contracts are binary. The way that the prediction market works is that there's a question that's asked, and at the end, if the event happened, you get paid out $1, and if it didn't happen, $0.

34:43Every position is what we would call fully collateralized, which means that if there's somebody that puts up 30 cents on one side in order for that actual exchange transaction to go through, there needs to be somebody who puts up 70 cents on the other side to make a dollar. Sometimes there are cases where we need to settle to a value that isn't zero or one. And this is what we might invoke to be like a last fair price, or it could be at times 50-50. A good example of 50-50 is if multiple people win an award.

35:08We might say that because no singular person won outright, but two people won, they'll each result a 50 cents. Will you rate the Super Bowl performance contract differently next year? We've actually already amended that contract. Basically, we wrote a longer contract that was a little bit clearer about what we would define as a performance and not. Okay. So under the new contract, would Cardi B have performed or not performed? The answer is no. You now have to be singing, including audible lead or backing vocals.

35:35I feel like there's a fundamental difference between people who believe in prediction markets and then a lot of the rest of the world who see primarily a betting market and don't see it as a useful tool for actually surfacing good information or for judging the value of information. So I'm asking you, I mean, this is as softball as it gets. I'm asking you to give the best evidence or argument you can for why what you're doing is actually valuable in a pro-social way.

36:04There was research that came out of the Federal Reserve a few months ago with some researchers that looked at our macroeconomics forecast and they compared them to consensus estimates. Consensus estimates the gold standard in financial markets. And they basically found that Calci markets actually outperformed consensus in their predictive ability, which is a finding that Calci research had had three months prior, but of course we're not researchers at the Fed. So we don't have the level of credibility that is assigned to them. It's not just valuable to know how people are thinking and get a temperature check in a market based

36:33Estimate kind of way there's power to that of course when you're giving people skin in the game and you're asking them to put their money where their mouth is they're more likely to represent to you what they think. Is going to happen in the future in an accurate way because their incentive of course is to make money and I think that we see that across all markets to the extent that you think that market based pricing is good. It also applies to pricing of the future the difference with prediction markets is that you're able to get direct exposure to an event that's going to happen.

37:01If you're talking about polling as a comparison and you ask who's going to win the next election, there are certain biases that might affect how people respond to that question. Potentially, they don't want their neighbors to know that they're going to be voting for the less savoury candidate. Maybe it is the case that if you're an expert forecaster, you don't want to step out of line with the consensus estimate for the next inflation print. But if you're taking market-based estimates that people are putting money behind, those kinds of biases just don't apply because the incentive construction is so different.

37:27It's not even just the pure play probability that is useful of the thing that's going to happen in the future. But there are a number of derivatives of that that are also useful in the paper that was put out by the researchers at the Fed. They basically said, well, not just do we have an understanding of what's going to happen with inflation or interest rates next month. But what's also interesting is we can understand the distribution of opinions for the first time. We can see whether they're tightly compressed around one outcome or whether they're more dispersed. And that, of course, carries its own layer of information.

37:59Earlier this year, Kalshi put out a research paper on New York's May Oral race, which Zoran Mamdani won after starting out as a very bong shop. The paper is called Slowly, Then, All at Once. We actually looked at Mamdani's Rise to Power and we mapped it. We then did a component analysis to try and figure out what were the events that moved the market. What that gives you is access to a credibility layer on all of the media that you're consuming. It can tell you What is it that people care about and are paying attention to? Are there certain news outlets that they might price in as more valuable than others? Is it the case that they're responding to polling in a certain way? The prediction markets, they're reflecting information that polling can't pick up on like momentum and they're doing it in a continuously updating way that is distributionally rich and that is not able to be offered by any parallel mechanism.

38:48Even if prediction markets are more reliable than political polling, and they have been for a long time, they still aren't close to foolproof. Both Kalshi and its big rival Polymarket had Francesca Hong at 95% to win Wisconsin's Democratic primary for governor. She lost. A week later, the markets gave Angie Nixon only a single-digit probability of winning her Florida Democratic primary for Senate, and she won easily.

39:16Here's what Calci CEO Tarek Mansour posted on X. Before the prediction markets got it wrong headlines roll in, a 5% probability doesn't mean it won't happen. It means it should happen one in 20 times. If 5% candidates never won, the markets would be broken. Markets on elections are always going to make a lot of noise, but there are other important markets that are much quieter. Here's Mansour again.

39:45One of the things that we're rolling out is like markets on FDA approvals. Figuring out where to place capital on different drugs is one of the main drivers of drug discovery over time. That's incredibly important. Like if something has a 5% chance of succeeding that's being marketed as 70% chance of succeeding, you want the prediction market there. If you have a significantly more accurate gauge in what's going to go through and what's not, it's going to enable a significantly better allocation of capital across the board. And here's Nicole Kagan again.

40:12One thing that we're looking into now is listing more granular biotech biopharma markets. One thing that we are wrestling with on our end is if you're Pfizer and you've got some clinical trials in motion, you as Pfizer are prohibited from trading on the calcium market because you have MNPI on how that trial is running. MNPI stands for material non-public information. You as an institution shouldn't be able to take a position on that market at the very least, not directly.

40:38So we need to think about also what the incentive is for other companies in the space to potentially want to hedge out risk for approvals for say their competitors new drug FDA approvals corporate decision-making election forecasting even if you are a prediction market skeptic You can probably see some value there, and you can see why Kalshi likes to talk it up. But here is an uncomfortable fact. Roughly 90% of Kalshi's trading volume is in one category we've barely mentioned yet. Can you guess which one? I'll give you a hint. It's the kind of trading that is making DraftKings and FanDuel nervous. That's coming up after the break. This is Freakin'omics Radio, and I'm Stephen Dubner. Pretty much every single human, you

41:41Me, anyone else you can think of, is making predictions all the time about everything. It's just that we're not very systematic or accurate, and we are rarely held to account if our predictions turn out to be wrong. The whole point of a prediction market, meanwhile, is this. The whole point is you get rewarded for truth-seeking, you get rewarded for being right, and you get punished for being wrong. That again is Tarek Mansour, CEO of the prediction market, Kalshee.

42:10will succeed when trust in those prediction markets becomes mainstream. I think it's starting. One of the statistics that I like the most is 70 to 80% of our active users don't trade. They're just logging in next to X or they read a few newspapers in the morning. It helps them calibrate or fact check. I understand you don't want to talk about how much Calci and you are worth, but this plainly is a lucrative enterprise. Can you explain how Calci makes money?

42:39So we are a neutral platform or like the New York Stock Exchange or NASDAQ. People are training at each other and I take 1% on average. That is independent, agnostic on who's going to win or lose. That fee is the fee that I get for running the business and offering the regulated infrastructure and all the things that come with trading on Calci. My job is to provide a fair neutral competitive grounds for people to compete and they do their thing. They do their research. They go out and seek information. They do whatever it is they do to get more accurate.

43:08predictions and then go and trade them in the open market against each other. Like, I think the word is getting more bifurcated, more polarized. I think the noise signal ratio is going up. The reason is because most ways that we ingest information, this is especially true for social media, the algorithm essentially incentivize clickbait. If you write a well-nuanced, well-measured, multi-paragraph take about something, you get four likes on X. If you write something extreme and off the rails, you get tens of thousands of likes.

43:36and incentivize clickbait. The incentive structure in prediction markets is the exact opposite. That multi-paragraph, nuance, boring, well-calibrated take, that's the one that makes money. You're doing about four billion overall a week and... I think it might be a little bit more now, right? I mean, there's crypto politics. I think culture is the fastest growing. And then like financials. Can you make an argument that culture betting is good for society as well? Or is that more of an entertainment? I think figuring out whether albums are going to succeed or not. People are like fanatics and passionate about it.

44:06When you ask a lot of these people, like, hey, do you trade financial markets like S&P? And they're like, no, because I don't gamble. They're just like, what do you mean by that? When we launch weather, people are like, oh my god, people are weather gambling and stuff like that. You ask those traders, they're like the most sophisticated hardcore researchers. They're like scraping satellite data. They tell you we don't trade in financial markets because it's gambling. And he's like, what do you mean by that? He's like, well, the game is rigged against us. There is no shot for us to be the Wall Street hedge funds, et cetera. Whereas here.

44:31I can do research, I've spent already copious amounts of time reading and getting informed and I can like make money on it. They're participating in a financial market where they can have an edge.

44:41When I look at your numbers, your trades, it looks like roughly 90% of the trades are on sports right now. So you've kind of become a sports betting site, at least for now. So talk about the benefits and risks of being that sports heavy, whether gambling addiction is a problem you think about much, whether you want to diminish your share of sports betting, et cetera. So sports actually, the share is going down over time pretty fast. So the other categories are actually growing faster than sports.

45:08But that's natural because if sports is 90 and the others are small, growing fast, I mean, you can double from one to two in a month. They're still pretty big. Don't forget, we're very big, right? So the 10% is still very big. I want to kind of draw a distinction between this and gambling first. And I think it's very, very important. There is speculation in all financial markets. You know, when you look at the percentage of, you know, hedging for speculation in any derivatives markets like grain futures or commodity futures or stock options, it tends to be very tilted towards speculation. And the reason you want speculation is you want liquidity.

45:36without speculation, you don't get liquid in any of these markets. Now, the thing about speculation is that speculation has similarities to gambling, right? Like you're putting money to make more money on something you don't control, right? That's basically the definition of gambling in a lot of states. And that's why historically, you know, every time there's a new financial instrument or more means of access, like when Robinhood came around, it was like, oh, you know, gambling in the stock market or When grain futures got legalized in the US, I don't know if you know this, but they got legalized here as Supreme Court decision, which was like, is this gambling or is this a financial instrument? People used to call grain futures gambling. But the key thing is it's less about whether there's speculation or not in the market and more so about the business model. One business model is a business model that seeks out losers and then blocks winners. They do not want price discovery to happen naturally. The other one is a model that's geared towards price discovery. You want the smart people.

46:30Now, does that mean that second model, like the calcium model or financial markets model has no risks? The answer is no, right? They have risks. In my view, it's like there's a responsible gambling and there's also a responsible trading. You know, how many times have you heard of people losing their house over options trading, especially with zero data expiry options or futures or active day trading of stocks and prediction markets present similar risks. And I think we have a responsibility as a platform, which is part of why we have this regulatory first approach.

46:56to monitor those risks and make sure that we don't let people kind of fall off the cliff. So how far does that responsibility go? Wherever the sheep accumulate in the world of markets, the wolves go there. Nearly 20 states have already taken legal or regulatory action against prediction markets. Coming up next time in part two of the series, we will hear from a former CFTC chairman who thinks Kalshi may be on shaky ground. This will end up Debated and discussed amongst nine individuals in a small conference room in Washington, DC, and that's called the Supreme Court. That's next time on the show. Until then, take care of yourself. And if you can, someone else to. Freakonomics Radio is produced by Renbud Radio. You can find our entire archive on any podcast app. It's also at Freakonomics.com.

47:50where we publish transcripts and show notes. This episode was produced by Teo Jacobs. It was edited by Ellen Frankman and mixed by Jake Loomis with help from Jeremy Johnston. The Freakonomics Radio Network staff also includes Dalvin Abouaji, Eleanor Osborn, Elsa Hernandez, Gabriel Roth, Ellaria Madtenacourt, and Pete Madden. Our theme song is Mr. Fortune by the Hitchhikers, and our composer is Luis Guerra. As always, thank you for listening.

48:20We had to have somebody watch the entirety of the Victoria's Secret fashion show to see if the fantasy bra we're going to get won and that's probably the most uncomfortable that anyone has ever felt in the office.