Mismatched Monetary Motivation in Manifold Markets

Why the current instantaneous implied probabilites may be less useful than they seem

I’ve been a casual follower of prediction markets for some time, which are a fascinating and promising way to aggregate “wisdom of the crowds” forecasts of uncertain events. But for the reasons Scott Alexander describes in this post and elsewhere, none of the incumbent markets have felt to me like they really nailed the effortless experience necessary to scale for mass adoption to really achieve their potential. That’s why I was excited to learn about Manifold Markets, which takes a usability-first approach to prediction markets. I’ve spent some time in the last couple days playing with it, and have found it fun and pleasantly easy to use, so I’m excited about its future (enough to dust off this blog that I haven’t touched in close to a decade).

Unfortunately, I think it gets some important incentives wrong, which will lead to miscalibrated probability estimates. This is because the typical expected payout differs substantially from the instantaneous implied probability in a way that significantly disincentivizes accurate price setting. I’ll start with a brief overview of how Manifold Markets betting works, then spend the rest of this post elaborating with a concrete example what I believe the issue is. Finally I’ll let you judge if I’m right by placing bets on whether or not Manifold’s developers will be convinced enough by this argument to alter their calculations – making this post something of an experiment in prediction-market-backed open research.

Dynamic Parimutuel Betting

First, let’s take a look at how Manifold’s betting system works. Briefly, it uses a dynamic parimutuel (DPM) betting system with a dynamic market making mechanism, described in more detail here. This is similar to traditional parimutuel betting (TPM) in which bettors take opposite sides of a proposition, put money into a pot, and then those on the winning side split the pot. Crucially, TPM betting happens during a relatively short window and is closed before the deciding event. Thus the implied odds at the close of the betting window apply fairly to all bettors, and nobody has an opportunity to come in after the event with fresh information and take advantage of the naive earlier bettors.

TPM works just fine for horse races, but interesting real world events often build up gradually over the course of days, weeks, or years. In a prediction market, we’d like to capture these time dynamics to provide continually updating probabilities. There’s already a traditional solution to this: the continual double-sided auction (CDA). This is the familiar system of the stock market, also employed by many prediction markets: buyers and sellers write down their prices, which go into a bid/ask queue, and whenever a bid and an ask come together at the same price, a trade is made. So why are we even talking about parimutuel betting when we already have CDA? Well, a CDA works great when you have a liquid market with lots of participants, but less well for small markets with few traders. Because Manifold wants to address large numbers of low-liquidity markets, they needed a different approach.

Finally, we come back to the DPM system. Compared to CDA it deals well with low liquidity, and unlike a TPM it allows for continuous incorporation of new information. The secret to this, and the heart of the DPM, is the pricing rule. The pricing rule determines how much of a share of the pot each bettor gets, as a function of the current state of the market. There are infinitely many possible pricing rules, and the chosen pricing rule has a lot of influence over the incentives of the traders. Manifold uses a quadratic pricing rule described in their technical guide. This pricing rule is perfectly functional, but as we’ll see shortly it leaves something to be desired.

Example Scenario

To see how, let’s consider a simple example of a hypothetical sports game. Suppose going into the game, the Red team is an underdog, widely agreed to have only a 20% chance of winning. Mandy has made a market at this price, staking an initial pool and setting the odds at 20%. Alice, an expert in hypothetical sports, sees an opportunity. According to her careful analysis and deep expertise, she believes the Red team’s offense has some tricks that the Blue team won’t see coming. Alice thinks Red are still the underdogs but they just might pull it off, maybe with a 30% chance of winning. She confidently bids the market from 20% up to 25%, hoping for a payout of somewhere between 4x to 5x in her favor should she win, making her bet comfortably +EV.

If we were to stop there, all would be well, but we would just be replicating TPM. But we’re not, this is dynamic, and the game is afoot! After a tense first half, the Red team’s gambits are paying off and they’ve taken the lead. Bob, who is watching the game unfold and knows how things tend to play out in situations like this, sees that Red are now at least 60% favorites and bids the market from 25% up to 50%, hoping for a payout somewhere between 2x and 4x if they pull through. They continue to fend off Blue for the rest of the game and the moment they achieve victory, Charlie, who knows nothing about hypothetical sports but is quick with his keyboard, sees an easy sure thing and bids the market from 50%->99%. He’s making essentially a risk-free bet, so he’s comfortable picking up whatever free money is sitting there.

Broken Incentives

Now let’s run the numbers and see how everybody did. First I’ll walk through each point in time giving concrete numbers for the market state in the form RedShares:BlueShares and the implied probability from Manifold’s quadratic cost pricing rule \(C(y,n) = \sqrt{y^2 + n^2}\):

• Market created by Mandy [447:894; p=0.2]
  • Mandy antes $1000 to stake, and gets 447 shares yes and 894 shares no
• Alice buys in [516:894; p=0.25]
  • Alice bets $33 and gets 69 yes shares
• Bob buys in [894:894; p=0.5]
  • Bob bets $232 and gets 378 yes shares
• Charlie buys in [8899:894; p=0.99]
  • Charlie bets $7679 and gets 8005 yes shares

At close, the market resolves to Yes. There is $8944 in the pot, split among 8899 yes shares, so each yes share redeems for $1.005 (we’ll ignore fees here for simplicity). How did everyone do?

• Mandy walks away with $449, 45% of her original stake or a loss of $551
• Alice gets back $69, 2.1x her bet or a profit of $37
• Bob gets back $380, 1.6x his bet or a profit of $148
• Charlie gets back $8045, 1.05x his bet or a profit of $366

This is an enormously unfair outcome! Alice based her bet on the assumption of getting at least a 4x payout on a win, but really she only got 2.1x, a huge discrepancy. Bob did okay, but still did substantially worse than the 2x+ he expected. Meanwhile Charlie did fine, picking up most of the pot with his large sure-thing bet. And Mandy got off better than she might have feared, given how far on the wrong side of the bet she ended up.

Let’s take a closer look at Alice’s situation. When she placed her bet, the Manifold interface would have shown her a payout of at least 4x her bid, implying that this is what she would actually receive if the market closed in her favor. But she was deceived, as this was never a possible outcome! Because information about who won the game is available to all by the end of the game, we know going in that the final probability will be either very close to 0 (if Red loses) or very close to 1 (if Red wins). Thus Alice could know before betting that she would almost certainly either lose everything or end up redeeming her yes shares at very close to $1 each. This means her implied payout on a win was really only 2.1x from the start, not the 4x+ advertised by Manifold. This isn’t enough to justify her bid: she only thought Red had a true probability of 30%, while she would need close to 50% to justify a bet at the actual expected payout. In a true accounting, Alice never had an incentive to bet at all!

This same issue will be present in any market for which the resolution is expected to become clear before the close of the market and drive the price toward either 0 or 100%, which covers a wide class of prediction market use cases. And even if this is avoided, a similar issue is present for any market with continuous incorporation of information. The more a trader expects future information to eventually become available, the more the instantaneous implied probability will differ from the true expected payout. And although the former is what is displayed in Manifold’s interface, the latter is what actually matters to traders. This disincentivizes traders from correcting market prices based on current information when future information can be anticipated, and risks removing much of the utility of a prediction market.

I’m not going to propose a specific solution here, as I’m not very familiar with the literature on DPMs, though a quick search does turn up some leads. I also suspect an alternative solution would be to use a CDA backed by an automated market maker subsidized by the auction creator. Instead, my goal for now is to find out if the market thinks I am right about this being a problem, and being worth fixing. To that end, I’ve created a Manifold market, which you can sign up for free to place (fake money) bets on. I will resolve the market to Yes if a Manifold developer is influenced by the market to change the way Manifold evaluates bets, and No otherwise. Let the trading begin!

Update 3/13/2022: Manifold agreed to change their betting market to an automated market maker approach, so I resolved this post’s market to yes!

Update 3/28/2022: This post’s market was featured in a section of this Astral Codex Ten post on “Attention Markets”!