7 ms·

Metaculus shows a 74% chance Starship reaches orbit in 2024, and a projected date of March 26 for the next launch. That seems pretty bullish to me! Next year (2

Metaculus shows a 74% chance Starship reaches orbit in 2024, and a projected date of March 26 for the next launch. That seems pretty bullish to me! Next year (2025) will probably see the beginnings of practical customer usage of Starship. [0] https://www.metaculus.com/questions/?status=open&has_group=false&main-feed=true&type=forecast&order_by=-rank&search=starship https://www.metaculus.com/questions/?status=open&has_group=f...

bumby 3y ago
How does Metaculus make such predictions? I assume it can't be in the reliability engineering fashion used in industry.
- avionical 3y ago
  Unclear, but the predictions appear to be better on things like US elections than prediction markets or 538: https://firstsigma.substack.com/p/midterm-elections-forecast-comparison-analysis https://firstsigma.substack.com/p/midterm-elections-forecast...
- chpatrick 3y ago
  Wisdom of the crowds.
  - johnsimer 3y ago
    but also skin in the game and supply/demand Should be much more accurate than a simple poll, because people with the belief that the odds are undervalued will put money in the game to hopefully profit off their belief It's more like the Efficient Market Hypothesis
    - samatman 3y ago
      There's no skin in the game with Metaculus, it's a prediction aggregator, not a prediction market. Nothing is at stake. This is why you see a somewhat absurd skew on AI doom/safety questions, using Metaculus is a hobby of the cult^H^H^H^H movement, and no one's swooping in to steal their money because there isn't any.
      - gbear605 3y ago
        It’s somewhat more complex than that because Metaculus weights its aggregates by how accurate predictors have been in the past. Definitely not perfect though, and the efficient market hypothesis isn’t relevant at all.
  - bumby 3y ago
    How well does this work when there is fundamentally little information? It works in stock trading because so much information is standardized and regulated. I am skeptical of it being as useful here simply because the crowds don't have much relevant information, as far as I know. For example, do the crowds know about the design redundancy? What about the supplier quality control? Or the reliability data on the check valves used? etc. etc.
    - avmich 3y ago
      Engineering, physical principles, I guess. Those who place bets or making predictions may know a thing or two on mechanics, rocket science and history of building launchers.
      - bumby 3y ago
        To put a finer point on it, those seem like rather blunt instruments for prediction compared to how risk is typically measured by aerospace reliability engineers.
        avmich 3y ago
        I'm sure some aerospace professionals are wondering how it's possible to predict stock behavior on an exchange, but have no problem with educated predictions of launcher performance.
        bumby 3y ago
        That right. The point being, they are very different systems with different information inputs. Meaning what works for one may not be a good method the the other.
        persolb 3y ago
        Think of it like top down budget va bottom up budget. RAMS engineers do a bottom up analysis. However, it is fairly typical for the to be subtlety biased in a way that changes the final answer. (Usually by ‘rounding up’ on individual hazards). Top down takes an outside view and says ‘this looks like these other past projects. It should land in around the same place.’ I’ve found top down tends to be more accurate project wise. Bottom up more accurate for specific subsystems.
- ajkjk 3y ago
  They make it up. But everyone assumes it means something because it's an average of a bunch of people making it up.
  - falcor84 3y ago
    No need to assume, you can check the prediction history and see that they generally do quite well. You also look at the histories of individual predictors' profiles and see some with consistent successes.
    - ajkjk 3y ago
      Well what is interesting is not how well the aggregate does, but how much better it does than individuals. After all if individuals can predict stuff consistently then what does the prediction market add?
      - falcor84 3y ago
        My understanding is that the market, which rewards and punishes your predictions is what makes one a better predictor.
        ajkjk 3y ago
        Right, but that doesn't do anything unless there's life-savings-amounts of money on the line such that anyone is incentivized to actually try. And even when people are incentivized, just look at the stock market for clear evidence that it doesn't work: a market doesn't magically make people good at things; people screw up and lose bets and go bankrupt all the time. Sure some of the decently-competent banks and hedge funds don't---but there is no reason to think there are any actors close to that sophisticated on a prediction market.
        falcor84 3y ago
        You're of course that it's not yet as sophisticated as the stock exchange, but I wouldn't be as dismissive of the importance people attach to internet points.
gibolt 3y ago
They are aiming for 6 launches this year. That is a lot of chances to make it work. Even if there is a catastrophic failure that destroys the launch tower, they are building another and will still likely get 2-3 attempts.
hoseja 3y ago
Bullish? Is there someplace I can bet against them? I don't see how Starship isn't getting into orbit on this or the next launch, barring major regulatory sabotage.

bumby 3y ago

How does Metaculus make such predictions? I assume it can't be in the reliability engineering fashion used in industry.

avionical 3y ago
Unclear, but the predictions appear to be better on things like US elections than prediction markets or 538: https://firstsigma.substack.com/p/midterm-elections-forecast-comparison-analysis https://firstsigma.substack.com/p/midterm-elections-forecast...
chpatrick 3y ago
Wisdom of the crowds.
- johnsimer 3y ago
  but also skin in the game and supply/demand Should be much more accurate than a simple poll, because people with the belief that the odds are undervalued will put money in the game to hopefully profit off their belief It's more like the Efficient Market Hypothesis
  - samatman 3y ago
    There's no skin in the game with Metaculus, it's a prediction aggregator, not a prediction market. Nothing is at stake. This is why you see a somewhat absurd skew on AI doom/safety questions, using Metaculus is a hobby of the cult^H^H^H^H movement, and no one's swooping in to steal their money because there isn't any.
    - gbear605 3y ago
      It’s somewhat more complex than that because Metaculus weights its aggregates by how accurate predictors have been in the past. Definitely not perfect though, and the efficient market hypothesis isn’t relevant at all.
- bumby 3y ago
  How well does this work when there is fundamentally little information? It works in stock trading because so much information is standardized and regulated. I am skeptical of it being as useful here simply because the crowds don't have much relevant information, as far as I know. For example, do the crowds know about the design redundancy? What about the supplier quality control? Or the reliability data on the check valves used? etc. etc.
  - avmich 3y ago
    Engineering, physical principles, I guess. Those who place bets or making predictions may know a thing or two on mechanics, rocket science and history of building launchers.
    - bumby 3y ago
      To put a finer point on it, those seem like rather blunt instruments for prediction compared to how risk is typically measured by aerospace reliability engineers.
      - avmich 3y ago
        I'm sure some aerospace professionals are wondering how it's possible to predict stock behavior on an exchange, but have no problem with educated predictions of launcher performance.
        bumby 3y ago
        That right. The point being, they are very different systems with different information inputs. Meaning what works for one may not be a good method the the other.
      - persolb 3y ago
        Think of it like top down budget va bottom up budget. RAMS engineers do a bottom up analysis. However, it is fairly typical for the to be subtlety biased in a way that changes the final answer. (Usually by ‘rounding up’ on individual hazards). Top down takes an outside view and says ‘this looks like these other past projects. It should land in around the same place.’ I’ve found top down tends to be more accurate project wise. Bottom up more accurate for specific subsystems.
ajkjk 3y ago
They make it up. But everyone assumes it means something because it's an average of a bunch of people making it up.
- falcor84 3y ago
  No need to assume, you can check the prediction history and see that they generally do quite well. You also look at the histories of individual predictors' profiles and see some with consistent successes.
  - ajkjk 3y ago
    Well what is interesting is not how well the aggregate does, but how much better it does than individuals. After all if individuals can predict stuff consistently then what does the prediction market add?
    - falcor84 3y ago
      My understanding is that the market, which rewards and punishes your predictions is what makes one a better predictor.
      - ajkjk 3y ago
        Right, but that doesn't do anything unless there's life-savings-amounts of money on the line such that anyone is incentivized to actually try. And even when people are incentivized, just look at the stock market for clear evidence that it doesn't work: a market doesn't magically make people good at things; people screw up and lose bets and go bankrupt all the time. Sure some of the decently-competent banks and hedge funds don't---but there is no reason to think there are any actors close to that sophisticated on a prediction market.
        falcor84 3y ago
        You're of course that it's not yet as sophisticated as the stock exchange, but I wouldn't be as dismissive of the importance people attach to internet points.

gibolt 3y ago

They are aiming for 6 launches this year. That is a lot of chances to make it work. Even if there is a catastrophic failure that destroys the launch tower, they are building another and will still likely get 2-3 attempts.

hoseja 3y ago

Bullish? Is there someplace I can bet against them? I don't see how Starship isn't getting into orbit on this or the next launch, barring major regulatory sabotage.