3 ms·

Does anyone else see the "In X years time we'll have something Y times better than the competition has today" as a bit of a red flag? I saw this before in a pro

Does anyone else see the "In X years time we'll have something Y times better than the competition has today" as a bit of a red flag? I saw this before in a product plan and it flagged up 2 things that really worried me. Firstly, the competition were already ahead of us, and they're obviously going to continue developing their stuff, so it's great to promise our thing will be better than their thing today but we're not competing against that, we're competing against what they'll have once time has passed. And secondly, by measuring yourself against the leading edge today you're eliding how much you're going to improve. For example, Anthropic say they'll be 10x better than leading models today in 18 months. That sounds acheivable right (no actually) but you don't even have to do that - because you aren't starting with the market leading model, so you have to catch up first and then 10x it so are they 10x or 20x or 100x in 18 months?

bob1029 4y ago
The only reality in which Anthropic will take on OpenAI (et. al.), would be if someone involved possess some sacred knowledge regarding how to build an AGI system that is radically off the path that the current market is charging down (i.e. ever-larger GPU farms and transformer-style models). I suspect this is not the case. The same hustlers who brought you crypto scams didn't just disappear into the ether. All of that energy has to eventually go somewhere.
- nr2x 4y ago
  It’s not fair to compare them to crypto scams, this isn’t trying to juice retail investors for their life savings.
- adamsmith143 4y ago
  >The only reality in which Anthropic will take on OpenAI (et. al.), would be if someone involved possess some sacred knowledge regarding how to build an AGI system that is radically off the path that the current market is charging down (i.e. ever-larger GPU farms and transformer-style models). Train on More tokens with More GPUs isn't exactly rocket science. I assume the RLHF loop is complex but training the base model itself is pretty well understood.
skybrian 4y ago
Yes, predicting the future is difficult. Nobody knows who will really be ahead in X years. Nobody knows how much OpenAI will improve either. But they have ambitions to improve a lot and their plans are credible enough to satisfy their investors. Not sure what else you’re expecting? All VC investments have unquantifiable risks, but it doesn’t add up to a red flag if you like their chances.
nr2x 4y ago
100% agree. You need to catch up to OpenAI for starters and then figure out how to outpace them.
adamsmith143 4y ago
With the recent scaling law papers that have come out you can evidently predict with pretty high accuracy how good your model will be by plotting out the scaling curves. So performance @ X Flops and Y tokens can be reasonably well known ahead of time.

bob1029 4y ago

The only reality in which Anthropic will take on OpenAI (et. al.), would be if someone involved possess some sacred knowledge regarding how to build an AGI system that is radically off the path that the current market is charging down (i.e. ever-larger GPU farms and transformer-style models). I suspect this is not the case. The same hustlers who brought you crypto scams didn't just disappear into the ether. All of that energy has to eventually go somewhere.

nr2x 4y ago
It’s not fair to compare them to crypto scams, this isn’t trying to juice retail investors for their life savings.
adamsmith143 4y ago
>The only reality in which Anthropic will take on OpenAI (et. al.), would be if someone involved possess some sacred knowledge regarding how to build an AGI system that is radically off the path that the current market is charging down (i.e. ever-larger GPU farms and transformer-style models). Train on More tokens with More GPUs isn't exactly rocket science. I assume the RLHF loop is complex but training the base model itself is pretty well understood.

skybrian 4y ago

Yes, predicting the future is difficult. Nobody knows who will really be ahead in X years. Nobody knows how much OpenAI will improve either. But they have ambitions to improve a lot and their plans are credible enough to satisfy their investors. Not sure what else you’re expecting? All VC investments have unquantifiable risks, but it doesn’t add up to a red flag if you like their chances.

nr2x 4y ago

100% agree. You need to catch up to OpenAI for starters and then figure out how to outpace them.

adamsmith143 4y ago

With the recent scaling law papers that have come out you can evidently predict with pretty high accuracy how good your model will be by plotting out the scaling curves. So performance @ X Flops and Y tokens can be reasonably well known ahead of time.