4 ms·

Superintelligence startup Reflection AI launches with $130M in funding

38 points by t55 2y ago

_lvbh 2y ago
I'm surprised there's still so much hype funding 3 years in. There's still no evidence LLMs lead to "superintelligence" - it's interesting that's the word chosen by all these new startups
- t55 2y ago
  Well, it depends. on some tasks, they surely are already super-intelligent.
  - lblume 2y ago
    When talking about superintelligence it is generally clear that general intelligence is meant, as specialized superintelligence has been solved for many areas for decades now without modern AI.
    - highfrequency 2y ago
      True but misses the point - surely quantity matters? A machine that can do 0.001% of basic tasks vs. one that can do 20% of basic tasks seems qualitatively different.
      - lblume 2y ago
        How to classify that fraction? Let's say D. Gukesh (current chess champion) plays and learns chess for 50% of his time awake. Would you consider Stockfish to be a superintelligence that accounts for 50% of basic task wrt D. Gukesh? What if tasks (like some types of entertainment) usually are not relevantly attributed to intelligence at all? I would love to find answers to the questions but find them difficult to the point of being too vague to answer.
- highfrequency 2y ago
  What would constitute promising evidence that LLMs may lead to superintelligence in your eyes? I mean this as a serious question. Suddenly we can talk to computers in plain language, they can solve a broad range of technical and non-technical problems, they get significantly better every year… it’s hard for me to imagine more promising evidence that AGI is on the horizon besides actually achieving AGI.
  - missedthecue 2y ago
    When an LLM cares about something would be promising evidence to me. As they currently exist, they are essentially a novel and extremely sophisticated method to search, derive, and understand data. In fact almost all of the data ever recorded. To OPs point, every new LLM startup is just trying to build a bigger and more sophisticated way to search, derive, and understand data. It's not clear to me that bigger and more compute intensive methods will create an LLM that cares about anything.
    - JumpCrisscross 2y ago
      > they are essentially a novel and extremely sophisticated method to search, derive, and understand data What about this doesn’t sound supremely valuable?
      - jdlshore 2y ago
        The debate is about superintelligence, not value.
        deleted 2y ago
        [deleted]
  - vineyardmike 2y ago
    Not OP, but I'll bite. I, firstly, don't think it's fair to say "they get significantly better every year" after 2 years. I honestly think GPT 3/3.5 (maybe optimized for cost or speed but not retrained) would have been adequate for a significant chunk of the general purpose tasks that are asked of LLMs. I think most of the other gains we've seen since are related to fine-tuning on intentional application-specific tasks. IMO the only real "significant" improvement is the native multi-modal models. That said, I think that RL+thinking and long-context models will start to present enough combined incremental improvements towards my next points to be even more useful and capable at a wider variety of tasks, but there is (in their current usable forms like chat bot apps and public APIs) a fundamental limit on their implementation preventing them from being AGI. I think that models, even the "thinking" ones, fail to truly perform novel rationalization around general-purpose task solving. We have a ton of evidence that they're useful for a ton of tasks when it's trained in. Even going a tiny bit in a unique direction drops the model output quality a ton. Available thinking models today are really good at math, because they were RLed into solving math like a school child - rote practice. But that doesn't mean they're capable of even applying the same logical tools (they should have learned along) the way to novel mathematic and logic questions. Another impediment to being treated even as an application-specific "mini AGI" is their naïveté and hallucinations. This makes their use as agents suspect for anything important. They can't distinguish or even output a true confidence on what they "don't know", and this blind confidence is a setback. Humans are known to say incorrect things confidently, but they're also known to reflect on their lack of knowledge and recognize their limits. Humans have real memory to be able (imprecisely) associate an event with their learning, to aid confidence in recall. Similarly, LLMs trusting nature on input (eg. prompt injection attacks) prevent them from "intelligently" acting in the real world even when they're not hallucinating. Tools like "DeepResearch" are really useful, and impressive improvements on traditional human searching for processing the vastness of the internet. BUT the model can't genuinely distinguish between good and bad sources, and often can't intelligently reflect on the patterns and social context of the sources they sell. I can totally see a world where an LLM can output a confidence metric which is used to drive the tokens, and potentially suppress output, and I can totally see a world where long context and thinking (w/ RL) gives it enough reflection on everything to question to function even more autonomously. But I remain skeptical that it will be able to "think" and rationalize deeply enough to be a "super intelligence" on tasks it wasn't taught.
- spaceman_2020 2y ago
  I don't even know why superintelligence is a goal. I'll be happy with a 120IQ digital being that handles all my busy work
- arisAlexis 2y ago
  Actually your statement contradicts the top-3 AI h-index scientists and Nobel laureates. What makes you so sure? Curious
- arisAlexis 2y ago
  Actually your statement contradicts the top-3 AI h-index scientists + sama + amodei + Elon and Nobel laureates. What makes you so sure? Curious
jjtheblunt 2y ago
I wonder how this compares to Generally Intelligent / Imbue (renamed), whose hiring ads were regularly appearing in HN for months, but I've not seen that lately.
sunami-ai 2y ago
Part of me thinks that that before they gave the $130M they knew exactly how they'd get it back (thru some pre-arranged M&A) if it doesn't work out. Or at least that would be the smart thing to do.
- _cs2017_ 2y ago
  How can you get back the money that was spent? Fancy terms like M&A won't return the money that is gone.
  - sunami-ai 2y ago
    I spent $10 on a bad apple and it got $0 back. Then I go and sell that bad apple for $10. Whomever bought it is giving me back my money. This is kindergarten level arithmetic.
    - _cs2017_ 2y ago
      I didn't mean that it's mathematically impossible. I meant that it's practically not gonna happen. Much as they'd love to, no VC gets a chance to invest in an early stage startup knowing that if the startup fails, there's a buyer ready to make the VC whole.
vivzkestrel 2y ago
imagine what a fancy office with state of the art wallpapers, furniture and state of the art robotic coffee making machines you could add to your office with $130M funding
1stsentient 2y ago
1)It must be able to provide answers to Zen Koans as a test of its synthetic intuition. 2) Must solve P vs NP problem or it's equivalent. 3) Must be able to provide emotional support to a person who recently lost a loved one.

38 points by t55 2y ago

I'm surprised there's still so much hype funding 3 years in. There's still no evidence LLMs lead to "superintelligence" - it's interesting that's the word chosen by all these new startups

t55 2y ago
Well, it depends. on some tasks, they surely are already super-intelligent.
- lblume 2y ago
  When talking about superintelligence it is generally clear that general intelligence is meant, as specialized superintelligence has been solved for many areas for decades now without modern AI.
  - highfrequency 2y ago
    True but misses the point - surely quantity matters? A machine that can do 0.001% of basic tasks vs. one that can do 20% of basic tasks seems qualitatively different.
    - lblume 2y ago
      How to classify that fraction? Let's say D. Gukesh (current chess champion) plays and learns chess for 50% of his time awake. Would you consider Stockfish to be a superintelligence that accounts for 50% of basic task wrt D. Gukesh? What if tasks (like some types of entertainment) usually are not relevantly attributed to intelligence at all? I would love to find answers to the questions but find them difficult to the point of being too vague to answer.
highfrequency 2y ago
What would constitute promising evidence that LLMs may lead to superintelligence in your eyes? I mean this as a serious question. Suddenly we can talk to computers in plain language, they can solve a broad range of technical and non-technical problems, they get significantly better every year… it’s hard for me to imagine more promising evidence that AGI is on the horizon besides actually achieving AGI.
- missedthecue 2y ago
  When an LLM cares about something would be promising evidence to me. As they currently exist, they are essentially a novel and extremely sophisticated method to search, derive, and understand data. In fact almost all of the data ever recorded. To OPs point, every new LLM startup is just trying to build a bigger and more sophisticated way to search, derive, and understand data. It's not clear to me that bigger and more compute intensive methods will create an LLM that cares about anything.
  - JumpCrisscross 2y ago
    > they are essentially a novel and extremely sophisticated method to search, derive, and understand data What about this doesn’t sound supremely valuable?
    - jdlshore 2y ago
      The debate is about superintelligence, not value.
      - deleted 2y ago
        [deleted]
- vineyardmike 2y ago
  Not OP, but I'll bite. I, firstly, don't think it's fair to say "they get significantly better every year" after 2 years. I honestly think GPT 3/3.5 (maybe optimized for cost or speed but not retrained) would have been adequate for a significant chunk of the general purpose tasks that are asked of LLMs. I think most of the other gains we've seen since are related to fine-tuning on intentional application-specific tasks. IMO the only real "significant" improvement is the native multi-modal models. That said, I think that RL+thinking and long-context models will start to present enough combined incremental improvements towards my next points to be even more useful and capable at a wider variety of tasks, but there is (in their current usable forms like chat bot apps and public APIs) a fundamental limit on their implementation preventing them from being AGI. I think that models, even the "thinking" ones, fail to truly perform novel rationalization around general-purpose task solving. We have a ton of evidence that they're useful for a ton of tasks when it's trained in. Even going a tiny bit in a unique direction drops the model output quality a ton. Available thinking models today are really good at math, because they were RLed into solving math like a school child - rote practice. But that doesn't mean they're capable of even applying the same logical tools (they should have learned along) the way to novel mathematic and logic questions. Another impediment to being treated even as an application-specific "mini AGI" is their naïveté and hallucinations. This makes their use as agents suspect for anything important. They can't distinguish or even output a true confidence on what they "don't know", and this blind confidence is a setback. Humans are known to say incorrect things confidently, but they're also known to reflect on their lack of knowledge and recognize their limits. Humans have real memory to be able (imprecisely) associate an event with their learning, to aid confidence in recall. Similarly, LLMs trusting nature on input (eg. prompt injection attacks) prevent them from "intelligently" acting in the real world even when they're not hallucinating. Tools like "DeepResearch" are really useful, and impressive improvements on traditional human searching for processing the vastness of the internet. BUT the model can't genuinely distinguish between good and bad sources, and often can't intelligently reflect on the patterns and social context of the sources they sell. I can totally see a world where an LLM can output a confidence metric which is used to drive the tokens, and potentially suppress output, and I can totally see a world where long context and thinking (w/ RL) gives it enough reflection on everything to question to function even more autonomously. But I remain skeptical that it will be able to "think" and rationalize deeply enough to be a "super intelligence" on tasks it wasn't taught.
spaceman_2020 2y ago
I don't even know why superintelligence is a goal. I'll be happy with a 120IQ digital being that handles all my busy work
arisAlexis 2y ago
Actually your statement contradicts the top-3 AI h-index scientists and Nobel laureates. What makes you so sure? Curious
arisAlexis 2y ago
Actually your statement contradicts the top-3 AI h-index scientists + sama + amodei + Elon and Nobel laureates. What makes you so sure? Curious

jjtheblunt 2y ago

I wonder how this compares to Generally Intelligent / Imbue (renamed), whose hiring ads were regularly appearing in HN for months, but I've not seen that lately.

sunami-ai 2y ago

Part of me thinks that that before they gave the $130M they knew exactly how they'd get it back (thru some pre-arranged M&A) if it doesn't work out. Or at least that would be the smart thing to do.

_cs2017_ 2y ago
How can you get back the money that was spent? Fancy terms like M&A won't return the money that is gone.
- sunami-ai 2y ago
  I spent $10 on a bad apple and it got $0 back. Then I go and sell that bad apple for $10. Whomever bought it is giving me back my money. This is kindergarten level arithmetic.
  - _cs2017_ 2y ago
    I didn't mean that it's mathematically impossible. I meant that it's practically not gonna happen. Much as they'd love to, no VC gets a chance to invest in an early stage startup knowing that if the startup fails, there's a buyer ready to make the VC whole.

vivzkestrel 2y ago

imagine what a fancy office with state of the art wallpapers, furniture and state of the art robotic coffee making machines you could add to your office with $130M funding

1stsentient 2y ago

1)It must be able to provide answers to Zen Koans as a test of its synthetic intuition. 2) Must solve P vs NP problem or it's equivalent. 3) Must be able to provide emotional support to a person who recently lost a loved one.