10 ms·

I'm not super familiar with the state of the art technology in this space and how these demos were presented, but I think all of these conflicts seem like they

I'm not super familiar with the state of the art technology in this space and how these demos were presented, but I think all of these conflicts seem like they should be resolved if companies just put gigantic honking disclaimers on the work these AI tools produce. If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."

adamsmith143 4y ago
>If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results." I'll give you the benefit of the doubt since the demo is now offline but there was in fact a giant disclaimer that said more or less: "DO NOT TRUST THE OUTPUT OF A LANGUAGE MODEL WITHOUT VERIFICATION."
- mattkrause 4y ago
  It did...but it also said stuff like "ACCESS ALL OF HUMANITY'S KNOWLEDGE!!!" They were trying to have it both ways, where the headline giveth and the fine print taketh away. In a better world, this could have been released as "We trained a language model on the scientific literature. We're excited because it is starting to draw conclusions, but there is obviously a lot more to be done. See our evaluation here. Have fun and let us know what you find." The hype is really annoying and seems bizarre--everyone who even remotely cares knows about Meta already.
- tsimionescu 4y ago
  That was only present on the bottom half of the "Mission" page - the main page and other pages were full of examples of all the marvelous things it can do (that it can't really do reliably) [0]. https://web.archive.org/web/20221116161353mp_/https://galactica.org/ https://web.archive.org/web/20221116161353mp_/https://galact...
seydor 4y ago
They did have a disclaimer in every page that AI models hallucinate. But Lecun has been caught in the general 'facebook bad' sentiment and some human broken clocks who are constantly bashing anything DL-related. this is just a bunch of personal vendettas imho. The model was useful
- Veen 4y ago
  Useful for doing what? Certainly not what LeCun and Meta originally claimed it was useful for.
  - seydor 4y ago
    Do you know what they claimed it was doing?
    - Veen 4y ago
      I know what LeCun originally said on Twitter and what the copy on the Galactica web page said. But to repeat my question, what is the model useful for? How can it be used to achieve something that bears any resemblance to the original claims?
      - seydor 4y ago
        Afaik the claim was not that "it writes correct papers", but that it compiles knowledge. and to that extent it was accurate, along with the disclaimer that it hallucinates things, because it goes on tangents that are often unrelted The paper was useful for that reason: as a search engine for underresearched and speculative connections within fields, and from what i saw it does well in that. For example, when brainwashing ideas or looking for other research directions to review. In fact, we already have Perspectives papers being published in journals which are speculative in nature. This is a reasonable take on its usefulness: https://twitter.com/mad_enrico/status/1592664384031330309 https://twitter.com/mad_enrico/status/1592664384031330309 > It's still not perfect but a very interesting tool to search about new topics,
      - 4y ago
rb2k_ 4y ago
I think one of the problems with large parts of academia is that the system thrives by finding flaws within things other people have created. Criticism is certainly helpful and necessary for advancing the state of the art, but without something to balance it, it turns into a pretty bleak place to work. I guess this is one example that feels more like a personal vendetta than a constructive criticism of the work.
- timmytokyo 4y ago
  Science is, always has been, and always should be an adversarial endeavor. It's part of what makes it science.
  - rb2k_ 4y ago
    There should be a difference between applying critical thinking and being borderline hostile though.
    - timmytokyo 4y ago
      Agreed. One can be adversarial and professional at the same time.
dekhn 4y ago
I'll put it quite simply. If you promote an ML system that is ostensibly a scientific content generator, the expectation is that it will regurgitate facts, not lies. If you can't get that right, don't release it, except to publish a "negative results" paper.
- seydor 4y ago
  That is just false. The model had a disclaimer on every page that it is not to be trusted and that it hallucinates. Strawmanning is not a nice academic retort
  - paganel 4y ago
    To be honest projects that constantly and consistently generate smart falsehoods (the "bears in the sky" that the author of the article mentions) don't have any scientific need, so to speak. Or, if they do, I fail to see it.
    - seydor 4y ago
      They do. Try asking it about a field that is very novel or very very niche, and it will point out various links. Biology connections (which are often very vague and open) are good for that. I found that it generated some links between mechanisms that i found interesting. Unfortunately the model is down so i cant review it.
      - mynameisvlad 4y ago
        But how can you possibly trust anything the model spits out as accurste and true if it's happy to spit out a whole thing on bears in space, something that plainly never happened. Sure, it be spitting out these links between mechanisms, but it could have pulled them out of the same place it pulled an article about bears in space: the server's ass. And verification is going to take you far longer on something that actually seems realistic, which is a massive problem.

adamsmith143 4y ago

>If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results." I'll give you the benefit of the doubt since the demo is now offline but there was in fact a giant disclaimer that said more or less: "DO NOT TRUST THE OUTPUT OF A LANGUAGE MODEL WITHOUT VERIFICATION."

mattkrause 4y ago
It did...but it also said stuff like "ACCESS ALL OF HUMANITY'S KNOWLEDGE!!!" They were trying to have it both ways, where the headline giveth and the fine print taketh away. In a better world, this could have been released as "We trained a language model on the scientific literature. We're excited because it is starting to draw conclusions, but there is obviously a lot more to be done. See our evaluation here. Have fun and let us know what you find." The hype is really annoying and seems bizarre--everyone who even remotely cares knows about Meta already.
tsimionescu 4y ago
That was only present on the bottom half of the "Mission" page - the main page and other pages were full of examples of all the marvelous things it can do (that it can't really do reliably) [0]. https://web.archive.org/web/20221116161353mp_/https://galactica.org/ https://web.archive.org/web/20221116161353mp_/https://galact...

seydor 4y ago

They did have a disclaimer in every page that AI models hallucinate. But Lecun has been caught in the general 'facebook bad' sentiment and some human broken clocks who are constantly bashing anything DL-related. this is just a bunch of personal vendettas imho. The model was useful

Veen 4y ago
Useful for doing what? Certainly not what LeCun and Meta originally claimed it was useful for.
- seydor 4y ago
  Do you know what they claimed it was doing?
  - Veen 4y ago
    I know what LeCun originally said on Twitter and what the copy on the Galactica web page said. But to repeat my question, what is the model useful for? How can it be used to achieve something that bears any resemblance to the original claims?
    - seydor 4y ago
      Afaik the claim was not that "it writes correct papers", but that it compiles knowledge. and to that extent it was accurate, along with the disclaimer that it hallucinates things, because it goes on tangents that are often unrelted The paper was useful for that reason: as a search engine for underresearched and speculative connections within fields, and from what i saw it does well in that. For example, when brainwashing ideas or looking for other research directions to review. In fact, we already have Perspectives papers being published in journals which are speculative in nature. This is a reasonable take on its usefulness: https://twitter.com/mad_enrico/status/1592664384031330309 https://twitter.com/mad_enrico/status/1592664384031330309 > It's still not perfect but a very interesting tool to search about new topics,
    - 4y ago

rb2k_ 4y ago

I think one of the problems with large parts of academia is that the system thrives by finding flaws within things other people have created. Criticism is certainly helpful and necessary for advancing the state of the art, but without something to balance it, it turns into a pretty bleak place to work. I guess this is one example that feels more like a personal vendetta than a constructive criticism of the work.

timmytokyo 4y ago
Science is, always has been, and always should be an adversarial endeavor. It's part of what makes it science.
- rb2k_ 4y ago
  There should be a difference between applying critical thinking and being borderline hostile though.
  - timmytokyo 4y ago
    Agreed. One can be adversarial and professional at the same time.

dekhn 4y ago

I'll put it quite simply. If you promote an ML system that is ostensibly a scientific content generator, the expectation is that it will regurgitate facts, not lies. If you can't get that right, don't release it, except to publish a "negative results" paper.

seydor 4y ago
That is just false. The model had a disclaimer on every page that it is not to be trusted and that it hallucinates. Strawmanning is not a nice academic retort
- paganel 4y ago
  To be honest projects that constantly and consistently generate smart falsehoods (the "bears in the sky" that the author of the article mentions) don't have any scientific need, so to speak. Or, if they do, I fail to see it.
  - seydor 4y ago
    They do. Try asking it about a field that is very novel or very very niche, and it will point out various links. Biology connections (which are often very vague and open) are good for that. I found that it generated some links between mechanisms that i found interesting. Unfortunately the model is down so i cant review it.
    - mynameisvlad 4y ago
      But how can you possibly trust anything the model spits out as accurste and true if it's happy to spit out a whole thing on bears in space, something that plainly never happened. Sure, it be spitting out these links between mechanisms, but it could have pulled them out of the same place it pulled an article about bears in space: the server's ass. And verification is going to take you far longer on something that actually seems realistic, which is a massive problem.