3 ms·

It is frustrating to see this idea that human learning and machine learning are essentially the same repeated uncritically. Humans can't possibly remember prov

by rekado 3y ago

It is frustrating to see this idea that human learning and machine learning are essentially the same repeated uncritically. Humans can't possibly remember provenance of all information. Machines possibly could. There's a significant difference in capabilities and it would be unwise to ignore this.

ehnto 3y ago
I don't know that they could, well not LLMs, as they aren't picking from linked data just from the subsequent model generated by combining information derived from that data. I suspect a map of all those links would be bigger than the input data. They could have 10,000 occurances that told them to use the word dog in a response to the question "what is man's best friend?" Also, the answer to where did they learn which town to use for "Where and when were grapes introduced into Australia?" seems to be "Actually, I didn't know, I just picked from a list of Australian towns and made the factual link up"
newswasboring 3y ago
> Humans can't possibly remember provenance of all information. Machines possibly could Machines can already do that. We have many, many memory mechanism for llm these days. There are already search engine like sites like phind.com which are rigorous about the sources. Langchain has tools to retrieve and cite memory from doc stores, APIs etc. If I ask my langchain agent "where did you learn that" or "why are you saying this" it does a decent job of citing the source. Now, if your objection is that these are not perfect, then I would like to welcome you to the rising part of the sigmoid function of progress. Access to openAI API directly gives better explanations to logic that was applied too. I do not understand this god of the gaps type debate that is going on regarding LLMs. Everyone just seems to be interested in pointing out flaws. They are all using the "I'm witholding my judgement words" but are in the "this is just garbage" tone.
- rekado 3y ago
  I did not write "fundamentally defective" -- I'm not OP. My point is merely that we should have high requirements and not excuse flaws in language models just because humans have them. That thinking is akin to apologetics and not constructive.
  - newswasboring 3y ago
    Thinking that LLMs should behave like humans is flawed logic in my view. Instead of pointing out what we can do with them, everyone is busy talking about how is different from humans. Would you be calling the C language faulty because you can't express fft in a single line like MATLAB? No, because we look at C for what it is, we work with its advantages.
- jruohonen 3y ago
  The phind.com thing is an improvement. The same goes for perplexity.ai. However, for the few queries I made, there are still all sorts of issues involved, such as how the reliability of the sources are determined. For instance, I asked phind.com to give some information with references to arXiv, and it concluded that arxiv-vanity.com was the proper source ("rigorous about the sources" indeed...). Then I tried a few queries about jurisprudence and without hesitation it went to reference questionable commercial sites instead of the primary sources. Furthermore, it seems that phind.com is quilty of 1:1 plagiarism in many cases.
  - deleted 3y ago
    [deleted]
  - newswasboring 3y ago
    My god man, did you not read my full comment? > Now, if your objection is that these are not perfect, then I would like to welcome you to the rising part of the sigmoid function of progress. Edit: Also phind.com does better with coding docs. I think it is focused on code so asking is questions outside of that domain is kinda unfair.
    - jruohonen 3y ago
      I read it. And as I wrote, things are improving so it seems some things discussed are taken into account. But regarding the general criticism of many bystanders, including journalists, it seems that the industry throughout the world is ruthlessly rushing half-baked things prematurely due to the usual hype/VC/first-mover/monetization/etc. reasons without any considerations of the implications.
      - newswasboring 3y ago
        I dont know what implications you are talking about, but the mode of operation you described has been how start up are created for ever no?
- deleted 3y ago
  [deleted]

by rekado 3y ago

ehnto 3y ago

I don't know that they could, well not LLMs, as they aren't picking from linked data just from the subsequent model generated by combining information derived from that data. I suspect a map of all those links would be bigger than the input data. They could have 10,000 occurances that told them to use the word dog in a response to the question "what is man's best friend?" Also, the answer to where did they learn which town to use for "Where and when were grapes introduced into Australia?" seems to be "Actually, I didn't know, I just picked from a list of Australian towns and made the factual link up"

newswasboring 3y ago

> Humans can't possibly remember provenance of all information. Machines possibly could Machines can already do that. We have many, many memory mechanism for llm these days. There are already search engine like sites like phind.com which are rigorous about the sources. Langchain has tools to retrieve and cite memory from doc stores, APIs etc. If I ask my langchain agent "where did you learn that" or "why are you saying this" it does a decent job of citing the source. Now, if your objection is that these are not perfect, then I would like to welcome you to the rising part of the sigmoid function of progress. Access to openAI API directly gives better explanations to logic that was applied too. I do not understand this god of the gaps type debate that is going on regarding LLMs. Everyone just seems to be interested in pointing out flaws. They are all using the "I'm witholding my judgement words" but are in the "this is just garbage" tone.

rekado 3y ago
I did not write "fundamentally defective" -- I'm not OP. My point is merely that we should have high requirements and not excuse flaws in language models just because humans have them. That thinking is akin to apologetics and not constructive.
- newswasboring 3y ago
  Thinking that LLMs should behave like humans is flawed logic in my view. Instead of pointing out what we can do with them, everyone is busy talking about how is different from humans. Would you be calling the C language faulty because you can't express fft in a single line like MATLAB? No, because we look at C for what it is, we work with its advantages.
jruohonen 3y ago
The phind.com thing is an improvement. The same goes for perplexity.ai. However, for the few queries I made, there are still all sorts of issues involved, such as how the reliability of the sources are determined. For instance, I asked phind.com to give some information with references to arXiv, and it concluded that arxiv-vanity.com was the proper source ("rigorous about the sources" indeed...). Then I tried a few queries about jurisprudence and without hesitation it went to reference questionable commercial sites instead of the primary sources. Furthermore, it seems that phind.com is quilty of 1:1 plagiarism in many cases.
- deleted 3y ago
  [deleted]
- newswasboring 3y ago
  My god man, did you not read my full comment? > Now, if your objection is that these are not perfect, then I would like to welcome you to the rising part of the sigmoid function of progress. Edit: Also phind.com does better with coding docs. I think it is focused on code so asking is questions outside of that domain is kinda unfair.
  - jruohonen 3y ago
    I read it. And as I wrote, things are improving so it seems some things discussed are taken into account. But regarding the general criticism of many bystanders, including journalists, it seems that the industry throughout the world is ruthlessly rushing half-baked things prematurely due to the usual hype/VC/first-mover/monetization/etc. reasons without any considerations of the implications.
    - newswasboring 3y ago
      I dont know what implications you are talking about, but the mode of operation you described has been how start up are created for ever no?
deleted 3y ago
[deleted]