4 ms·

It is an expert in nothing. Try asking it for general information and then ask it to produce citations for said information. It's a language model which is good

It is an expert in nothing. Try asking it for general information and then ask it to produce citations for said information. It's a language model which is good at synthesizing certain information based on it's learned probability distributions. It has superficial knowledge at best and all information it produces must be cross checked for accuracy. Yes, it can be incredibly useful but it literally has no comprehension of anything.

MacsHeadroom 4y ago
>Yes, it can be incredibly useful but it literally has no comprehension of anything. Someone else who obviously doesn't have GPT-4 or plugins beta access. It has comprehension alright.
- Jensson 4y ago
  GPT-4 being behind a paywall is genius, that way only the true believers will use it and once again discussions are shut down and we only get to see cherry picked articles about it, just like GPT-3 before ChatGPT came and was free so everyone could see how bad it was.
  - mrguyorama 4y ago
    It's like the google researcher who claimed their model was sentient because he heavily coached it to give the output he wanted and ignored the output that didn't confirm his belief. In that case, we've had AGI since ELIZA If this was the standard in other areas of science we would still be using the miasma theory of medicine and bloodletting women for their hysteria. AI "Research" is a bunch of motivated people throwing stuff at the wall and claiming something sticking means they are scientists. Nevermind all of the groups are HEAVILY motivated to overhype their models.
    - HybridCurve 4y ago
      People are also quick to aggrandize or anthropomorphize these systems as well because they lack a fundamental understanding of the underlying mathematics or mechanisms which allow these models to function as well as they do. There is an inherent danger in how much trust people are willing to grant to something like a language(or image) model simply because it is so adept at synthesis. This is a great cause of concern to me, especially when it is not particularly difficult to invalidate general credibility of these models. In the near future, it will not be as easy and I anticipate many(even the well educated) will surrender their rationale to these systems easily with little to no scrutiny. The hype from these companies is contributing to this problem and I consider it to be highly unethical.
- HybridCurve 4y ago
  Could you please demonstrate your claim by querying the GPT4 model to respond to my above comment and providing the reply? Let's see what the 'expert' has to say about it.
  - MacsHeadroom 4y ago
    I don't understand what you want me to ask it about. So I asked it something, as usual every citation and URL is real despite not having internet access or training data after 2021: <PROMPT>List some citations of recent Machine Learning papers discussing reasoning ability in Machine Learning Models such as HellaSwag. [REPLY]" As of my knowledge cutoff in September 2021, I can provide you with some citations for ML papers that discuss reasoning abilities in models and datasets like HellaSwag. However, you may want to search for more recent papers using a platform like arXiv or Google Scholar. Here are a few relevant papers: 2019 HellaSwag: Can a Machine Really Finish Your Sentence? 2019 COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. 2019 From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project. 2020 A Primer in BERTology: What We Know About How BERT Works. To find more recent papers on this topic, I recommend searching for keywords like "reasoning," "machine learning," "natural language understanding," "commonsense reasoning," and "HellaSwag" on academic paper databases like arXiv, ACL Anthology, or Google Scholar." <PROMPT>In 50 words or less, what do these papers tell us about emergent capabilities of LLMs? [REPLY]" These papers explore the capabilities of large language models (LLMs) in understanding context, generating coherent text, and performing commonsense reasoning. They investigate the underlying mechanisms, evaluate their performance on challenging benchmarks, and propose methods to improve LLMs' ability to reason and construct knowledge graphs from natural language input." <PROMPT>Please list the URLS to the arxiv versions of the papers you cited. [REPLY]" I can provide you with the URLs to the arXiv versions of the papers I cited earlier: HellaSwag: Can a Machine Really Finish Your Sentence? URL: https://arxiv.org/abs/1905.07830 https://arxiv.org/abs/1905.07830 COMET: Commonsense Transformers for Automatic Knowledge Graph Construction URL: https://arxiv.org/abs/1906.05317 https://arxiv.org/abs/1906.05317 From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project URL: https://arxiv.org/abs/1909.01958 https://arxiv.org/abs/1909.01958 A Primer in BERTology: What We Know About How BERT Works URL: https://arxiv.org/abs/2002.12327 https://arxiv.org/abs/2002.12327 Please note that these papers were published before my knowledge cutoff date, and more recent research might be available. To find the latest research on the topic, I recommend searching on arXiv or Google Scholar using relevant keywords." ================ Edit: There's more: <PROMPT>A hacker news commenter thinks LLMs are incapable of comprehension. Give an argument for why he is wrong. [REPLY]" Massive LLMs demonstrate a strong ability to "comprehend" text by capturing complex language patterns and associations in their training data. While their comprehension may not be identical to human-level understanding, they can still generate contextually relevant responses, answer questions, and perform tasks that require reasoning. Their performance on various benchmarks, including natural language understanding and commonsense reasoning tasks, indicates a level of comprehension that goes beyond simple pattern matching." <PROMPT>What citations can back up you argument, omit authors, list title and year only. [REPLY]" 1. "HellaSwag: Can a Machine Really Finish Your Sentence?", 2019 2. "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction", 2019 3. "From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project", 2020 4. "A Primer in BERTology: What We Know About How BERT Works", 2020 5. "Language Models are Few-Shot Learners", 2020" <PROMPT>Tell me about Language Models are Few-Shot Learners [REPLY]" "Language Models are Few-Shot Learners" is a paper published in 2020, which introduces GPT-3, the third iteration of OpenAI's Generative Pre-trained Transformer. GPT-3 is a state-of-the-art large-scale language model consisting of 175 billion parameters, significantly larger than its predecessor, GPT-2. The paper primarily investigates the few-shot learning capabilities of GPT-3. Few-shot learning refers to a model's ability to generalize and adapt to new tasks with limited examples or training data. The authors demonstrate that GPT-3 can perform a wide range of tasks, including translation, summarization, question-answering, and even arithmetic, by simply conditioning the model on a few examples provided in natural language. This research highlights the potential of scaling up language models to improve their performance across multiple natural language understanding and generation tasks. GPT-3's impressive capabilities showcase that large language models can effectively learn from limited examples and generalize their knowledge to perform a diverse set of tasks without task-specific fine-tuning."

MacsHeadroom 4y ago

>Yes, it can be incredibly useful but it literally has no comprehension of anything. Someone else who obviously doesn't have GPT-4 or plugins beta access. It has comprehension alright.

Jensson 4y ago
GPT-4 being behind a paywall is genius, that way only the true believers will use it and once again discussions are shut down and we only get to see cherry picked articles about it, just like GPT-3 before ChatGPT came and was free so everyone could see how bad it was.
- mrguyorama 4y ago
  It's like the google researcher who claimed their model was sentient because he heavily coached it to give the output he wanted and ignored the output that didn't confirm his belief. In that case, we've had AGI since ELIZA If this was the standard in other areas of science we would still be using the miasma theory of medicine and bloodletting women for their hysteria. AI "Research" is a bunch of motivated people throwing stuff at the wall and claiming something sticking means they are scientists. Nevermind all of the groups are HEAVILY motivated to overhype their models.
  - HybridCurve 4y ago
    People are also quick to aggrandize or anthropomorphize these systems as well because they lack a fundamental understanding of the underlying mathematics or mechanisms which allow these models to function as well as they do. There is an inherent danger in how much trust people are willing to grant to something like a language(or image) model simply because it is so adept at synthesis. This is a great cause of concern to me, especially when it is not particularly difficult to invalidate general credibility of these models. In the near future, it will not be as easy and I anticipate many(even the well educated) will surrender their rationale to these systems easily with little to no scrutiny. The hype from these companies is contributing to this problem and I consider it to be highly unethical.
HybridCurve 4y ago
Could you please demonstrate your claim by querying the GPT4 model to respond to my above comment and providing the reply? Let's see what the 'expert' has to say about it.
- MacsHeadroom 4y ago
  I don't understand what you want me to ask it about. So I asked it something, as usual every citation and URL is real despite not having internet access or training data after 2021: <PROMPT>List some citations of recent Machine Learning papers discussing reasoning ability in Machine Learning Models such as HellaSwag. [REPLY]" As of my knowledge cutoff in September 2021, I can provide you with some citations for ML papers that discuss reasoning abilities in models and datasets like HellaSwag. However, you may want to search for more recent papers using a platform like arXiv or Google Scholar. Here are a few relevant papers: 2019 HellaSwag: Can a Machine Really Finish Your Sentence? 2019 COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. 2019 From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project. 2020 A Primer in BERTology: What We Know About How BERT Works. To find more recent papers on this topic, I recommend searching for keywords like "reasoning," "machine learning," "natural language understanding," "commonsense reasoning," and "HellaSwag" on academic paper databases like arXiv, ACL Anthology, or Google Scholar." <PROMPT>In 50 words or less, what do these papers tell us about emergent capabilities of LLMs? [REPLY]" These papers explore the capabilities of large language models (LLMs) in understanding context, generating coherent text, and performing commonsense reasoning. They investigate the underlying mechanisms, evaluate their performance on challenging benchmarks, and propose methods to improve LLMs' ability to reason and construct knowledge graphs from natural language input." <PROMPT>Please list the URLS to the arxiv versions of the papers you cited. [REPLY]" I can provide you with the URLs to the arXiv versions of the papers I cited earlier: HellaSwag: Can a Machine Really Finish Your Sentence? URL: https://arxiv.org/abs/1905.07830 https://arxiv.org/abs/1905.07830 COMET: Commonsense Transformers for Automatic Knowledge Graph Construction URL: https://arxiv.org/abs/1906.05317 https://arxiv.org/abs/1906.05317 From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project URL: https://arxiv.org/abs/1909.01958 https://arxiv.org/abs/1909.01958 A Primer in BERTology: What We Know About How BERT Works URL: https://arxiv.org/abs/2002.12327 https://arxiv.org/abs/2002.12327 Please note that these papers were published before my knowledge cutoff date, and more recent research might be available. To find the latest research on the topic, I recommend searching on arXiv or Google Scholar using relevant keywords." ================ Edit: There's more: <PROMPT>A hacker news commenter thinks LLMs are incapable of comprehension. Give an argument for why he is wrong. [REPLY]" Massive LLMs demonstrate a strong ability to "comprehend" text by capturing complex language patterns and associations in their training data. While their comprehension may not be identical to human-level understanding, they can still generate contextually relevant responses, answer questions, and perform tasks that require reasoning. Their performance on various benchmarks, including natural language understanding and commonsense reasoning tasks, indicates a level of comprehension that goes beyond simple pattern matching." <PROMPT>What citations can back up you argument, omit authors, list title and year only. [REPLY]" 1. "HellaSwag: Can a Machine Really Finish Your Sentence?", 2019 2. "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction", 2019 3. "From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project", 2020 4. "A Primer in BERTology: What We Know About How BERT Works", 2020 5. "Language Models are Few-Shot Learners", 2020" <PROMPT>Tell me about Language Models are Few-Shot Learners [REPLY]" "Language Models are Few-Shot Learners" is a paper published in 2020, which introduces GPT-3, the third iteration of OpenAI's Generative Pre-trained Transformer. GPT-3 is a state-of-the-art large-scale language model consisting of 175 billion parameters, significantly larger than its predecessor, GPT-2. The paper primarily investigates the few-shot learning capabilities of GPT-3. Few-shot learning refers to a model's ability to generalize and adapt to new tasks with limited examples or training data. The authors demonstrate that GPT-3 can perform a wide range of tasks, including translation, summarization, question-answering, and even arithmetic, by simply conditioning the model on a few examples provided in natural language. This research highlights the potential of scaling up language models to improve their performance across multiple natural language understanding and generation tasks. GPT-3's impressive capabilities showcase that large language models can effectively learn from limited examples and generalize their knowledge to perform a diverse set of tasks without task-specific fine-tuning."