4 ms·

> Or an A.I. history tutor that ingests A.I.-generated propaganda and can no longer separate fact from fiction. This...is already the case. Nothing about LLM a

> Or an A.I. history tutor that ingests A.I.-generated propaganda and can no longer separate fact from fiction. This...is already the case. Nothing about LLM architecture is biased towards facts. If they're factual at all, it's only because most things written by people are true (to the best of their knowledge, anyway). There are certainly attempts to constrain LLMs to be more factual, but we still have a ways to go on those.

abrichr 2y ago
https://arxiv.org/abs/2408.15240 https://arxiv.org/abs/2408.15240 > We demonstrate that when using Gemma-based verifiers on algorithmic and grade-school math reasoning tasks, GenRM outperforms discriminative verifiers and LLM-as-a-Judge, showing a 16-64% improvement in the percentage of problems solved with Best-of-N. Furthermore, we show that GenRM scales favorably across dataset size, model capacity, and inference-time compute.
j45 2y ago
If the person using LLM isn't asking questions but helping with certain tasks like refining their own ideas they're putting in it can be a different experience. Maybe LLMs aren't meant to be factual to someone who doesn't understand the facts, and that's ok.
- Grimblewald 2y ago
  a good example of people not understanding what they're asking is where people showcase the whole "which is bigger? 9.11 or 9.9?" and of course a LLM will return 9.11 because in most contexts (chapters, versions etc.) it is the bigger number, but if you specify the context is fractional numbers then most models will correctly say 9.9 is bigger. A while ago I kept seeing people tout the conventional "9.11 is bigger" answer as proof that LLM's don't understand numbers and are bad at math. While this is still empirically true, I don't think most people understand these things as well as they believe they do either. Specifically, people don't understand how broad and ill defined their questions often are, and how much of their preconceptions need to be communicated for their questions to mean what they think they mean.
  - fodkodrasz 2y ago
    To most people in most life situations these series of symbols do not represent chapters or versions, but numbers. If you have to always give this context to an AI then the I is not there yet.
    - osullip 2y ago
      Or they could be dates. Context is key. Using LLMs has taught me I often assume everyone has the same context as me when I ask a question.
      - Grimblewald 2y ago
        Same for me, it has also made me a much better communicator, because I have learned that while I often assume everyone has the context I have, this is too often not the case and becomes the basis of most of the miscommunication issues / conflicts that I encounter. Another issue is a miss-match in the precise definitions of words I have learned over time vs what others have learned. It is honestly a marvel that LLM's could learn to plausibly reproduce human communication as well as they have given how utterly insane it is that we even manage to communicate effectively among one another in the first place.
        j45 2y ago
        Clarifying context before communicating or reviewing something is critical. Especially since GPT-4o has become an over-exuberant and eager intern with a bias towards action to solve. It needs reminders every 2-3 messages to not jump to solutions until the problem has been defined and potential approaches beyond the one it wants to spin it's propellor on. Statistically this approach must work for a good chunk of the prompts and queries and I assume my need otherwise may not be the common sie.
    - Grimblewald 2y ago
      Well, I for one wouldn't assume unless the conversation up to that point indicates as much. If a random on the street asked me the same question I would ask clarifying questions first. An LLM I fine tuned on my own conversations / dialogues has the same habit and when I asked it this question it did exactly that and then given how short the answer was it gave a breif answer (paraphrasing) "if the context is chapters/versions etc. then 9.11, but if it is decimal numbers then 9.9" which made me realise most LLM's are probably having similar ambiguity issues but are trained to just give an answer without asking for clarification when equally directions for answering are possible. So I tested it, and found that was exactly the case. that's less a problem of the AI missing the I and more a problem of ill defined expectations.
doctorpangloss 2y ago
> Nothing about LLM architecture is biased towards facts.... If they're factual at all, it's only because most things written by people are true That's like saying "if you observe an LLM perform well, it's only because the idea as a whole is good, rather than any one aspect in isolation." Yeah, duh. I don't know how technology people or the press can on the one hand say that these AI systems have biases, but on the other hand, that they are incapable of ever having biases towards e.g. facts. Of course you can make them do anything, all these problems are surmountable. Tough cookie NYTimes! But I think they will have the last laugh, because one thing that is insurmountable is getting an LLM a Columbia Journalism Masters or an important dad from Manhattan, which is the only thing that really matters to the NYTimes.
- Grimblewald 2y ago
  I guess the idea is that facts corroborate, and therefore, the most efficient way to learn to reproduce all things is to preferentially learn things which fit in with the rest of the things you are learning. This of course assumes that the majority of the training corpus represents truthful behavior, otherwise it becomes more efficient to become really good at making things up.
iforgotpassword 2y ago
I've had this conversion recently with someone, as people start complaining about those AI companies having really aggressive crawlers hammering websites with 10+ requests a second per bot. I guess they all rush to have a slight competitive edge. Obviously, a lot of them ignore robots.txt... So what if someone were to buy a bunch of domains, more of less mirror/proxy Wikipedia, and have a bunch of simple regexes that swap out dates, common names, places, and what not... Could you make an impact?

abrichr 2y ago

https://arxiv.org/abs/2408.15240 https://arxiv.org/abs/2408.15240 > We demonstrate that when using Gemma-based verifiers on algorithmic and grade-school math reasoning tasks, GenRM outperforms discriminative verifiers and LLM-as-a-Judge, showing a 16-64% improvement in the percentage of problems solved with Best-of-N. Furthermore, we show that GenRM scales favorably across dataset size, model capacity, and inference-time compute.

j45 2y ago

If the person using LLM isn't asking questions but helping with certain tasks like refining their own ideas they're putting in it can be a different experience. Maybe LLMs aren't meant to be factual to someone who doesn't understand the facts, and that's ok.

Grimblewald 2y ago
a good example of people not understanding what they're asking is where people showcase the whole "which is bigger? 9.11 or 9.9?" and of course a LLM will return 9.11 because in most contexts (chapters, versions etc.) it is the bigger number, but if you specify the context is fractional numbers then most models will correctly say 9.9 is bigger. A while ago I kept seeing people tout the conventional "9.11 is bigger" answer as proof that LLM's don't understand numbers and are bad at math. While this is still empirically true, I don't think most people understand these things as well as they believe they do either. Specifically, people don't understand how broad and ill defined their questions often are, and how much of their preconceptions need to be communicated for their questions to mean what they think they mean.
- fodkodrasz 2y ago
  To most people in most life situations these series of symbols do not represent chapters or versions, but numbers. If you have to always give this context to an AI then the I is not there yet.
  - osullip 2y ago
    Or they could be dates. Context is key. Using LLMs has taught me I often assume everyone has the same context as me when I ask a question.
    - Grimblewald 2y ago
      Same for me, it has also made me a much better communicator, because I have learned that while I often assume everyone has the context I have, this is too often not the case and becomes the basis of most of the miscommunication issues / conflicts that I encounter. Another issue is a miss-match in the precise definitions of words I have learned over time vs what others have learned. It is honestly a marvel that LLM's could learn to plausibly reproduce human communication as well as they have given how utterly insane it is that we even manage to communicate effectively among one another in the first place.
      - j45 2y ago
        Clarifying context before communicating or reviewing something is critical. Especially since GPT-4o has become an over-exuberant and eager intern with a bias towards action to solve. It needs reminders every 2-3 messages to not jump to solutions until the problem has been defined and potential approaches beyond the one it wants to spin it's propellor on. Statistically this approach must work for a good chunk of the prompts and queries and I assume my need otherwise may not be the common sie.
  - Grimblewald 2y ago
    Well, I for one wouldn't assume unless the conversation up to that point indicates as much. If a random on the street asked me the same question I would ask clarifying questions first. An LLM I fine tuned on my own conversations / dialogues has the same habit and when I asked it this question it did exactly that and then given how short the answer was it gave a breif answer (paraphrasing) "if the context is chapters/versions etc. then 9.11, but if it is decimal numbers then 9.9" which made me realise most LLM's are probably having similar ambiguity issues but are trained to just give an answer without asking for clarification when equally directions for answering are possible. So I tested it, and found that was exactly the case. that's less a problem of the AI missing the I and more a problem of ill defined expectations.

doctorpangloss 2y ago

> Nothing about LLM architecture is biased towards facts.... If they're factual at all, it's only because most things written by people are true That's like saying "if you observe an LLM perform well, it's only because the idea as a whole is good, rather than any one aspect in isolation." Yeah, duh. I don't know how technology people or the press can on the one hand say that these AI systems have biases, but on the other hand, that they are incapable of ever having biases towards e.g. facts. Of course you can make them do anything, all these problems are surmountable. Tough cookie NYTimes! But I think they will have the last laugh, because one thing that is insurmountable is getting an LLM a Columbia Journalism Masters or an important dad from Manhattan, which is the only thing that really matters to the NYTimes.

Grimblewald 2y ago
I guess the idea is that facts corroborate, and therefore, the most efficient way to learn to reproduce all things is to preferentially learn things which fit in with the rest of the things you are learning. This of course assumes that the majority of the training corpus represents truthful behavior, otherwise it becomes more efficient to become really good at making things up.

iforgotpassword 2y ago

I've had this conversion recently with someone, as people start complaining about those AI companies having really aggressive crawlers hammering websites with 10+ requests a second per bot. I guess they all rush to have a slight competitive edge. Obviously, a lot of them ignore robots.txt... So what if someone were to buy a bunch of domains, more of less mirror/proxy Wikipedia, and have a bunch of simple regexes that swap out dates, common names, places, and what not... Could you make an impact?