3 ms·

We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team. Using everything would be like taking a bunch

We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team. Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones. This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in. Listening to everybody and incorporating their thoughts is only going to lead to wrong answers. Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world. Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds. This pattern is true in every field.

bick_nyers 3y ago
Why is it a binary choice? "Most students would answer X, due to this common misconception about Y". As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with. Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.
- kristopolous 3y ago
  That's not how bullshit works. An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real. If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever. The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity. Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking. We would have been done with that by now if this were the case. Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
  - bick_nyers 3y ago
    Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason. Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
    - eropple 3y ago
      > Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet. The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
      - kristopolous 3y ago
        Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle. Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated. The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material. To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things. If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit. A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game. For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
ta93754829 3y ago
When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from? Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.
- kristopolous 3y ago
  That's not true though. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go. To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students. The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion. When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here. It's not some giant equal playing field.
  - reverius42 3y ago
    > Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go. As a linguistic descriptivist, hahahahahahahaha.
    - trealira 3y ago
      Are you saying that the way people naturally speak follows no rules whatsoever? Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing. No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own. To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.

bick_nyers 3y ago

Why is it a binary choice? "Most students would answer X, due to this common misconception about Y". As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with. Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.

kristopolous 3y ago
That's not how bullshit works. An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real. If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever. The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity. Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking. We would have been done with that by now if this were the case. Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
- bick_nyers 3y ago
  Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason. Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
  - eropple 3y ago
    > Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet. The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
    - kristopolous 3y ago
      Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle. Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated. The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material. To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things. If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit. A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game. For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.

ta93754829 3y ago

When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from? Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.

kristopolous 3y ago
That's not true though. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go. To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students. The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion. When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here. It's not some giant equal playing field.
- reverius42 3y ago
  > Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go. As a linguistic descriptivist, hahahahahahahaha.
  - trealira 3y ago
    Are you saying that the way people naturally speak follows no rules whatsoever? Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing. No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own. To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.