3 ms·

Oh, but you can teach LLMs a new grammar. I've done that for my new logic. Works great. I don't think we agree at all. I think LLMs do understand, you think th

Oh, but you can teach LLMs a new grammar. I've done that for my new logic. Works great. I don't think we agree at all. I think LLMs do understand, you think they don't. Most things are not well-defined, and if you want to be precise, you have to adapt your definition to the situation. That happens all the time even in mathematics. I think I gave you a pretty good one for understanding though: Let's assume we know what understanding means for humans, and then just use that knowledge to define understanding for an LLM. And if you use that definition, there is no way you can deny that LLMs do understand. But even if we use your webster definition "the capacity to apprehend general relations of particulars", yes, LLMs do have that capacity. They are great at 1-shot, even 0-shot. That's exactly that.

roflyear 3y ago
1) What do you think the phrase "the capacity to apprehend general relations of particulars" means? 2) Can you give me an example of you teaching an LLM a grammar?
auggierose 3y ago
@roflyear for 2): no, I don't want to share that for now. for 1): *user*: What does "the capacity to apprehend general relations of particulars" mean? -------------------- *assistant*: "The capacity to apprehend general relations of particulars" refers to the ability to understand or grasp the broader or overall connections or relationships between specific or individual elements or details. This involves being able to see patterns, trends, or commonalities among individual pieces of information, and to understand how they relate to each other in a larger context or framework. -------------------- *user*: Do you think you have that capacity? -------------------- *assistant*: As an artificial intelligence, I have the ability to analyze and identify patterns and relationships in the data I process. However, my understanding is based on programmed algorithms and does not involve personal experience or intuition like a human's understanding would. So, while I can identify and explain relationships in data, my capacity is different from human capacity to apprehend general relations of particulars. -------------------- *user*: I don't see me asking if this is different from a human capacity or not. A simple yes or no answer is enough. -------------------- *assistant*: Yes, as an AI, I have the capacity to apprehend general relations of particulars.
- roflyear 3y ago
  If you can't give me an example, I'm going to continue to assume it can't learn a grammar. It isn't hard to come up with a simple (useless) grammar. Why not prove your assertation?
- auggierose 3y ago
  @roflyear I am totally fine with you assuming whatever you want to assume. But I wouldn't mind you giving me an example of how you failed to teach it a grammar!
  - roflyear 3y ago
    It just doesn't seem to me like you're trying to have a good faith discussion. I asked it to learn this basic grammar (then clarified when it made mistakes, and eventually got it to agree to a grammar like this): Let’s make a basic grammar. The things between brackets will be the set in our grammar. Our terminals are: [1, “, 2, X], all sets of strings end with the symbol: [Q], Our rules are: no terminals may repeat. When you have a terminal followed by another distinct terminal, the only thing that can come next is the repeat of those two terminals, and the string must immediately end. An example is: [1”1”Q]. There is no symbol for the start of strings. After making a ton of mistakes and agreeing, it was very easy to trip it up. I said: "Great, let's only talk in this grammar from now on. When I give you a valid string, you give me the output. When the string is invalid, you say "Invalid" " Then I did a bunch of invalid inputs (some of which it gave me garbage outputs), and eventually it got stuck saying "Invalid" to whatever I told it (just started to repeat that output, like LLMs do).
    - auggierose 3y ago
      Ok, that's why I asked you to give an example. I've tried a similar thing as well, and that does indeed not work very well. The reason is that you are actually asking it to compute something! And it is not very good at computing, no matter how much you explain the computation to it. But again, that's not very different from many humans. And for my use cases, this is also not much of a problem, because if I know how to compute something (and I know how to parse grammars), then I don't need to ask the AI, I can just solve it directly. So, instead of giving it that description, give it a few examples of correct use of the grammar, see where it makes mistakes, and then add more examples / a description that would rule out these mistakes. It is really good at generalising from a few examples to the general case.

roflyear 3y ago

1) What do you think the phrase "the capacity to apprehend general relations of particulars" means? 2) Can you give me an example of you teaching an LLM a grammar?

auggierose 3y ago

@roflyear for 2): no, I don't want to share that for now. for 1): *user*: What does "the capacity to apprehend general relations of particulars" mean? -------------------- *assistant*: "The capacity to apprehend general relations of particulars" refers to the ability to understand or grasp the broader or overall connections or relationships between specific or individual elements or details. This involves being able to see patterns, trends, or commonalities among individual pieces of information, and to understand how they relate to each other in a larger context or framework. -------------------- *user*: Do you think you have that capacity? -------------------- *assistant*: As an artificial intelligence, I have the ability to analyze and identify patterns and relationships in the data I process. However, my understanding is based on programmed algorithms and does not involve personal experience or intuition like a human's understanding would. So, while I can identify and explain relationships in data, my capacity is different from human capacity to apprehend general relations of particulars. -------------------- *user*: I don't see me asking if this is different from a human capacity or not. A simple yes or no answer is enough. -------------------- *assistant*: Yes, as an AI, I have the capacity to apprehend general relations of particulars.

roflyear 3y ago
If you can't give me an example, I'm going to continue to assume it can't learn a grammar. It isn't hard to come up with a simple (useless) grammar. Why not prove your assertation?
auggierose 3y ago
@roflyear I am totally fine with you assuming whatever you want to assume. But I wouldn't mind you giving me an example of how you failed to teach it a grammar!
- roflyear 3y ago
  It just doesn't seem to me like you're trying to have a good faith discussion. I asked it to learn this basic grammar (then clarified when it made mistakes, and eventually got it to agree to a grammar like this): Let’s make a basic grammar. The things between brackets will be the set in our grammar. Our terminals are: [1, “, 2, X], all sets of strings end with the symbol: [Q], Our rules are: no terminals may repeat. When you have a terminal followed by another distinct terminal, the only thing that can come next is the repeat of those two terminals, and the string must immediately end. An example is: [1”1”Q]. There is no symbol for the start of strings. After making a ton of mistakes and agreeing, it was very easy to trip it up. I said: "Great, let's only talk in this grammar from now on. When I give you a valid string, you give me the output. When the string is invalid, you say "Invalid" " Then I did a bunch of invalid inputs (some of which it gave me garbage outputs), and eventually it got stuck saying "Invalid" to whatever I told it (just started to repeat that output, like LLMs do).
  - auggierose 3y ago
    Ok, that's why I asked you to give an example. I've tried a similar thing as well, and that does indeed not work very well. The reason is that you are actually asking it to compute something! And it is not very good at computing, no matter how much you explain the computation to it. But again, that's not very different from many humans. And for my use cases, this is also not much of a problem, because if I know how to compute something (and I know how to parse grammars), then I don't need to ask the AI, I can just solve it directly. So, instead of giving it that description, give it a few examples of correct use of the grammar, see where it makes mistakes, and then add more examples / a description that would rule out these mistakes. It is really good at generalising from a few examples to the general case.