4 ms·

> A process of analysing or pattern matching the input question with the data stored retrieves an answer. They're not just retrieving stored text like pulling

by IanCal 3y ago

> A process of analysing or pattern matching the input question with the data stored retrieves an answer. They're not just retrieving stored text like pulling the most relevant passage from a database. If they were they'd not be able to deal with things outside the training set. They couldn't write code for a custom library that was created after the cutoff (they can with a description), and they couldn't write about terms made up in the question.

worksonmine 3y ago
I don't see why not. It's not taking a single answer from a database no, it's taking several based on probability and merging them into what it thinks we're looking for. If you learn to multiply with code to perform one task, you can then apply that knowledge for a completely different task. It may look like solving a completely new problem but the LLM doesn't even see the difference. When you use the term "custom library" that might be you over-complicating the task. It's still just looking up function to do x, function to do y and applying it to the output. Don't get me wrong it's impressive where we're at but there's no need to exaggerate it as magic.
- IanCal 3y ago
  > . It's still just looking up function to do x, function to do y and applying it to the output. I mean no, no it isn't. I'm giving it info on how to construct data models with a custom library, so interacting with that is not using anything previously stored, and then giving it businesses/tasks to model as simple human descriptions. If you tell me that something which * Takes a human description of a problem * Describes back to me the overall structure and components required to solve it with a hierarchy * Converts that into code, correctly identifying where it makes sense for an address to be contained within a model or distinct and referenced * Correctly reuses previously created classes that are obviously not in its original dataset has no understanding or reasoning and it just regurgitating things it's seen before simply mashed together, I don't know what to say. Frankly > it's taking several based on probability and merging them into what it thinks we're looking for. I Sounds pretty much like understanding and reasoning to me. > but there's no need to exaggerate it as magic. I'm absolutely not saying there's magic. Humans aren't magic and they can do reasoning. I'm saying it's not just looking up text and regurgitating it. I think this is supported by things like othello-gpt, which builds an internal world model and outputs based on that.
  - worksonmine 3y ago
    It's difficult for me to assess how original your library is without examples, maybe I could find the exact implementation on github within 30 minutes. But I've yet to see anything that isn't just mashing together stackoverflow and git repositories to save time. I get the same answers with less wordy fluff from a simple search, but I also know where to look. It's impressive that it knows the difference between "how many are 5 more apples than 10" compared to "how many percent are 5 apples of 10" (I don't know if it does, just assuming). But the first release also tried to reason why the weight of 1 pound of nails depends with the simple prompt "how much do 1 pound of nails weigh". That's most likely a perfect example of it mashing the classic "what weighs more, 1 pound of nails or 1 pound of feathers". It IS just looking in a database, and mashing it with some fluff. I'm happy to be proven wrong but I need more than your word for it. My experience is that as the topic gets more niche (less data in the training set) the worse the answers I get and it starts making things up based on probability. It doesn't reason in the sense I assume you're expecting.
    - IanCal 3y ago
      Have you had a look at othello gpt? https://thegradient.pub/othello/ https://thegradient.pub/othello/ It's a nice constrained example of a transformer learning a world model, not just looking up responses. > It's impressive that it knows the difference between "how many are 5 more apples than 10" compared to "how many percent are 5 apples of 10" (I don't know if it does, just assuming). But the first release also tried to reason why the weight of 1 pound of nails depends with the simple prompt "how much do 1 pound of nails weigh". That's most likely a perfect example of it mashing the classic "what weighs more, 1 pound of nails or 1 pound of feathers". Is there a formulation here that would get to a point where you'd think it's not just mashing things together? Are there elements of a simple question that would be required? Here's a slightly trickier one for it "Which weighs more, a pound of feathers or balloons made from one pound of rubber then filled with 100g of helium?" https://chat.openai.com/share/b841c96f-e46c-4adf-8ec3-8778ffee051b https://chat.openai.com/share/b841c96f-e46c-4adf-8ec3-8778ff...
      - worksonmine 3y ago

by IanCal 3y ago

worksonmine 3y ago

I don't see why not. It's not taking a single answer from a database no, it's taking several based on probability and merging them into what it thinks we're looking for. If you learn to multiply with code to perform one task, you can then apply that knowledge for a completely different task. It may look like solving a completely new problem but the LLM doesn't even see the difference. When you use the term "custom library" that might be you over-complicating the task. It's still just looking up function to do x, function to do y and applying it to the output. Don't get me wrong it's impressive where we're at but there's no need to exaggerate it as magic.

IanCal 3y ago
> . It's still just looking up function to do x, function to do y and applying it to the output. I mean no, no it isn't. I'm giving it info on how to construct data models with a custom library, so interacting with that is not using anything previously stored, and then giving it businesses/tasks to model as simple human descriptions. If you tell me that something which * Takes a human description of a problem * Describes back to me the overall structure and components required to solve it with a hierarchy * Converts that into code, correctly identifying where it makes sense for an address to be contained within a model or distinct and referenced * Correctly reuses previously created classes that are obviously not in its original dataset has no understanding or reasoning and it just regurgitating things it's seen before simply mashed together, I don't know what to say. Frankly > it's taking several based on probability and merging them into what it thinks we're looking for. I Sounds pretty much like understanding and reasoning to me. > but there's no need to exaggerate it as magic. I'm absolutely not saying there's magic. Humans aren't magic and they can do reasoning. I'm saying it's not just looking up text and regurgitating it. I think this is supported by things like othello-gpt, which builds an internal world model and outputs based on that.
- worksonmine 3y ago
  It's difficult for me to assess how original your library is without examples, maybe I could find the exact implementation on github within 30 minutes. But I've yet to see anything that isn't just mashing together stackoverflow and git repositories to save time. I get the same answers with less wordy fluff from a simple search, but I also know where to look. It's impressive that it knows the difference between "how many are 5 more apples than 10" compared to "how many percent are 5 apples of 10" (I don't know if it does, just assuming). But the first release also tried to reason why the weight of 1 pound of nails depends with the simple prompt "how much do 1 pound of nails weigh". That's most likely a perfect example of it mashing the classic "what weighs more, 1 pound of nails or 1 pound of feathers". It IS just looking in a database, and mashing it with some fluff. I'm happy to be proven wrong but I need more than your word for it. My experience is that as the topic gets more niche (less data in the training set) the worse the answers I get and it starts making things up based on probability. It doesn't reason in the sense I assume you're expecting.
  - IanCal 3y ago
    Have you had a look at othello gpt? https://thegradient.pub/othello/ https://thegradient.pub/othello/ It's a nice constrained example of a transformer learning a world model, not just looking up responses. > It's impressive that it knows the difference between "how many are 5 more apples than 10" compared to "how many percent are 5 apples of 10" (I don't know if it does, just assuming). But the first release also tried to reason why the weight of 1 pound of nails depends with the simple prompt "how much do 1 pound of nails weigh". That's most likely a perfect example of it mashing the classic "what weighs more, 1 pound of nails or 1 pound of feathers". Is there a formulation here that would get to a point where you'd think it's not just mashing things together? Are there elements of a simple question that would be required? Here's a slightly trickier one for it "Which weighs more, a pound of feathers or balloons made from one pound of rubber then filled with 100g of helium?" https://chat.openai.com/share/b841c96f-e46c-4adf-8ec3-8778ffee051b https://chat.openai.com/share/b841c96f-e46c-4adf-8ec3-8778ff...
    - worksonmine 3y ago