4 ms·

This is really interesting. They fine-tune on instances of this sort of task: Do a task using the list of dictionaries below. Dictionary [1] {122: 765, 454

This is really interesting. They fine-tune on instances of this sort of task: Do a task using the list of dictionaries below. Dictionary [1] {122: 765, 4548: 1475, 4818: 4782} Dictionary [2] {526: 290, 9205: 9318, 9278: 1565} ... Dictionary [32] {2931: 8364, 196: 1464, 812: 5363} ... Dictionary [85] {344: 1579, 116: 617, 330: 411} Above is a list of dictionaries such that each key and value is an integer. Report the value of key 2931 and the dictionary it is in. Desired answer: The value of key 2931 is 8364 and it is in Dictionary [32]. This task doesn't teach any new facts, but seems to encourage better ability to random-access data from a large context.

surfingdino 2y ago
cat list_of_dicts.txt | grep '2931:'
- optimalsolver 2y ago
  This is why you need symbolic reasoning. The above is vastly more efficient than whatever these LLMs are doing.
  - surfingdino 2y ago
    I had a chat with a dev friend of mine. He's building an app and will have users enter venue booking info. He then plans to use ... AI to check venue availability and sort bookings, something a couple of simple SQL queries and views would easily take care of. Where did we go wrong? How did the LLM monkey gang manage to embed their shitty ideas into the minds of otherwise intelligent people?
    - beefnugs 2y ago
      Yeah I don't know how we can avoid this circle jerk of inevitability: flood the internet with cheap generated content, so that the rest of us need ai to try and filter out the vast amount of shit to get to the meat? Then we finally get back to the shit covered meat we all crave
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
    - sgaur 2y ago
      Hope he’s using the AI to convert user inputs into query parameters and doing the actual retrieval through queries. Or perhaps there are subjective descriptions or reviews of the venues that need to be queried?
    - EGreg 2y ago
      Sounds like blockchain. Actually very much like blockchain, in that it’s slower and less efficient but they put it into everything inclusinf Iced Tea. The only difference is that AI weights are inscrutable and people don’t know what it will do, while blockchain’s actual achivement is precisely the opposite — knowing and proving what the code will do. But still, it is not a hammer for every nail!
      - surfingdino 2y ago
        Good analogy.
  - EGreg 2y ago
    In 10 years: “LLM, make your lookup more efficient. kthxbai”
vessenes 2y ago
Interesting. You could add counting and other jobs as well, eg ”how many times does value 8364 appear, and in which dictionaries?” A lot of the needle trained models clearly can’t reason with stuff everywhere in a long context even if they can retrieve. I would like to see some extensions that require some reasoning. I guess you could use words as dict vals and ask it what sentence a set of keys spells, and to answer the sentence with a set of keys. Lots of interesting possibilities.
avree 2y ago
If this type of fine-tuning severely harms the general-purpose capabilities of the model, why is it advantageous versus using a slimmer model more suited to your goal from the start? I guess I don't fully grok the paper.

surfingdino 2y ago

cat list_of_dicts.txt | grep '2931:'

optimalsolver 2y ago
This is why you need symbolic reasoning. The above is vastly more efficient than whatever these LLMs are doing.
- surfingdino 2y ago
  I had a chat with a dev friend of mine. He's building an app and will have users enter venue booking info. He then plans to use ... AI to check venue availability and sort bookings, something a couple of simple SQL queries and views would easily take care of. Where did we go wrong? How did the LLM monkey gang manage to embed their shitty ideas into the minds of otherwise intelligent people?
  - beefnugs 2y ago
    Yeah I don't know how we can avoid this circle jerk of inevitability: flood the internet with cheap generated content, so that the rest of us need ai to try and filter out the vast amount of shit to get to the meat? Then we finally get back to the shit covered meat we all crave
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - sgaur 2y ago
    Hope he’s using the AI to convert user inputs into query parameters and doing the actual retrieval through queries. Or perhaps there are subjective descriptions or reviews of the venues that need to be queried?
  - EGreg 2y ago
    Sounds like blockchain. Actually very much like blockchain, in that it’s slower and less efficient but they put it into everything inclusinf Iced Tea. The only difference is that AI weights are inscrutable and people don’t know what it will do, while blockchain’s actual achivement is precisely the opposite — knowing and proving what the code will do. But still, it is not a hammer for every nail!
    - surfingdino 2y ago
      Good analogy.
- EGreg 2y ago
  In 10 years: “LLM, make your lookup more efficient. kthxbai”

vessenes 2y ago

Interesting. You could add counting and other jobs as well, eg ”how many times does value 8364 appear, and in which dictionaries?” A lot of the needle trained models clearly can’t reason with stuff everywhere in a long context even if they can retrieve. I would like to see some extensions that require some reasoning. I guess you could use words as dict vals and ask it what sentence a set of keys spells, and to answer the sentence with a set of keys. Lots of interesting possibilities.

avree 2y ago

If this type of fine-tuning severely harms the general-purpose capabilities of the model, why is it advantageous versus using a slimmer model more suited to your goal from the start? I guess I don't fully grok the paper.