4 ms·

Whoa, imagine you get a good base LLM model and save all conversations with it. Run a batch process every night to fine tune a LORA on convo dataset. If I ever

Whoa, imagine you get a good base LLM model and save all conversations with it. Run a batch process every night to fine tune a LORA on convo dataset. If I ever came across such a chat bot I'd probably freak out as to why it remembers things outside of the context window, without summarisation

MichaelZuo 3y ago
That's a pretty neat idea, I would be surprised if no one is already working on that.
- notuseful 3y ago
  I did it years ago on a lark with a seq2seq model in a matrix chat room.
  - eurekin 3y ago
    How did it perform? Was it well received by members?
    - notuseful 3y ago
      Poorly! It was a small seq2seq and was gibberish to start with. Although it did tell my friend that it loved him which was nice.