4 ms·

> 3. Skim-read the paper to get the gist of the jargon Or, you know, you could ask ChatGPT to explain it to you... Granted the term was coined 2021>=. Even if

by mszcz 4y ago

> 3. Skim-read the paper to get the gist of the jargon Or, you know, you could ask ChatGPT to explain it to you... Granted the term was coined 2021>=. Even if it wasn't but the paper is less than 32k tokens... 0.6c for the answer doesn't seem all that steep. edit: grammar

Grothendank 4y ago
This actually works! It works astoundingly well with poorly written technical manuals. Looking at you, CMake reference manual O_O. It also helps translate unix man pages from Neckbeardese into clean and modern speech. With science papers it's a bit more work. You must copy section by section into GPT4, despite the increased token limit. But sure. Here's how it can work: 1. Copy relevant sections of the paper 2. As questions about the jargon: "Explain ____ like I'm 5. What is ____ useful for? Why do we even need it?" "Ah, now I understand _____. But I'm still confused about _____. Why do you mean when you say _____?" "I'm starting to get it. One final question. What does it mean when ______?" "I am now enlightened. Please lay down a sick beat and perform the Understanding Dance with me. Dances" This actually works surprisingly well.
- mszcz 4y ago
  Yeah, I think education is a great use case here. Sure, the knowledge that's built into the model might be inaccurate or wrong but you can feed the model the knowledge you want to learn/processed. What you get is a teacher that never tires, is infinitely patient, has infinite time, doesn't limit questions, doesn't judge you, really listens and has broad, multidisciplinary knowledge that correct-ish (for when it's needed). I've recently read somewhere that Stanford (?) has almost as many admin workers as they do students. Seems to me that this is a really bad time to be that bloated. Makes you wonder what you really spend your money on, is it worth it (yeah, I know, it's not just education that you get in return) and if you can get the same-ish effect for a lot cheaper and on your timetable. Not that the models or field, now, are in a state that would produce a good teaching experience. I can however imagine a future not so distant that this would be possible. Recently on a whim I've asked it to produce a options trading curriculum for me. It did a wonderful job. I wouldn't trust it if I didn't know a little bit myself about the subject before but I came off really impressed.
lordswork 4y ago
No need to pay yourself. Uploaded https://arxiv.org/pdf/2106.09685.pdf https://arxiv.org/pdf/2106.09685.pdf to scisummary: This text discusses various studies and advancements in the field of natural language processing (NLP) and machine learning. One study focuses on parameter-efficient transfer learning, and another examines the efficiency of adapter layers in NLP models. Further studies evaluate specific datasets for evaluating NLP models. The article proposes a method called LoRA (low rank adaptation) for adapting pre-trained neural network models to new tasks with fewer trainable parameters. LoRA allows for partial fine-tuning of pre-trained parameters and reduces VRAM usage. The article provides experimental evidence to support the claims that changing the rank of Delta W can affect the performance of models, and that LoRA outperforms other adaptation methods across different datasets. The authors propose LoRA as a more parameter-efficient approach to adapt pre-trained language models to multiple downstream applications.

by mszcz 4y ago

Grothendank 4y ago

This actually works! It works astoundingly well with poorly written technical manuals. Looking at you, CMake reference manual O_O. It also helps translate unix man pages from Neckbeardese into clean and modern speech. With science papers it's a bit more work. You must copy section by section into GPT4, despite the increased token limit. But sure. Here's how it can work: 1. Copy relevant sections of the paper 2. As questions about the jargon: "Explain ____ like I'm 5. What is ____ useful for? Why do we even need it?" "Ah, now I understand _____. But I'm still confused about _____. Why do you mean when you say _____?" "I'm starting to get it. One final question. What does it mean when ______?" "I am now enlightened. Please lay down a sick beat and perform the Understanding Dance with me. Dances" This actually works surprisingly well.

mszcz 4y ago
Yeah, I think education is a great use case here. Sure, the knowledge that's built into the model might be inaccurate or wrong but you can feed the model the knowledge you want to learn/processed. What you get is a teacher that never tires, is infinitely patient, has infinite time, doesn't limit questions, doesn't judge you, really listens and has broad, multidisciplinary knowledge that correct-ish (for when it's needed). I've recently read somewhere that Stanford (?) has almost as many admin workers as they do students. Seems to me that this is a really bad time to be that bloated. Makes you wonder what you really spend your money on, is it worth it (yeah, I know, it's not just education that you get in return) and if you can get the same-ish effect for a lot cheaper and on your timetable. Not that the models or field, now, are in a state that would produce a good teaching experience. I can however imagine a future not so distant that this would be possible. Recently on a whim I've asked it to produce a options trading curriculum for me. It did a wonderful job. I wouldn't trust it if I didn't know a little bit myself about the subject before but I came off really impressed.

lordswork 4y ago

No need to pay yourself. Uploaded https://arxiv.org/pdf/2106.09685.pdf https://arxiv.org/pdf/2106.09685.pdf to scisummary: This text discusses various studies and advancements in the field of natural language processing (NLP) and machine learning. One study focuses on parameter-efficient transfer learning, and another examines the efficiency of adapter layers in NLP models. Further studies evaluate specific datasets for evaluating NLP models. The article proposes a method called LoRA (low rank adaptation) for adapting pre-trained neural network models to new tasks with fewer trainable parameters. LoRA allows for partial fine-tuning of pre-trained parameters and reduces VRAM usage. The article provides experimental evidence to support the claims that changing the rank of Delta W can affect the performance of models, and that LoRA outperforms other adaptation methods across different datasets. The authors propose LoRA as a more parameter-efficient approach to adapt pre-trained language models to multiple downstream applications.