3 ms·

Ask HN: Guides, books or repos for LLM fine-tuning

10 points by Eric_BB 3y ago

Hi everyone, I’m a student working on my thesis. I need to do classification using LLM fine-tuning and have access to my university’s HPC resources. I’m looking for guides, books or repositories that can help me understand how to do LLM fine-tuning. I don’t have a budget for training using APIs, but I’m interested in understanding what the cost would look like. Does anyone have any suggestions? Thank you so much!

ilaksh 3y ago
Please state the assignment in detail. What are you classifying exactly, for example. If it's just some simple text that is fairly regular then you might not need actual fine tuning. You could just use the OpenAI API. https://github.com/leehanchung/lora-instruct https://github.com/leehanchung/lora-instruct What exactly are the HPC resources. Are they GPUs and what type and how many.
- Eric_BB 3y ago
  Thank you for your answer. The objective of the assignment is to classify agents of misinformation based on their tweets. An example element of the dataset can be found at this link: https://ibb.co/16VMTCN https://ibb.co/16VMTCN. There is a dataset for the control group and a dataset of misinformation agents. The idea is to make the model closer to how misinformation agents are via fine-tuning on these datasets. The available HPC resources, including information about GPUs and their quantity, can be viewed at this link: https://www.carc.usc.edu/user-information/user-guides/hpc-basics/discovery-resources https://www.carc.usc.edu/user-information/user-guides/hpc-ba...
  - ilaksh 3y ago
    Ok. I think I understand the assignment. I don't believe you need to fine tune any model. You can probably just use the OpenAI ChatGPT model and ask it something like: "Does this user's tweet say anything negative about the government of ______ or contradict any of these official party statements? __________" You can probably just ask Falcon or Llama the same thing without any training. But if you decide you have to do the fine tuning then try with my link above using the A100 GPU nodes. I think the whole thing is nonsense though. Because whoever the arbiter of truth is always has an agenda and often makes mistakes.
    - deleted 3y ago
      [deleted]
    - Eric_BB 3y ago
      Thanks! Btw, a link to resources would still be appreciated if I need to apply the knowledge to personal projects in the future.
    - janalsncm 3y ago
      If the assignment is to classify the type of misinformation (assuming each tweet is misinformation) then it’s essentially topic modeling which is very doable without fine tuning as well.
      - Eric_BB 3y ago
        This is what I was thinking about using LLM for: 1. As a feature extractor. For example, given the text of misinformation agents, what are the characteristics? C1, C2, C3, etc. Then, do these characteristics appear in these new texts? Assign a label accordingly. 2. I'll give LLM the text on how they usually behave and ask if these new ones are behaving similarly. If so, label them accordingly. (There may also be the possibility to pass graph data in a graph-less way.) 3. Use the extracted information to enhance topology-driven classification
        janalsncm 3y ago
        Those might work to some extent but keep in mind the model doesn’t have access to outside information, and it’s going to be nearly impossible to build a social graph given Twitter API limits. IMO the easiest way to fine tune your model would be to use something like BERT embeddings fine tuned with triplet loss i.e. (example, positive, negative) to train the model to minimize distance between similar examples and maximize between dissimilar ones.
        Eric_BB 3y ago
        Very interesting! Thank you for the idea. I will try to figure out how to do that
  - ilaksh 3y ago
    This is a better link since it mentions custom.data: https://yashugupta-gupta11.medium.com/qlora-efficient-finetuning-of-large-language-model-falcon-7b-using-quantized-low-rank-adapters-2df59a7982d5 https://yashugupta-gupta11.medium.com/qlora-efficient-finetu...
extasia 3y ago
I'd recommend huggingface's transformers library. You probably don't wanna be finetuning these LLMs yourself since that's a big endeavour, but secondly because they already have the NLU capabilities to solve your problem. You basically want to train a single classification layer on top of your models output. I believe it'll be something like "CausalLMForClassification" in the huggingface docs. As far as resources go for finetuning I haven't found many great resources. But again, you probably don't wanna be finetuning these massive LLMs for your particular use case -\_/- Buenos suertes.
- Eric_BB 3y ago
  Thank you for the suggestion! I will check out the library and try it without any fine-tuning

10 points by Eric_BB 3y ago

ilaksh 3y ago

Please state the assignment in detail. What are you classifying exactly, for example. If it's just some simple text that is fairly regular then you might not need actual fine tuning. You could just use the OpenAI API. https://github.com/leehanchung/lora-instruct https://github.com/leehanchung/lora-instruct What exactly are the HPC resources. Are they GPUs and what type and how many.

Eric_BB 3y ago
Thank you for your answer. The objective of the assignment is to classify agents of misinformation based on their tweets. An example element of the dataset can be found at this link: https://ibb.co/16VMTCN https://ibb.co/16VMTCN. There is a dataset for the control group and a dataset of misinformation agents. The idea is to make the model closer to how misinformation agents are via fine-tuning on these datasets. The available HPC resources, including information about GPUs and their quantity, can be viewed at this link: https://www.carc.usc.edu/user-information/user-guides/hpc-basics/discovery-resources https://www.carc.usc.edu/user-information/user-guides/hpc-ba...
- ilaksh 3y ago
  Ok. I think I understand the assignment. I don't believe you need to fine tune any model. You can probably just use the OpenAI ChatGPT model and ask it something like: "Does this user's tweet say anything negative about the government of ______ or contradict any of these official party statements? __________" You can probably just ask Falcon or Llama the same thing without any training. But if you decide you have to do the fine tuning then try with my link above using the A100 GPU nodes. I think the whole thing is nonsense though. Because whoever the arbiter of truth is always has an agenda and often makes mistakes.
  - deleted 3y ago
    [deleted]
  - Eric_BB 3y ago
    Thanks! Btw, a link to resources would still be appreciated if I need to apply the knowledge to personal projects in the future.
  - janalsncm 3y ago
    If the assignment is to classify the type of misinformation (assuming each tweet is misinformation) then it’s essentially topic modeling which is very doable without fine tuning as well.
    - Eric_BB 3y ago
      This is what I was thinking about using LLM for: 1. As a feature extractor. For example, given the text of misinformation agents, what are the characteristics? C1, C2, C3, etc. Then, do these characteristics appear in these new texts? Assign a label accordingly. 2. I'll give LLM the text on how they usually behave and ask if these new ones are behaving similarly. If so, label them accordingly. (There may also be the possibility to pass graph data in a graph-less way.) 3. Use the extracted information to enhance topology-driven classification
      - janalsncm 3y ago
        Those might work to some extent but keep in mind the model doesn’t have access to outside information, and it’s going to be nearly impossible to build a social graph given Twitter API limits. IMO the easiest way to fine tune your model would be to use something like BERT embeddings fine tuned with triplet loss i.e. (example, positive, negative) to train the model to minimize distance between similar examples and maximize between dissimilar ones.
        Eric_BB 3y ago
        Very interesting! Thank you for the idea. I will try to figure out how to do that
- ilaksh 3y ago
  This is a better link since it mentions custom.data: https://yashugupta-gupta11.medium.com/qlora-efficient-finetuning-of-large-language-model-falcon-7b-using-quantized-low-rank-adapters-2df59a7982d5 https://yashugupta-gupta11.medium.com/qlora-efficient-finetu...

extasia 3y ago

I'd recommend huggingface's transformers library. You probably don't wanna be finetuning these LLMs yourself since that's a big endeavour, but secondly because they already have the NLU capabilities to solve your problem. You basically want to train a single classification layer on top of your models output. I believe it'll be something like "CausalLMForClassification" in the huggingface docs. As far as resources go for finetuning I haven't found many great resources. But again, you probably don't wanna be finetuning these massive LLMs for your particular use case -\_/- Buenos suertes.

Eric_BB 3y ago
Thank you for the suggestion! I will check out the library and try it without any fine-tuning