3 ms·

ColossalChat: An Open-Source Solution for Cloning ChatGPT with a RLHF Pipeline

• rasengan 4y ago
  This is still based off llama though isn't it?
  • lumost 4y ago
    Also depends on getting the rlhf data. Meaning training annotations, paying them, getting the appropriate volume of annotations…
• mdaniel 4y ago
  > open-source a complete RLHF pipeline ... based on the LLaMA pre-trained model I've gotten to where when I see "open source AI" I now know it's "well, except for $some_other_dependencies" Anyway: https://scribe.rip/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b https://scribe.rip/@yangyou_berkeley/colossalchat-an-open-so... and https://github.com/hpcaitech/ColossalAI#readme https://github.com/hpcaitech/ColossalAI#readme (Apache 2) can save you some medium.com heartache at least