3 ms·
Well I was able to run the original code with the 7B model on 16GB vram: https://news.ycombinator.com/item?id=35013604 https://news.ycombinator.com/item?id=3501
by TaylorAlexander 4y ago
Well I was able to run the original code with the 7B model on 16GB vram:
https://news.ycombinator.com/item?id=35013604 https://news.ycombinator.com/item?id=35013604
The output I got was underwhelming, though I did not attempt any tuning.
- fnbr 4y agoparameter tuning is pretty necessary, according to anecdotes. People on twitter have got good results by changing the default parameters.
- int_19h 4y agoFor 13b and 30b, it really needs high temperature to produce good outputs.
- fdb 4y agoThe author just made an update that makes the generation much better, even with the 7B model: https://twitter.com/ggerganov/status/1634310199170179075 https://twitter.com/ggerganov/status/1634310199170179075 I tried it out myself (git pull && make) and the difference in results are day and night! It's amazing to play with, although you should prompt it differently than ChatGPT (more like the GPT-3 API).