2099miles

6 ms·

31.
by 2099miles 3y ago
Source? I’ve seen this anecdotally and heard it, but is there a paper you’re referencing?
32.
by 2099miles 3y ago
The LLM itself should realize it’s too big and only put the important parts on the gpu. If you’re asking questions about literature there’s no need to have all the params on the gpu, just tell it to put only the ones for literature on there
33.
by 2099miles 3y ago
Do you guys really get that much help from these local LLMs? Chatgpt has SO much more functionality and it still is quite limited. Why is it worth it for you guys to run these things locally like what are they doing that provides you so muc
34.
by 2099miles 3y ago
That doesn’t quite work. b. Redistribution and Use. i. If you distribute or make the Llama Materials, or any derivative works thereof, available to a third party, you shall provide a copy of this Agreement to such third party. It would be e
35.
by 2099miles 3y ago
I run llama chat 70b on a p3 8x large (4 Tesla) and it runs at like 1-5 tokens per sec. And I’m running the model with only 4 bit precision. Are you doing anything else?
36.
by 2099miles 3y ago
Talked about this idea last month since astrobiology still isn’t all dubbed to English. Thank you for actually making the tool, it’s awesome, huge Kudos.
37.
by 2099miles 3y ago
The prompts they used were also different than the ones given like “is this the right order” was “is this the right order, consider the distance from the sun” they put this in their post on Google dev blog. This one seems to be super straig
38.
by 2099miles 3y ago
Can you talk about the attention visualization a bit. What’s the benefit. I have been plotting attention masks and even visualizing them over sentences but it obv doesn’t result in a ton of insight due to the sentences changing from words t