3 ms·
From my understanding, which may be wrong, you only need the massive compute resources initially to create a compiled vector space LLM - and then that LLM once
by loceng 3y ago
From my understanding, which may be wrong, you only need the massive compute resources initially to create a compiled vector space LLM - and then that LLM once compiled can be run locally?
This is why anti-CSAM measures policy is possible so compiled-release LLMs can have certain vector spaces removed before release; but apparently people are creating cracks for these types of locks?
- extr 3y agoYou are a little confused. There’s no “compiling” of LLMs. It’s just once it’s trained, inference takes less compute than further training. So you can run things locally that you couldn’t necessarily train locally. Not sure where you are getting the CSAM bit. We aren’t that good at blanking out weights in any kind of model, certainly not good enough to lobotomize specific types of content.
- JyB 3y agoI love that you are using the word lobotomize.
- loceng 3y agoThanks for the clarification. The CSAM bit seems to then be propaganda from at least one AI company putting out PR to falsely quell people's concerns about their LLMs being able to generate content involving children that's sexualized. I've yet to see details of how much compute-minimum server requirements are necessary to run LLMs. Maybe you know a source who's compiling a list in a feature matrix that includes such details?
- dwaltrip 3y agoLarge LLMs like gpt-3 and gpt-4 need very serious hardware. They have hundreds of billions of parameters (or more) which need to be loaded in memory all at once.