3 ms·
GLM-130B[1] (a 130 billion parameter model vs GPT-3's 175 billion parameter model) is able to run optimally on consumer level high-end hardware, 4xRTX 3090 in p
by somenameforme 4y ago
GLM-130B[1] (a 130 billion parameter model vs GPT-3's 175 billion parameter model) is able to run optimally on consumer level high-end hardware, 4xRTX 3090 in particular. That's < $4k at current prices, and as hardware prices go one can only imagine what it'll be in a year or two. It also enables running with degraded performance on lesser systems.
It's a whole lot cheaper to run neural net style systems than to train them. "Somebody on Twitter"[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. Cliff notes being a fraction of a penny per query, with each taking about 16s to generate. The output's pretty terrible, but it's unclear to me whether that's inherent or a result of priority. I expect OpenAI spent a lot of manpower on supervised training, whereas this system probably had minimal, especially in English (it's from a Chinese university).
[1] - https://github.com/THUDM/GLM-130B https://github.com/THUDM/GLM-130B
[2] - https://twitter.com/alexjc/status/1617152800571416577 https://twitter.com/alexjc/status/1617152800571416577
- TOMDM 4y agoApples hardware, and 4 RTX 3090's being able to run a GPT scale model quantized to 4 byte ints are worlds apart. Who knows though, maybe someone manages to get 4byte quantization producing good results and Apple makes a chip that can do the required ops for whatever that looks like with ~100GB of memory attached and then GP's comment might be relevant. I started this reply rather skeptical, but with the boundaries Apple has been pushing, and the pace of AI research, honestly who knows.
- earthboundkid 4y agoIsn’t the research now on using float8? Anyway, it will take time to move battleship, but they’re well aligned and motivated. Their UI team has gone to hell in recent years, but hardware has been strong with nothing to waste the power on. Now there is something to use it for.
- somenameforme 4y agoAre we not putting the cart before the horse? Where's the widespread usage for this that people would actually want? I can think of quite a lot of uses, but oddly enough most are quite negative and not widespread in any case. Media companies can use it to fire employees and increase profit margins. Hollywood can use it to rapidly churn out even more overtly derivative and generic movies. Students can use it to cheat on papers. Governments, and marketing agencies alike, can use it for endless contextually sensitive propaganda on social media. Even in search I think it will, at most, be a sort of sidebox that says 'Super clippy says the answer to your query is [blah].' Because a single opaque source of information, which will continue to struggle with truthfulness (both inadvertently, and by design) is really just going to supplement endless dynamic content on a topic. I'm just not seeing the big use for this (outside of the brief period of 'wow' novelty) in anything remotely like its current state.