8 ms·
Faster Than Llama 2: DeciLM LLM with Variable GQA
- slimsag 3y ago> permissively licensed, open-source foundation LLM Sounds great > This license is, in part, based on the LLAMA 2 Community License Agreement, with a series of modifications. Use of DeciLM for hosted services may require a separate license. Ah, so a bait-and-switch. It's not open source, and not permissively licensed.
- generalizations 3y agoCall it "limited open source" or something, but the dishonesty leaves a worse taste than admitting it's not open source.
- deleted 3y ago[deleted]
- karim79 3y agoThree pricing tiers and no listed prices on the "pricing" page. Only the option to talk to sales. So, depending on the outcome of the conversation one could end up paying less for the Enterprise tier than for the Basic one. No single indicator of a starting/base price.
- SubiculumCode 3y agoWell, I admit that I now feel bad for posting it to HN; I should have looked at the actual details of the license rather than the headline they are trying to sell.
- calderknight 3y agoJust call it what it is: proprietary, nonfree software
- demarq 3y agoThis doesn’t make sense, they don’t control Meta. They have absolutely nothing to do with that part of the license
- jules22 3y agoThey seem to have edited out "open-source" now. Still, a free for personal use model is better than none. Hopefully, we will soon have some adaptation from Llama 2 that matches it, fairly soon.
- coder543 3y agoThey also announced a faster version of Stable Diffusion, supposedly: https://deci.ai/blog/decidiffusion-1-0-3x-faster-than-stable-diffusion-same-quality/ https://deci.ai/blog/decidiffusion-1-0-3x-faster-than-stable... I haven’t personally tested either yet.
- brucethemoose2 3y agoUhhh, 50 iterations isnt really standard in diffusion land anymore. Neither is unmodified HF diffusers (or worse, the SAI implementation). Some of the techniques they describe are neat though
- SubiculumCode 3y agohow many iterations are usual these days? In terms of unmodified diffusers, you mean fine tuned to whatever they are working on? I agree.
- brucethemoose2 3y ago> how many iterations are usual these days? I dunno about a precise number...Less than 30? The UniPC scheduler in particular can get great results below 20 iterations. Karras DPM schedulers are pretty good with 30. > In terms of unmodified diffusers, you mean fine tuned to whatever they are working on? No, people hack non-default augmentations onto the vanilla diffusers (or SAI) pipe to increase speed or increase quality. Some examples included the channels last memory format, downsampling some stages to FP16, TomeSD, self attention, and some more user-guided augmentations like controlnet or tiled diffusion. "alternate" runtimes like the ones available via torch.compile or AITemplate provide some pretty massive speed gains (~2x) if you need them, but are less common since they are hard to install and HF diffusers only (where most of the SD community uses the original SAI implementation). Just as an example implementation, VoltaML is quite speedy out-of-the-box, with most of the sane optimizations already implemented, though its missing some features (like tiled diffusion/area prompting) that the SAI based pipelines have: https://github.com/VoltaML/voltaML-fast-stable-diffusion https://github.com/VoltaML/voltaML-fast-stable-diffusion
- yieldcrv 3y agoIs there any community or resource that keeps up with all combinations of performance improvements? another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned? equivalent cost per hour if using various cloud solutions when optimized?
- nickpsecurity 3y agoBest one I know for local AI's is below. The people who make the quantizations and OSS software often post there. You can find a lot on models, building rigs, etc using search. https://www.reddit.com/r/localLLaMA/ https://www.reddit.com/r/localLLaMA/
- kiratp 3y agoThis thing is basically closed source and closed license. The license explicitly blocks “Hosting Use” and does not even differentiate between “model as an API” and in-application use. I hope the community rejects this kind of BS.
- dTal 3y agoWell I hope the community decides not to press the issue for a good while, lulling companies into a false sense of security that they can release large high-quality models with "licenses" attached, before the seminal court case that rules that language models are not in fact copyrightable and no one can stop us doing anything we like with such data.
- daguava 3y agoThis looks like absolute trash - every single graph is about FAST FAST FAST. 5x FASTER, 15x FASTER FAST FAST FAST!!!! How about accuracy? Are the results good? I can shit out a blank string at gigabits per second easily... Searching for the string "accu" yields no results. "acc" only yields "acceleration". This is stupidity and should be ignored. EDIT: HOLY KEK - they have a "carbon emissions" chart showing them saving 95%. Jesus christ, ignore everything they say, they are blind deaf and dumb.
- WithinReason 3y agoSearch for the string "metric"
- natsucks 3y agocuriously, they don't mention MMLU, which is one of the most commonly reported metrics for comparison.