3 ms·
Minigpt4 Inference on CPU
- quickthrower2 3y agoNot heard of minigpt4. Why that name? Is it claiming to be specifically a gpt4 competitor?
- sanxiyn 3y agoMiniGPT-4 is a multimodal model. The name (a bad one, IMO) is a reference to GPT-4's multimodal capability.
- Zondartul 3y agoThey should rename it to ManyGPT.
- WantonQuantum 3y agoThere's more info here: https://github.com/Vision-CAIR/MiniGPT-4 https://github.com/Vision-CAIR/MiniGPT-4 (Linked in the Readme of the repo.)
- Der_Einzige 3y agoAny data on inference speed? I’ve found that the non quantized model was much faster on GPU than the quantized versions due to lower GPU utilization.
- api 3y agoIt's a RAM tradeoff. If you have enough GPU RAM to load the non-quantized model it may be faster.
- heyitsguay 3y agoI know it's not the main point of this, but... so many multimodal models now that take frozen vision encoders and language decoders and weld them together with a projection layer! I wanna grab the EVA02-CLIP-E image encoder and the Llama-2 33B model and do the same, I bet that'd be fun :D
- GaggiX 3y agoNot just a projection layer but also Q-former, in this case it was already trained for that specific vision encoder but if you change it you would need to train a Q-former from scratch.
- famouswaffles 3y agoNot for mini gpt-4 but it's just a projection layer for many others(like Llava). The Qformer isn't a necessary part of the equation.
- famouswaffles 3y agoQformer isn't necessary just to be clear. Llava is just a projection layer
- pizzafeelsright 3y agoI am not an ML expert. I want to know how to add my own documents without sending them off to a 3rd party.
- extasia 3y agoUse a local LLM like LLaMa 2.