4 ms·
what would you recommend as a maximum-2B model for coding/reasoning domain ?
by algo_trader 2y ago
what would you recommend as a maximum-2B model for coding/reasoning domain ?
- coder543 2y agoFor reasoning tasks and coding tasks where you’re chatting with the model, there are no 2B models that I would recommend at this point.
- selcuka 2y ago> maximum-2B model If you are resource limited, remember that you can also play with the quantization to fit more parameters into less amount of RAM. Phi-3-mini [1] (a 3.8B model) is 7.64GB with full (16-bit floating point) precision, but it is only 2.39GB when quantized to 4 bits. That being said, I haven't personally tested it, but have heard good things for CodeGemma 2B [2]. [1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf... [2] https://huggingface.co/google/codegemma-2b-GGUF https://huggingface.co/google/codegemma-2b-GGUF
- coder543 2y agoCodeGemma-2b does not come in the "-it" (instruction tuned) variant, so it can't be used in a chat context. It is just a base model designed for tab completion of code in an editor, which I agree it is pretty good at.