3 ms·
By chance I noticed that Fabrice Bellard's TextSynth server has newly added support for Mistral 7B model. > 2023-10-21: CUDA support in the Windows version, mi
by lioeters 3y ago
By chance I noticed that Fabrice Bellard's TextSynth server has newly added support for Mistral 7B model.
> 2023-10-21: CUDA support in the Windows version, mistral model support. Speculative sampling is supported. BNF grammar and JSON schema sampling.
> mistral_7B_instruct_q4 - 3.9GB - Mistral 7B chat model
https://bellard.org/ts_server/ https://bellard.org/ts_server/