3 ms·
I’ve gotten mine running with FastChat - they have a Metal/MPS option. Sadly 7b is not very good for SQL tasks. I think even with RAG it would struggle.
by SOLAR_FIELDS 3y ago
I’ve gotten mine running with FastChat - they have a Metal/MPS option.
Sadly 7b is not very good for SQL tasks. I think even with RAG it would struggle.