I wrote a small POC of getting this model working on my box (I felt inspired after reading this). If anybody else is wanting to try this out, give it a shot her

I wrote a small POC of getting this model working on my box (I felt inspired after reading this). If anybody else is wanting to try this out, give it a shot here! https://github.com/lunabrain-ai/dolly-v2-12b-8bit-example https://github.com/lunabrain-ai/dolly-v2-12b-8bit-example (It's garbage code and this should really just be used as a starting POC. I hope it helps!)