3 ms·
Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128G
by Archit3ch 3mo ago
Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.
- sebastianconcpt 3mo agoThis is the way.
- fragmede 3mo agoFollowed you on GitHub to get notified when you are!
- xtracto 3mo agoI will be delighted to try. I have a 128gb macbookpro m4 waiting for this.
- ukd1 3mo agods4 is pretty optimized for this btw - https://github.com/antirez/ds4 https://github.com/antirez/ds4
- arm32 3mo agoLink it already!