3 ms·

Thanks a lot for sharing. Haven't tested Strix Halo myself. Did you consider DGX Spark as well? What is your target use case? Curious what feels suboptimal abo

• andy99 9mo ago
  Re DGX, I’m mostly interested in local inference, it might have been nice to try but it was more expensive for similar performance (or so I think). I do lots of different experiments, synthetic data generation along the lines of Magpie is one of the things I wanted a local machine for, as well as just general access to a decent sized LLM to try different things, without having to spin up a cloud machine each time. I would prefer PyTorch / HF transformers to llama.cpp as I fine the latter less flexible if I want to change anything.