4 ms·
uv doesn’t fundamentally solve the issues. It didn’t invent venv or pip. What fundamentally solves the issue is to use an onnx version of the model.
by ComputerGuru 1y ago
uv doesn’t fundamentally solve the issues. It didn’t invent venv or pip.
What fundamentally solves the issue is to use an onnx version of the model.
- simonw 1y agoDo you know if it's possible to run ONNX versions of models on a Mac? I should try those on the NVIDIA Spark, be interesting to see if they are easy to work with on ARM64.
- ComputerGuru 1y agoYup. The beauty of it is that the underlying ai accelerator/hardware is completely abstracted away. There’s a CoreML ONNX execution provider, though I haven’t used it. No more fighting with hardcoded cuda:0 everywhere. The only pain point is that you’ll often have to manually convert a PyTorch model from huggingface to onnx unless it’s very popular.