3 ms·
you can load it but you cant run inference? whats the issue?
by swyx 4y ago
you can load it but you cant run inference? whats the issue?
- tarruda 4y agoNo issue, I'm simply unfamiliar with python machine learning APIs. I managed to run inference locally by installing the requirements and running app.py from the demo: https://huggingface.co/spaces/replit/replit-code-v1-3b-demo/tree/main https://huggingface.co/spaces/replit/replit-code-v1-3b-demo/... It is very fast on my RTX 3070, VRAM usage goes to ~= 6.3GB during inference.