4 ms·
Yes you can download the trained model and run it on your machine. The article has a link to a hugging face model when you can play with in the web browser as a
by ghilston 4y ago
Yes you can download the trained model and run it on your machine. The article has a link to a hugging face model when you can play with in the web browser as a toy example and then download it locally and use with code.
- swyx 4y agoanother noob here - does this hugging face model expose an api? i have a light classification use case i might wanna try it out on but think running it on my machine/a beefy cloud machine would be overkill
- espadrine 4y agoAll spaces do[0], but please don’t abuse it: it is just for demo purposes. If you hammer it, it will be down for everyone, and they might not bring it back up. It can be run locally with ~16GB VRAM GPU; you might be able to configure it at a lower precision to run it with GPUs with half the RAM. [0]: https://huggingface.co/spaces/togethercomputer/GPT-JT/blob/main/app.py https://huggingface.co/spaces/togethercomputer/GPT-JT/blob/m...
- wongarsu 4y agoThere are also a number of commercial services that offer GPT-J APIs (and surely in a couple days GPT-JT APIs) on a pay-per-token or pay-per-compute-second basis. For light use cases those can be extremely affordable.
- codedokode 4y agoCannot one do inference using a CPU?
- swyx 4y agothank you!