2 ms·
Thanks for sharing it! I'm using their "Always Free" tier to host an Ampere-accelerated GPT-J chatbot right now. Works like a charm, and best of all, it's free!
by smoldesu 4y ago
Thanks for sharing it! I'm using their "Always Free" tier to host an Ampere-accelerated GPT-J chatbot right now. Works like a charm, and best of all, it's free!
- jocaal 4y agoI don't understand, the Ampere they refer to in their free tier are cpu's not gpu's. How did you manage to do that
- smoldesu 4y agoCustom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplace/en_US/listin... Not as fast as a GPU, but less than 5 seconds for a 250 token response is good enough for a Discord bot.
- nl 4y agoThis is the most interesting thing I've read in this thread. How have I never heard of this accelerator?!
- damascus 4y agoDo you have any code from your discord bot you're willing to share? I'd be happy to share back any updates I made to it. I've been wanting to play with this idea for a bit.
- deleted 4y ago[deleted]