3 ms·
I don't understand, the Ampere they refer to in their free tier are cpu's not gpu's. How did you manage to do that
by jocaal 4y ago
I don't understand, the Ampere they refer to in their free tier are cpu's not gpu's. How did you manage to do that
- smoldesu 4y agoCustom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplace/en_US/listin... Not as fast as a GPU, but less than 5 seconds for a 250 token response is good enough for a Discord bot.
- nl 4y agoThis is the most interesting thing I've read in this thread. How have I never heard of this accelerator?!