3 ms·
Custom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplac
by smoldesu 4y ago
Custom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplace/en_US/listin...
Not as fast as a GPU, but less than 5 seconds for a 250 token response is good enough for a Discord bot.
- nl 4y agoThis is the most interesting thing I've read in this thread. How have I never heard of this accelerator?!