5 ms·

Are you sure about that? I can't remember where I saw the table of memory requirements, but I'm sure some of the larger instances here [1] will surely be able t

Are you sure about that? I can't remember where I saw the table of memory requirements, but I'm sure some of the larger instances here [1] will surely be able to cope (assuming they're available!) Oracle gives you a $300 free trial, which equates to running BM.GPU4.8 for over 10 hours - enough for a focused day of prompting [1] https://www.oracle.com/cloud/compute/gpu/ https://www.oracle.com/cloud/compute/gpu/

v64 4y ago
> Are you sure about that? I'm not. The only way to know it is to try :) thank you for the link!
- linearalgebra45 4y ago
  You only get a single month-long window to spend the credit! And I'm sure not going to spend any of my own money on prompting experiments. I might be suffering from FOMO to some degree, I've just got to tell myself that this won't have been the only time model weights get leaked!
  - mynameisvlad 4y ago
    > And I'm sure not going to spend any of my own money on prompting experiments. This certainly sounds a lot like whining that others aren’t doing the work you yourself don’t want to do.
    - linearalgebra45 4y ago
      "prompting experiments" is just my use-case. According to v64 a lot of people have had the same idea of spinning up a trial instance to run inference, which is unsurprising. I'm not in a position to put in any meaningful work towards optimising this model for lower-end hardware, or working on the tooling/documentation/user experience.
smoldesu 4y ago
Thanks for sharing it! I'm using their "Always Free" tier to host an Ampere-accelerated GPT-J chatbot right now. Works like a charm, and best of all, it's free!
- jocaal 4y ago
  I don't understand, the Ampere they refer to in their free tier are cpu's not gpu's. How did you manage to do that
  - smoldesu 4y ago
    Custom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplace/en_US/listin... Not as fast as a GPU, but less than 5 seconds for a 250 token response is good enough for a Discord bot.
    - nl 4y ago
      This is the most interesting thing I've read in this thread. How have I never heard of this accelerator?!
- damascus 4y ago
  Do you have any code from your discord bot you're willing to share? I'd be happy to share back any updates I made to it. I've been wanting to play with this idea for a bit.
  - deleted 4y ago
    [deleted]
fswd 4y ago
If you actually try and do this, the sales people will stop you due to some internal rule. No GPUs on free credit. Unless the situation has changed of course..

v64 4y ago

> Are you sure about that? I'm not. The only way to know it is to try :) thank you for the link!

linearalgebra45 4y ago
You only get a single month-long window to spend the credit! And I'm sure not going to spend any of my own money on prompting experiments. I might be suffering from FOMO to some degree, I've just got to tell myself that this won't have been the only time model weights get leaked!
- mynameisvlad 4y ago
  > And I'm sure not going to spend any of my own money on prompting experiments. This certainly sounds a lot like whining that others aren’t doing the work you yourself don’t want to do.
  - linearalgebra45 4y ago
    "prompting experiments" is just my use-case. According to v64 a lot of people have had the same idea of spinning up a trial instance to run inference, which is unsurprising. I'm not in a position to put in any meaningful work towards optimising this model for lower-end hardware, or working on the tooling/documentation/user experience.

smoldesu 4y ago

Thanks for sharing it! I'm using their "Always Free" tier to host an Ampere-accelerated GPT-J chatbot right now. Works like a charm, and best of all, it's free!

jocaal 4y ago
I don't understand, the Ampere they refer to in their free tier are cpu's not gpu's. How did you manage to do that
- smoldesu 4y ago
  Custom PyTorch with on-chip acceleration: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/125935163 https://cloudmarketplace.oracle.com/marketplace/en_US/listin... Not as fast as a GPU, but less than 5 seconds for a 250 token response is good enough for a Discord bot.
  - nl 4y ago
    This is the most interesting thing I've read in this thread. How have I never heard of this accelerator?!
damascus 4y ago
Do you have any code from your discord bot you're willing to share? I'd be happy to share back any updates I made to it. I've been wanting to play with this idea for a bit.
- deleted 4y ago
  [deleted]

fswd 4y ago

If you actually try and do this, the sales people will stop you due to some internal rule. No GPUs on free credit. Unless the situation has changed of course..