3 ms·

In regards to improving the original dataset of 345/745M, I'm encountering this while I've been building an open-sourced tool at https://writeup.ai https://writ

In regards to improving the original dataset of 345/745M, I'm encountering this while I've been building an open-sourced tool at https://writeup.ai https://writeup.ai I'm not certain how random users feel when knowing their selection knowing it's being used to improve an algorithm. In this case, it's relatively easy for me to log the prompt and the option that was selected -- just doing it felt a little ... bad, and I'm a little too scared of GDPR for a fun project. The other thing is humans sometimes select the funnier option even though it may not necessarily be the best one. In a Show Reddit post, the most upvoted response is a Game of Throne's character Tyrion and a brothel story. https://www.reddit.com/r/FanFiction/comments/d5s9yh/i_made_a_bot_that_writes_fan_fiction_with_you/ https://www.reddit.com/r/FanFiction/comments/d5s9yh/i_made_a... Open-sourced Code: https://github.com/jeffshek/writeup-frontend https://github.com/jeffshek/writeup-frontend

Jack000 7y ago
just curious how you're hosting this? The cheapest GCP GPU instance is $0.5/hr, which is pretty expensive long-term.
- jeffshek 7y ago
  GCP Instances that are scaling up and down based on usage. Lot of image freezing, for faster boots, etc. One hack is for the medium-level models, you can actually run them on Cascade Lake (which are sort-of more optimized for ML) than traditional processors. There's a 30% performance there. Mathematically, GPU vs CPU in performance at inference time, you're paying an annoying premium! Right now, the default writing style "medium" is running in Cascade Lake (no gpu). I also (over)optimized the microservices running the endpoints too.
  - Jack000 7y ago
    didn't know about Cascade Lake, thanks! Every time I did the numbers it didn't add up for cloud GPU. I could buy a new gaming desktop, serve the model from that, and make my money back in 2-3 months

Jack000 7y ago

just curious how you're hosting this? The cheapest GCP GPU instance is $0.5/hr, which is pretty expensive long-term.

jeffshek 7y ago
GCP Instances that are scaling up and down based on usage. Lot of image freezing, for faster boots, etc. One hack is for the medium-level models, you can actually run them on Cascade Lake (which are sort-of more optimized for ML) than traditional processors. There's a 30% performance there. Mathematically, GPU vs CPU in performance at inference time, you're paying an annoying premium! Right now, the default writing style "medium" is running in Cascade Lake (no gpu). I also (over)optimized the microservices running the endpoints too.
- Jack000 7y ago
  didn't know about Cascade Lake, thanks! Every time I did the numbers it didn't add up for cloud GPU. I could buy a new gaming desktop, serve the model from that, and make my money back in 2-3 months