vishnukool

7 ms·

1. 61.▲
   by vishnukool 5y ago

   Thanks for this, I wasn't aware of it. From reading the docs, I don't see anywhere if Grid automatically handles spot interruptions to resume from last checkpoint which was the main focus of our internal tool. Have you used it btw
2. 62.▲
   by vishnukool 5y ago

   We built a tool called spotML to make training on AWS/GCP cheaper. Spot Instances are 70% cheaper than On-Demand instances but are prone to interruptions. We mitigate the downside of these interruptions through the use of persistence f
3. 63.▲
   Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Spot Instances(spotml.io)

   31 points by vishnukool 5y ago|17 comments
4. 64.▲
   by vishnukool 5y ago

   Yes we liked the elegance of both the tool and the docs. So it's very much inspired from it. I must also give credit to another great tool https://spotty.cloud/ from which this project was adopted.
5. 65.▲
   by vishnukool 5y ago

   Thank you! Interesting, we actually tried aws batch ourselves. 1) How were you able to handle spot interruptions and resuming from the latest checkpoint ? 2) Not to mention fallback to OnDemand on spot interruptions 3) then switching back t
6. 66.▲
   by vishnukool 5y ago

   Sure, in our own startup, we used to spend roughly $1000 for training a StyleGAN model for a class and then additional latent space manipulation models. In the early days we recklessly wasted a lot of our AWS credits during experimentation
7. 67.▲
   by vishnukool 5y ago

   We tried to keep it minimal. All you need to do is specify the format to resume last checkpoint in the spotml.yml file. So let's say your checkpoint files are saved as ckpt00.pt, ckpt01.pt, ckpt03.pt and so on. You can configure check
8. 68.▲
   by vishnukool 5y ago

   This is interesting, from what I've read AWS no longer recommends setting spot instance prices so that they manage it themselves. I wonder, if there's an actual advantage of avoiding spot interruptions by setting the spot price ev
9. 69.▲
   by vishnukool 5y ago

   Thanks for the feedback. The biggest upside is, if you have a long running training(say hours, days) and the spot training is interrupted. You probably don't want to manually monitor to check for the next available spot instance and ki
10. 70.▲
    by vishnukool 5y ago

    does it need to be an app ? The friction of downloading an app is discouraging to try it out.
11. 71.▲
    by vishnukool 5y ago

    Thanks for trying it out ! Ah, yes. It doesn't work great with sectionals yet. The model does an Ok job on the kind of sofa styles it's seen during training but still needs to generalize better. Also the UI demo is a quick sub sec
12. 72.▲
    by vishnukool 5y ago

    Thanks, do let me know any feedback when you try it. Regarding the name, yeah we'll probably need to pick something more generic. Right now though it's only home decor categories like sofa, chairs but the idea is to expand to bro
13. 73.▲
    by vishnukool 5y ago

    Thanks, yes. The 3D asset that's generated is pretty low poly at the moment. But I see how this could be used as a base for further modeling work.
14. 74.▲
    Show HN: REST API for 2D product image to 3D conversion(homely.ai)

    3 points by vishnukool 5y ago|7 comments