9 ms·

Shareable Jupyter Notebooks That Run on Free Cloud GPUs

199 points by hsikka 7y ago

hsikka 7y ago
Hey HN, really thrilled to see this launch. I was a Research Fellow at the Paperspace Advanced Technologies Group this summer and saw this project develop. Gradient Notebooks were an indispensable tool in the work I was doing, and had several advantages for my work over other notebook services. As a researcher, it’s great to see an emphasis being placed on starting projects with no fuss or issues around setting up infrastructure and sharing/forking models. Giving anyone access to free GPUs and powerful tooling seems like an incredible opportunity. I'd love to hear what you all think!
- ArtWomb 7y ago
  It's a home run in my book. Perfect for "Intro to ML" courses and allows students instant access to real env. Personally can use it for art & design experiments like GAN Breeder ;) Am wondering what to use for the data tier? Is there a dedicated backing cloud store supporting BigQuery style syntax? Import form S3 datasets?
  - DTE 7y ago
    Hey there ! I'm one of the co-founders of Paperspace. We currently have a couple of options for data ingest (and more coming soon!). The current system provides a single persistent mount that you can access from any notebook/experiment/etc in the /storage directory. We also mount a special directory called /artifacts where you can pipe out any models, files, etc and they will be pushed to an S3-compatible object store. Checkout the docs here -> https://docs.paperspace.com/gradient/data/storage https://docs.paperspace.com/gradient/data/storage And happy to answer any other questions
    - MarkMMullin 7y ago
      OK, did a quick scan on free and paid tiers and wondering about ability to bring in specialized python libraries- in my case, https://github.com/opencog/link-grammar https://github.com/opencog/link-grammar which almost always requires hand building and migrating, and some custom packages on top of my own even more horrible C++ . - I have to run a jupyterhub ami on an ec2 instance right now
      - DTE 7y ago
        Yes, one of the differences in how we handle notebooks is that everything is actually run in a container behind the scenes. This means that is isn't just the .ipynb that we are hosting and if you install and dependencies, libraries, etc it will actually persist all of it inside of a container. This makes it much easier to share your work with others so that i.e. I could fork your notebook if you made it public and get all of the installed libraries and compiled dependencies by default. Hope that helps! Edit: we also have another tool called GradientCI (https://docs.paperspace.com/gradient/projects/gradientci https://docs.paperspace.com/gradient/projects/gradientci) that might also be of interest. Basically it lets you connect a GitHub repo directly to a project and you can use it to build your container automatically.
        MarkMMullin 7y ago
        Gotcha - yah, I can containerize, it's not like I'm screwing with drivers or whatnot on the AMI - I'll keep an eye on you, not ready for GPU yet, as I can't even rationally define the vectors I'm extracting from the Link stuff. Best of luck to you, lot of people piling into that battleground, and the Dunning-Kreuger effect is rampant. :-) Side question, if you're willing to entertain it. Tired me tells a notebook to checkpoint, wanders off to bed, comes back next day and wonders why its taking an aeon to open the notebook..... oh yeah, damn, I've got gigs of images and panda crap in it . -- do you wrangle this problem, i.e. please don't save your notebooks as gihugic files representing a mental state you can't possibly remember ?
        DTE 7y ago
        I should also mention you can just as easily run these on CPU-backed instances as well. The GPU is not a hard requirement. As for checkpointing data, that is still a relatively difficult problem to solve and our current recommendation is to use a combination of the persitent /storage directory and the notebook home directory. There are definitely issues with doing 100K+ of small files and committing those to the primary docker layer. When you get to testing it out don't hesitate to reach out to use and we can try to see what the best solution is for your particular project. To date there isn't a "one size fits all" solution but we are working hard on making more intelligent choices behind the scenes to unblock some of these IO constraints.
    - lostmsu 7y ago
      Hi! Do you guys have a trademark for the name Gradient? I am developing a C#/.NET binding to TensorFlow with the same name: https://losttech.software/gradient.html https://losttech.software/gradient.html If you do, do you mind that clash of names? Secondly, would you be interested to invest in providing .NET-powered notebooks? I just got it working on Azure Notebooks for F# ( http://ml.blogs.losttech.software/What-New-In-Preview-6.4/ http://ml.blogs.losttech.software/What-New-In-Preview-6.4/ ), but I feel like there would be more interest from C# developers. There is a good C# Jupyter kernel out there.
- minxomat 7y ago
  Suspiciously absent is a comparison with Google Colab, which not only includes free GPU support (T4) and TPU support, but supports pretty much everything I've seen here. It allows access to the underlying VM system and has native integration with Google Drive to store and retrieve huge files (like checkpoints) and GitHub to version notebooks. Other similar services include Kaggle (way more locked down) and Baidu's free GPU powered notebook platform (only open to Chinese citizens)
- solidasparagus 7y ago
  Sharing free GPUs is amazing! GPU costs are such a huge blocker for people learning deep learning. But as a cloud provider, I would be worried about abuse by cryptojackers. I hope that's not a problem and this is sustainable.
  - applecrazy 7y ago
    Google has run a similar service for a while (Colab) and they have methods to detect crypto mining within their instances. I'm sure Paperspace does too.
  - fierarul 7y ago
    I don't understand this obsession with cloud-based deep learning for beginners. It creates this hyper-focus on cost: always remember to shut down your instances! With the occasional slippage which causes psychological (and / or financial) pain... The mood is all wrong for somebody entering a new (work-)field. Google Colab is decent and free. But you can do computing on cheaper hardware too. The CPU is good enough for learning and a GPU is not outside the budget of many people that presumably already own laptops and such. I know a local group that shares an i9 / dual GTX "server" and are learning on this shared hardware. I think it's great! I had a small budget for this learning curiosity and bought a Ryzen and a GTX with only 4GB of RAM. Got a job offer after a while which I had to turn down as it seemed to actually kill my interest in the field. Doing some small personal project now without much fuss to rekindle the fire. And using the CPU for it since it's so small.
doctorpangloss 7y ago
Google Collaboratory works pretty well as an alternative to this. In fact it works so well, like there’s so little time spent in IT, my total usage of it is quite low, because so much of what I’ve been communicating or seen communicated with notebooks is low quality in the first place.
- llampx 7y ago
  > so much of what I’ve been communicating or seen communicated with notebooks is low quality in the first place. Do you think there's a format which is better suited to sharing data stories?
jfim 7y ago
I tried browsing the documentation for several minutes, but it does not answer this question: which Jupyter kernels are supported? Is it only Python like Google colab?
- DTE 7y ago
  Because everything is running in a docker container behind the scenes we support any kernel you would like. We have a handful of pre-built containers and you can also add a custom container very easily or build one off of a base template such as the Jupyter R stack. Here is a list of some of the container we provide by default https://docs.paperspace.com/gradient/notebooks/notebook-containers https://docs.paperspace.com/gradient/notebooks/notebook-cont...
  - jaredscheib 7y ago
    I've gone ahead and added this response to the docs here: https://docs.paperspace.com/gradient/notebooks/about#containers https://docs.paperspace.com/gradient/notebooks/about#contain... :)
    - jfim 7y ago
      Awesome, thanks a lot!
Traster 7y ago
Sorry, I'm very new to this, how do notebooks/shared notebooks play with version control?
- psv1 7y ago
  Badly - under the hood jupyter notebooks are json which stores not only the code but all of the metadata as well. I know that there are tools that help with integrating jupyter and git but I just end up going back and forth between .py files in VSCode and notebooks in jupyter lab depending on what I'm working on.
  - jimmyvalmer 7y ago
    Aye, notebooks truly represent a wrong turn in scientific computing. The absence of version control alone is a showstopper, but it's insane generally to launch long-running estimations interactively via browser-based notebook interfaces, with outputs that are not readily greppable. But corporate philistines continue to ooh and ah over jupyter's visuals, and companies like Gradient are more than happy to cater to their FOMO.
    - DTE 7y ago
      I respectfully disagree :) I think that the notebook environment is a nice entry-point for lots of applications. We actually use it pretty regularly to launch more sophisticated experiments / multi-node training jobs, etc using our python SDK. You are right that versioning is still an issue and we largely punt on it by using the docker container (with layer commits on each notebook teardown) as the versioning mechanism. Maybe not the best solution but it does have it's advantages.
      - jimmyvalmer 7y ago
        If I were you, I'd have angrily railed against GP, so I salute the restraint. I don't approve of what you're doing but I also would happily trade positions with you (unemployed versus founding principal of a company -- any company -- even one whose mission is nonsense).
solidasparagus 7y ago
Can someone clarify how this works in terms of data? I see "5GB of persistent data", but that's nowhere near enough for my training data. Is there also a larger amount of non-persistent storage that I can download data to?
- DTE 7y ago
  The 5GB of persistent data is available for by default for all free accounts running on the free instances. You can easily upgrade your storage up to 1TB by upgrading you subscription within the console. We can also provide up to 4TB by opening a ticket.
  - deleted 7y ago
    [deleted]
snrji 7y ago
Does anyone now if this system supports Google Docs-like live editing? I'm struggling to find such a service.
- lrem 7y ago
  Google Colab? Disclaimer: I use Colab regularly as part of my job in Google.
  - mjn 7y ago
    I don't believe Colab is real-time live in the way that Google Docs is. I'm using it to teach a class and would love that kind of functionality, but unless I'm missing some option, when I make changes to the notebook, students who have it open don't see the changes immediately, only when they reload.
    - lrem 7y ago
      I haven't tried coding live, but comment threads do seem real-time. Probably wrong assumption on my part, sorry.
  - snrji 7y ago
    With which option! I have used Colab myself and I don't see live editing. I would like to see the cursor of the collaborators in real time while they edit. Also, the option to share the runtime.
    - lrem 7y ago
      I haven't tried coding live, but comment threads do seem real-time. Probably wrong assumption on my part, sorry.
- deleted 7y ago
  [deleted]
- kvlr 7y ago
  Can't say if it does but at https://nextjournal.com https://nextjournal.com we do. We also support running on GPUs.
deleted 7y ago
[deleted]
zerop 7y ago
Question: Why use this over Google colab?
sabalaba 7y ago
Hey HN, just wanted to post here to say that if you need more than 6 hours training (free tier limit) we offer a 4x GPU machine that is only $1.5/hour. Just wanted to let everybody know what kind of competition is out there while you're doing your price comparisons: https://lambdalabs.com/service/gpu-cloud https://lambdalabs.com/service/gpu-cloud
- JonathanFly 7y ago
  I'd be interested in whatever you could offer that's lowest specs in every respect except GPU ram. So 1 GPU, moderate to low everything else, but 24GB of RAM. Right now there's no great way to run those cheaply, even if I want to do something that might only take a few hours.
  - foobarbecue 7y ago
    Does a card like that exist? Last time I shopped for them (a few years ago) it didn't.
    - dkobran 7y ago
      It does and we offer it. It's called the P6000 and it includes 24GB GPU RAM. It's one of the most popular chips we offer for any kind of CV task as you can fit a ton of images, video frames etc. in GPU memory. In any case, here's a link to the full lineup we offer: https://gradient.paperspace.com/instances https://gradient.paperspace.com/instances
  - darkmighty 7y ago
    Perhaps you could try CPU training. I believe it should take about 10x longer on a many-core machine, but still might be worth it (if you require lots of RAM and can afford to wait). There are also various techniques to reduce memory usage.
  - dkobran 7y ago
    Paperspace team here. We offer a 24GB GPU option with minimal RAM. Check out the P6000 here: https://gradient.paperspace.com/instances https://gradient.paperspace.com/instances Hope that's what you're looking for :)
tastroder 7y ago
fyi: your chatbot makes a "popping" noise when scrolling down and puts "(2) New Messages" in the tab title. Both distract from your content.

199 points by hsikka 7y ago

Hey HN, really thrilled to see this launch. I was a Research Fellow at the Paperspace Advanced Technologies Group this summer and saw this project develop. Gradient Notebooks were an indispensable tool in the work I was doing, and had several advantages for my work over other notebook services. As a researcher, it’s great to see an emphasis being placed on starting projects with no fuss or issues around setting up infrastructure and sharing/forking models. Giving anyone access to free GPUs and powerful tooling seems like an incredible opportunity. I'd love to hear what you all think!

ArtWomb 7y ago
It's a home run in my book. Perfect for "Intro to ML" courses and allows students instant access to real env. Personally can use it for art & design experiments like GAN Breeder ;) Am wondering what to use for the data tier? Is there a dedicated backing cloud store supporting BigQuery style syntax? Import form S3 datasets?
- DTE 7y ago
  Hey there ! I'm one of the co-founders of Paperspace. We currently have a couple of options for data ingest (and more coming soon!). The current system provides a single persistent mount that you can access from any notebook/experiment/etc in the /storage directory. We also mount a special directory called /artifacts where you can pipe out any models, files, etc and they will be pushed to an S3-compatible object store. Checkout the docs here -> https://docs.paperspace.com/gradient/data/storage https://docs.paperspace.com/gradient/data/storage And happy to answer any other questions
  - MarkMMullin 7y ago
    OK, did a quick scan on free and paid tiers and wondering about ability to bring in specialized python libraries- in my case, https://github.com/opencog/link-grammar https://github.com/opencog/link-grammar which almost always requires hand building and migrating, and some custom packages on top of my own even more horrible C++ . - I have to run a jupyterhub ami on an ec2 instance right now
    - DTE 7y ago
      Yes, one of the differences in how we handle notebooks is that everything is actually run in a container behind the scenes. This means that is isn't just the .ipynb that we are hosting and if you install and dependencies, libraries, etc it will actually persist all of it inside of a container. This makes it much easier to share your work with others so that i.e. I could fork your notebook if you made it public and get all of the installed libraries and compiled dependencies by default. Hope that helps! Edit: we also have another tool called GradientCI (https://docs.paperspace.com/gradient/projects/gradientci https://docs.paperspace.com/gradient/projects/gradientci) that might also be of interest. Basically it lets you connect a GitHub repo directly to a project and you can use it to build your container automatically.
      - MarkMMullin 7y ago
        Gotcha - yah, I can containerize, it's not like I'm screwing with drivers or whatnot on the AMI - I'll keep an eye on you, not ready for GPU yet, as I can't even rationally define the vectors I'm extracting from the Link stuff. Best of luck to you, lot of people piling into that battleground, and the Dunning-Kreuger effect is rampant. :-) Side question, if you're willing to entertain it. Tired me tells a notebook to checkpoint, wanders off to bed, comes back next day and wonders why its taking an aeon to open the notebook..... oh yeah, damn, I've got gigs of images and panda crap in it . -- do you wrangle this problem, i.e. please don't save your notebooks as gihugic files representing a mental state you can't possibly remember ?
        DTE 7y ago
        I should also mention you can just as easily run these on CPU-backed instances as well. The GPU is not a hard requirement. As for checkpointing data, that is still a relatively difficult problem to solve and our current recommendation is to use a combination of the persitent /storage directory and the notebook home directory. There are definitely issues with doing 100K+ of small files and committing those to the primary docker layer. When you get to testing it out don't hesitate to reach out to use and we can try to see what the best solution is for your particular project. To date there isn't a "one size fits all" solution but we are working hard on making more intelligent choices behind the scenes to unblock some of these IO constraints.
  - lostmsu 7y ago
    Hi! Do you guys have a trademark for the name Gradient? I am developing a C#/.NET binding to TensorFlow with the same name: https://losttech.software/gradient.html https://losttech.software/gradient.html If you do, do you mind that clash of names? Secondly, would you be interested to invest in providing .NET-powered notebooks? I just got it working on Azure Notebooks for F# ( http://ml.blogs.losttech.software/What-New-In-Preview-6.4/ http://ml.blogs.losttech.software/What-New-In-Preview-6.4/ ), but I feel like there would be more interest from C# developers. There is a good C# Jupyter kernel out there.
minxomat 7y ago
Suspiciously absent is a comparison with Google Colab, which not only includes free GPU support (T4) and TPU support, but supports pretty much everything I've seen here. It allows access to the underlying VM system and has native integration with Google Drive to store and retrieve huge files (like checkpoints) and GitHub to version notebooks. Other similar services include Kaggle (way more locked down) and Baidu's free GPU powered notebook platform (only open to Chinese citizens)
solidasparagus 7y ago
Sharing free GPUs is amazing! GPU costs are such a huge blocker for people learning deep learning. But as a cloud provider, I would be worried about abuse by cryptojackers. I hope that's not a problem and this is sustainable.
- applecrazy 7y ago
  Google has run a similar service for a while (Colab) and they have methods to detect crypto mining within their instances. I'm sure Paperspace does too.
- fierarul 7y ago
  I don't understand this obsession with cloud-based deep learning for beginners. It creates this hyper-focus on cost: always remember to shut down your instances! With the occasional slippage which causes psychological (and / or financial) pain... The mood is all wrong for somebody entering a new (work-)field. Google Colab is decent and free. But you can do computing on cheaper hardware too. The CPU is good enough for learning and a GPU is not outside the budget of many people that presumably already own laptops and such. I know a local group that shares an i9 / dual GTX "server" and are learning on this shared hardware. I think it's great! I had a small budget for this learning curiosity and bought a Ryzen and a GTX with only 4GB of RAM. Got a job offer after a while which I had to turn down as it seemed to actually kill my interest in the field. Doing some small personal project now without much fuss to rekindle the fire. And using the CPU for it since it's so small.

doctorpangloss 7y ago

Google Collaboratory works pretty well as an alternative to this. In fact it works so well, like there’s so little time spent in IT, my total usage of it is quite low, because so much of what I’ve been communicating or seen communicated with notebooks is low quality in the first place.

llampx 7y ago
> so much of what I’ve been communicating or seen communicated with notebooks is low quality in the first place. Do you think there's a format which is better suited to sharing data stories?

jfim 7y ago

I tried browsing the documentation for several minutes, but it does not answer this question: which Jupyter kernels are supported? Is it only Python like Google colab?

DTE 7y ago
Because everything is running in a docker container behind the scenes we support any kernel you would like. We have a handful of pre-built containers and you can also add a custom container very easily or build one off of a base template such as the Jupyter R stack. Here is a list of some of the container we provide by default https://docs.paperspace.com/gradient/notebooks/notebook-containers https://docs.paperspace.com/gradient/notebooks/notebook-cont...
- jaredscheib 7y ago
  I've gone ahead and added this response to the docs here: https://docs.paperspace.com/gradient/notebooks/about#containers https://docs.paperspace.com/gradient/notebooks/about#contain... :)
  - jfim 7y ago
    Awesome, thanks a lot!

Traster 7y ago

Sorry, I'm very new to this, how do notebooks/shared notebooks play with version control?

psv1 7y ago
Badly - under the hood jupyter notebooks are json which stores not only the code but all of the metadata as well. I know that there are tools that help with integrating jupyter and git but I just end up going back and forth between .py files in VSCode and notebooks in jupyter lab depending on what I'm working on.
- jimmyvalmer 7y ago
  Aye, notebooks truly represent a wrong turn in scientific computing. The absence of version control alone is a showstopper, but it's insane generally to launch long-running estimations interactively via browser-based notebook interfaces, with outputs that are not readily greppable. But corporate philistines continue to ooh and ah over jupyter's visuals, and companies like Gradient are more than happy to cater to their FOMO.
  - DTE 7y ago
    I respectfully disagree :) I think that the notebook environment is a nice entry-point for lots of applications. We actually use it pretty regularly to launch more sophisticated experiments / multi-node training jobs, etc using our python SDK. You are right that versioning is still an issue and we largely punt on it by using the docker container (with layer commits on each notebook teardown) as the versioning mechanism. Maybe not the best solution but it does have it's advantages.
    - jimmyvalmer 7y ago
      If I were you, I'd have angrily railed against GP, so I salute the restraint. I don't approve of what you're doing but I also would happily trade positions with you (unemployed versus founding principal of a company -- any company -- even one whose mission is nonsense).

solidasparagus 7y ago

Can someone clarify how this works in terms of data? I see "5GB of persistent data", but that's nowhere near enough for my training data. Is there also a larger amount of non-persistent storage that I can download data to?

DTE 7y ago
The 5GB of persistent data is available for by default for all free accounts running on the free instances. You can easily upgrade your storage up to 1TB by upgrading you subscription within the console. We can also provide up to 4TB by opening a ticket.
- deleted 7y ago
  [deleted]

snrji 7y ago

Does anyone now if this system supports Google Docs-like live editing? I'm struggling to find such a service.

lrem 7y ago
Google Colab? Disclaimer: I use Colab regularly as part of my job in Google.
- mjn 7y ago
  I don't believe Colab is real-time live in the way that Google Docs is. I'm using it to teach a class and would love that kind of functionality, but unless I'm missing some option, when I make changes to the notebook, students who have it open don't see the changes immediately, only when they reload.
  - lrem 7y ago
    I haven't tried coding live, but comment threads do seem real-time. Probably wrong assumption on my part, sorry.
- snrji 7y ago
  With which option! I have used Colab myself and I don't see live editing. I would like to see the cursor of the collaborators in real time while they edit. Also, the option to share the runtime.
  - lrem 7y ago
    I haven't tried coding live, but comment threads do seem real-time. Probably wrong assumption on my part, sorry.
deleted 7y ago
[deleted]
kvlr 7y ago
Can't say if it does but at https://nextjournal.com https://nextjournal.com we do. We also support running on GPUs.

deleted 7y ago

[deleted]

zerop 7y ago

Question: Why use this over Google colab?

sabalaba 7y ago

Hey HN, just wanted to post here to say that if you need more than 6 hours training (free tier limit) we offer a 4x GPU machine that is only $1.5/hour. Just wanted to let everybody know what kind of competition is out there while you're doing your price comparisons: https://lambdalabs.com/service/gpu-cloud https://lambdalabs.com/service/gpu-cloud

JonathanFly 7y ago
I'd be interested in whatever you could offer that's lowest specs in every respect except GPU ram. So 1 GPU, moderate to low everything else, but 24GB of RAM. Right now there's no great way to run those cheaply, even if I want to do something that might only take a few hours.
- foobarbecue 7y ago
  Does a card like that exist? Last time I shopped for them (a few years ago) it didn't.
  - dkobran 7y ago
    It does and we offer it. It's called the P6000 and it includes 24GB GPU RAM. It's one of the most popular chips we offer for any kind of CV task as you can fit a ton of images, video frames etc. in GPU memory. In any case, here's a link to the full lineup we offer: https://gradient.paperspace.com/instances https://gradient.paperspace.com/instances
- darkmighty 7y ago
  Perhaps you could try CPU training. I believe it should take about 10x longer on a many-core machine, but still might be worth it (if you require lots of RAM and can afford to wait). There are also various techniques to reduce memory usage.
- dkobran 7y ago
  Paperspace team here. We offer a 24GB GPU option with minimal RAM. Check out the P6000 here: https://gradient.paperspace.com/instances https://gradient.paperspace.com/instances Hope that's what you're looking for :)

tastroder 7y ago

fyi: your chatbot makes a "popping" noise when scrolling down and puts "(2) New Messages" in the tab title. Both distract from your content.