10 ms·

I too use VS Code as my Jupyter platform (running remotely on a powerful EC2 instance with 32 CPUs and 256GB RAM — my own desktop is a 7 year old Intel core i7

by wenc 3y ago

I too use VS Code as my Jupyter platform (running remotely on a powerful EC2 instance with 32 CPUs and 256GB RAM — my own desktop is a 7 year old Intel core i7 with 8GB RAM). VS Code’s Remote extension is amazing, works over any SSH host and seamlessly blends local and remote. It’s also fast since the UI is local while the filesystem and execution is remote. The experience is a lot better than JupyterLab (which I am forced to use from time to time on SageMaker). The VS Code UI is cleaner plus I get a full language server which means I can rename variables and refactor fearlessly. I also get full access to VS Code plugins.

itairal 3y ago
That sounds amazing. I am going to try this today. I have been irrational to the point of not trying this because I just love the notebook so much even though I love VS Code too. I imagine you just would never go back to notebooks. That is nice the company pays for all that cloud compute but for an individual it would just seem more practical to build a beast of a machine.
kachnuv_ocasek 3y ago
I'm curious about your setup as I might have to do something similar soon due to my machine's performance constraints. I understand you can connect to a Jupyter server remotely, but how do you sync your code? Do you have your Git repo cloned on the remote host and just run Git commands over SSH? Or does VS Code have some kind of integration for remote file systems with version control?
- wenc 3y ago
  My code resides entirely on the remote EC2 instance while VS Code UI runs locally. There is no sync as such. You’re working off the remote copy via SSH. (I do back up my code periodically) VS code takes care of spinning up the remote Jupyter server. All I have to do is create a new .ipynb file and everything happens automatically. Execution and disk are remote, only the UI is local. This is the magic. It’s exactly like SSH except you have a rich client IDE in VS Code. The only data that moves over the network are your keystrokes and pastes and what is needed to display output in VS Code. You have to try it to see.
- algas 3y ago
  Check out the remote SSH (https://code.visualstudio.com/docs/remote/ssh https://code.visualstudio.com/docs/remote/ssh) extension. It makes everything really painless. I'm currently developing a website that resides in a docker container in a digitalocean server, and editing the code in VSCode feels literally no different at all to running it locally. If you have the SSH keys set up it's completely painless. The other nice thing about VSCode is that you can extend it with VSCode Neovim (https://marketplace.visualstudio.com/items?itemName=asvetliakov.vscode-neovim https://marketplace.visualstudio.com/items?itemName=asvetlia...), which runs a headless version of Neovim and allows you to do all the wonderful things that that entails, including stuff like VSCode's native multiple cursor implementation (and Lua config files!). All in all it's a great workflow, it's pretty light, and if you're paying for (or self-hosting) a beefy server it can turn any laptop into a powerhouse.
  - 0fox 3y ago
    This is great, thanks. Commenting to bookmark for future reference.
digdugdirk 3y ago
Really interesting setup. What kind of monthly expense does this run? And on a separate but related note, does it change the way you think about how you spend your time coding? (Assuming the costs do ramp up with usage such that time literally does equal money?)
- wenc 3y ago
  I’ve no idea what the cost is since the company pays for it — I do need the horsepower to run some really large models and I suspect most people don’t need this kind of spec. But for my company it’s just part of the cost of doing business. There’s no IT and I can provision instances of any type (subject to limits) at any time.
  - riialist 3y ago
    VSCode is free if you self-host it. There are corporate tiers with virtual desktops etc, and you can pay for services as GitHub pilot if you want. Anyhow there a wealth of free extensions to customize it and the setup is really straightforward. I have version management git in a private GitHub project for version management. You can add extensions for rendering graphs in good quality and importing and exporting stuff is easy. I have not been able to figure out why some people prefer to use Jupiter notebook as it is.
  - Jgrubb 3y ago
    Do you turn that lightbulb off at night when you clock out? Just curious.
    - deleted 3y ago
      [deleted]
- zokier 3y ago
  32 vCPU / 256 GB instances like r6a.8xlarge is about $900/month (r6ad which has local disk is about $100/month more), I don't see there being much other major costs with such setup?

by wenc 3y ago

itairal 3y ago

That sounds amazing. I am going to try this today. I have been irrational to the point of not trying this because I just love the notebook so much even though I love VS Code too. I imagine you just would never go back to notebooks. That is nice the company pays for all that cloud compute but for an individual it would just seem more practical to build a beast of a machine.

kachnuv_ocasek 3y ago

I'm curious about your setup as I might have to do something similar soon due to my machine's performance constraints. I understand you can connect to a Jupyter server remotely, but how do you sync your code? Do you have your Git repo cloned on the remote host and just run Git commands over SSH? Or does VS Code have some kind of integration for remote file systems with version control?

wenc 3y ago
My code resides entirely on the remote EC2 instance while VS Code UI runs locally. There is no sync as such. You’re working off the remote copy via SSH. (I do back up my code periodically) VS code takes care of spinning up the remote Jupyter server. All I have to do is create a new .ipynb file and everything happens automatically. Execution and disk are remote, only the UI is local. This is the magic. It’s exactly like SSH except you have a rich client IDE in VS Code. The only data that moves over the network are your keystrokes and pastes and what is needed to display output in VS Code. You have to try it to see.
algas 3y ago
Check out the remote SSH (https://code.visualstudio.com/docs/remote/ssh https://code.visualstudio.com/docs/remote/ssh) extension. It makes everything really painless. I'm currently developing a website that resides in a docker container in a digitalocean server, and editing the code in VSCode feels literally no different at all to running it locally. If you have the SSH keys set up it's completely painless. The other nice thing about VSCode is that you can extend it with VSCode Neovim (https://marketplace.visualstudio.com/items?itemName=asvetliakov.vscode-neovim https://marketplace.visualstudio.com/items?itemName=asvetlia...), which runs a headless version of Neovim and allows you to do all the wonderful things that that entails, including stuff like VSCode's native multiple cursor implementation (and Lua config files!). All in all it's a great workflow, it's pretty light, and if you're paying for (or self-hosting) a beefy server it can turn any laptop into a powerhouse.
- 0fox 3y ago
  This is great, thanks. Commenting to bookmark for future reference.

digdugdirk 3y ago

Really interesting setup. What kind of monthly expense does this run? And on a separate but related note, does it change the way you think about how you spend your time coding? (Assuming the costs do ramp up with usage such that time literally does equal money?)

wenc 3y ago
I’ve no idea what the cost is since the company pays for it — I do need the horsepower to run some really large models and I suspect most people don’t need this kind of spec. But for my company it’s just part of the cost of doing business. There’s no IT and I can provision instances of any type (subject to limits) at any time.
- riialist 3y ago
  VSCode is free if you self-host it. There are corporate tiers with virtual desktops etc, and you can pay for services as GitHub pilot if you want. Anyhow there a wealth of free extensions to customize it and the setup is really straightforward. I have version management git in a private GitHub project for version management. You can add extensions for rendering graphs in good quality and importing and exporting stuff is easy. I have not been able to figure out why some people prefer to use Jupiter notebook as it is.
- Jgrubb 3y ago
  Do you turn that lightbulb off at night when you clock out? Just curious.
  - deleted 3y ago
    [deleted]
zokier 3y ago
32 vCPU / 256 GB instances like r6a.8xlarge is about $900/month (r6ad which has local disk is about $100/month more), I don't see there being much other major costs with such setup?