4 ms·

Usually disposable vm is enough, like windows sandbox. Googlers by default, though, aren’t allowed to run vms (VMware/virtualbox style) on their desktops/works

by galkk 3y ago

Usually disposable vm is enough, like windows sandbox. Googlers by default, though, aren’t allowed to run vms (VMware/virtualbox style) on their desktops/workstations.

xnx 3y ago
I thought most Googler's main machine was virtual. Maybe not for developers? https://cloud.google.com/blog/topics/developers-practitioners/googles-virtual-desktop-future https://cloud.google.com/blog/topics/developers-practitioner...
- UncleMeat 3y ago
  A huge number of people now do most of their dev work on cloud machines. I'm one of them. I don't even have a workstation anymore. Just ssh into the cloud box to build and run code. Eliminating general access to the internet on my cloud machine wouldn't affect me at all.
  - lopkeny12ko 3y ago
    I have never in my career seen a good implementation of cloud development. At every company I've ever worked for, "cloud development" is nothing but a Linux VM that gets provisioned for you in AWS, a file watcher that syncs your local files to the VM, and some extra CLI tools to run builds and tests on the VM. And every time I've used this, the overhead of verifying the syncing is up to date, the environment is consistent between my laptop and VM is the same, all this other mess...every time I end up just abandoning "cloud dev" and doing things on my laptop. God forbid you change a file in your cloud VM and forget to sync in the reverse direction. Not only is local development more reliable, but it's also faster (no remote network hop in the critical path of building things).
    - s1gnp0st 3y ago
      Why would you need files locally when most IDEs support a session over SSH that doesn't even involve a remote desktop tool? Just edit the files over there.
      - lopkeny12ko 3y ago
        Like an sshfs mount? Those are painfully slow and unreliable.
        leetcrew 3y ago
        idk how it's implemented, but vs code server works just fine for me. there's something to be said for building and running tests in the same environment your service targets. that's usually not going to be macOS or your favorite linux distro.
        coryrc 3y ago
        Not a sshfs Mount. The IDE has a process running on the other machine and communicate over ssh tunnel. Your text change actions are sent, I presume, and only ~visible portions of text are transmitted to your local machine. Uh, like a mainframe terminal :)
        saagarjha 3y ago
        It’s basically a sshfs mount
        s1gnp0st 3y ago
        No, it's not at all an sshfs mount. It's your IDE sending commands over to another instance of itself, and it's perfectly responsive as if operating locally. VSCode and IntelliJ support this very well.
        saagarjha 3y ago
        Idk what srcfs you’re using but it’s not the one that I’m familiar with. There are literally patches in IntelliJ/Android Studio to SIGSTOP the IDE so it doesn’t get all confused when your credentials expire.
        lopkeny12ko 3y ago
        Reading and manipulating files on my local disk is near-instantaneous. Not sure how you could ever replicate that level of latency as long as there's a remote network hop required for every read and write.
    - galkk 3y ago
      I don’t know about Facebook, but Google dev infra, it’s level of integration and convenience is seriously best in the world (ok, of what I’ve seen). All the problems that you’re talking about of keeping in sync etc are just non existent. You have root directory /google/ and magic happens there. Look up articles about srcfs, objfs, piper, forge. I honestly was looking if something exists open source so I can set it up at least for code editing, but all that I saw is just shit. When I compile at Google even using local build everything just works. Code on nfs network share + cachefilesd + all suggested flags for performance, with share at home nas (diskstation) on 1gbps lan + cachefilesd - constant issues like permissions. Simple test of cloning abseil library to that share and building takes 50%-80% more time than building locally. Same test with qmk firmware spice is just unbearable. Even just doing git clone there is awful due to lots of small files, I guess. Tried cifs - also bad.
    - vineyardmike 3y ago
      You can use GitHub code spaces (and competitors, incl. ones from google and open source ones) where you access VSCode in-browser and the entirety of the FS is on the VM - nothing local. They're quite widely available today. They work great in the (strong-internet availability) environments I've used them. Also, as an ex-googler, I can attest that google's implementation was enough that I never missed local development.
    - UncleMeat 3y ago
      The way it works at Google is that there is no syncing necessary. I edit code in a browser IDE. I build and run code on a cloud machine that is automatically synchronized with the state of a virtual workspace I'm editing with the browser IDE. No file containing code or test input is ever stored on my laptop's filesystem. This works great and is basically identical to working on my laptop directly except that my terminal is running over ssh rather than locally.
  - btgeekboy 3y ago
    How do you access your remote development machine? Is it a graphic connection / can you use an IDE? I love the idea of a powerful remote development machine but I’ve yet to find a tolerable RDP/VNC-type connection, and that leaves either something browser-based (VS Code maybe?) or something like JetBrains’ remote development tools which aren’t quite there yet.
    - filoleg 3y ago
      I use regular terminal SSH+VS Code remote machine feature. It pretty much feels like i am using VS Code with code on my local machine, except it is on the cloudtop. That covers heavy majority of use cases I care about, so I have zero need for graphical access or RDP. But in an ultra rare case I need it (happens only about once per year at most), I can just RDP using chrome remote desktop, and it works pretty well.
    - UncleMeat 3y ago
      A VSCode-esque web IDE for editing code running in my browser on my laptop. ssh for terminal. Works great.
- strstr 3y ago
  Desktop replacement is a VM. Physical desktops stopped allowing VMs by default as an L1TF mitigation.
  - saagarjha 3y ago
    You sure it’s not just because Santa doesn’t run on a VM?
- galkk 3y ago
  Think about cloudtop as about just box where you ssh. I was speaking about VMware workstation, virtualbox and others. Running them is (was) not allowed
kobalsky 3y ago
that's really interesting. is there any public document or manual describing their workstations and policies? like the SRE manual
- galkk 3y ago
  https://research.google/pubs/pub47055/ https://research.google/pubs/pub47055/

by galkk 3y ago

Usually disposable vm is enough, like windows sandbox. Googlers by default, though, aren’t allowed to run vms (VMware/virtualbox style) on their desktops/workstations.

xnx 3y ago

I thought most Googler's main machine was virtual. Maybe not for developers? https://cloud.google.com/blog/topics/developers-practitioners/googles-virtual-desktop-future https://cloud.google.com/blog/topics/developers-practitioner...

UncleMeat 3y ago
A huge number of people now do most of their dev work on cloud machines. I'm one of them. I don't even have a workstation anymore. Just ssh into the cloud box to build and run code. Eliminating general access to the internet on my cloud machine wouldn't affect me at all.
- lopkeny12ko 3y ago
  I have never in my career seen a good implementation of cloud development. At every company I've ever worked for, "cloud development" is nothing but a Linux VM that gets provisioned for you in AWS, a file watcher that syncs your local files to the VM, and some extra CLI tools to run builds and tests on the VM. And every time I've used this, the overhead of verifying the syncing is up to date, the environment is consistent between my laptop and VM is the same, all this other mess...every time I end up just abandoning "cloud dev" and doing things on my laptop. God forbid you change a file in your cloud VM and forget to sync in the reverse direction. Not only is local development more reliable, but it's also faster (no remote network hop in the critical path of building things).
  - s1gnp0st 3y ago
    Why would you need files locally when most IDEs support a session over SSH that doesn't even involve a remote desktop tool? Just edit the files over there.
    - lopkeny12ko 3y ago
      Like an sshfs mount? Those are painfully slow and unreliable.
      - leetcrew 3y ago
        idk how it's implemented, but vs code server works just fine for me. there's something to be said for building and running tests in the same environment your service targets. that's usually not going to be macOS or your favorite linux distro.
      - coryrc 3y ago
        Not a sshfs Mount. The IDE has a process running on the other machine and communicate over ssh tunnel. Your text change actions are sent, I presume, and only ~visible portions of text are transmitted to your local machine. Uh, like a mainframe terminal :)
        saagarjha 3y ago
        It’s basically a sshfs mount
        s1gnp0st 3y ago
        No, it's not at all an sshfs mount. It's your IDE sending commands over to another instance of itself, and it's perfectly responsive as if operating locally. VSCode and IntelliJ support this very well.
        saagarjha 3y ago
        Idk what srcfs you’re using but it’s not the one that I’m familiar with. There are literally patches in IntelliJ/Android Studio to SIGSTOP the IDE so it doesn’t get all confused when your credentials expire.
        lopkeny12ko 3y ago
        Reading and manipulating files on my local disk is near-instantaneous. Not sure how you could ever replicate that level of latency as long as there's a remote network hop required for every read and write.
  - galkk 3y ago
    I don’t know about Facebook, but Google dev infra, it’s level of integration and convenience is seriously best in the world (ok, of what I’ve seen). All the problems that you’re talking about of keeping in sync etc are just non existent. You have root directory /google/ and magic happens there. Look up articles about srcfs, objfs, piper, forge. I honestly was looking if something exists open source so I can set it up at least for code editing, but all that I saw is just shit. When I compile at Google even using local build everything just works. Code on nfs network share + cachefilesd + all suggested flags for performance, with share at home nas (diskstation) on 1gbps lan + cachefilesd - constant issues like permissions. Simple test of cloning abseil library to that share and building takes 50%-80% more time than building locally. Same test with qmk firmware spice is just unbearable. Even just doing git clone there is awful due to lots of small files, I guess. Tried cifs - also bad.
  - vineyardmike 3y ago
    You can use GitHub code spaces (and competitors, incl. ones from google and open source ones) where you access VSCode in-browser and the entirety of the FS is on the VM - nothing local. They're quite widely available today. They work great in the (strong-internet availability) environments I've used them. Also, as an ex-googler, I can attest that google's implementation was enough that I never missed local development.
  - UncleMeat 3y ago
    The way it works at Google is that there is no syncing necessary. I edit code in a browser IDE. I build and run code on a cloud machine that is automatically synchronized with the state of a virtual workspace I'm editing with the browser IDE. No file containing code or test input is ever stored on my laptop's filesystem. This works great and is basically identical to working on my laptop directly except that my terminal is running over ssh rather than locally.
- btgeekboy 3y ago
  How do you access your remote development machine? Is it a graphic connection / can you use an IDE? I love the idea of a powerful remote development machine but I’ve yet to find a tolerable RDP/VNC-type connection, and that leaves either something browser-based (VS Code maybe?) or something like JetBrains’ remote development tools which aren’t quite there yet.
  - filoleg 3y ago
    I use regular terminal SSH+VS Code remote machine feature. It pretty much feels like i am using VS Code with code on my local machine, except it is on the cloudtop. That covers heavy majority of use cases I care about, so I have zero need for graphical access or RDP. But in an ultra rare case I need it (happens only about once per year at most), I can just RDP using chrome remote desktop, and it works pretty well.
  - UncleMeat 3y ago
    A VSCode-esque web IDE for editing code running in my browser on my laptop. ssh for terminal. Works great.
strstr 3y ago
Desktop replacement is a VM. Physical desktops stopped allowing VMs by default as an L1TF mitigation.
- saagarjha 3y ago
  You sure it’s not just because Santa doesn’t run on a VM?
galkk 3y ago
Think about cloudtop as about just box where you ssh. I was speaking about VMware workstation, virtualbox and others. Running them is (was) not allowed

kobalsky 3y ago

that's really interesting. is there any public document or manual describing their workstations and policies? like the SRE manual

galkk 3y ago
https://research.google/pubs/pub47055/ https://research.google/pubs/pub47055/