4 ms·

There's more history than this. Disclaimer: Xoogler (2010-2017). When I first started the environment you used depended entirely on language. In the C++ and Py

by cletus 5mo ago

There's more history than this. Disclaimer: Xoogler (2010-2017). When I first started the environment you used depended entirely on language. In the C++ and Python space, there was the vim and emacs divide. With Java it was more complicated. Some still used vim/emacs but a lot of people used Eclipse. Now Eclipse was a real problem at Google because of the source control system. Java IDEs are primarily built to import binaries, specifically jars. In the outside world, these dependencies are managed via Ant (very early days), Maven/Gradle or the like. At Google there's a mono-repo (Perforce/Piper) and you check out parts of it locally and rely on the rest via a network connection (to SrcFS IIRC, it's been awhile). This was neat because you could edit a file locally and the dependencies would just recompile (via Blaze). So for Eclipse a whole lot of initialization had to be done and the IDE would fall over. A lot. It had a team of ~10 working on it at one point. Then somebody did a 20% project called magicjar. Magicjar took a Perforce client and built all the dependencies as jars that could be imported directly without parsing the entire source tree (which was usually huge). This made it possible, even preferred, to use IntelliJ, which is what I did. Magicjar was great. Other people actually made CLion work reasonably well with C++ too. That was nice. This was a much bigger undertaking with many more corner cases just given how C++ works (ie headers and templates). So checking out a client was relatively heavyweight, even with a minimal local tree. And, if you worked on Google3, you had to do this a lot. You might need to do a config file change. This was the real starting point for Cider because it was way nicer to do config file changes with it. Obviously I don't know where all this went from there. VS Studio as a Cider frontend? Ok, that was news to me. Engineers being unhappy when things change and when the slightest thing works differently is the least surprising thing I've ever heard. Oh it's worth adding that in my time many people didn't use Perforce (P4) directly. They used somebody else's project, which was a Git frontend for it, called Git5. I believe it was already being deprecated while I was still there. But Git5 modelled a P4 change as a branch so you could play around with your Git commits locally and then squash them into a single P4 change. I actually liked this a lot.

ncruces 5mo ago
There are mercurial and jujutsu frontends now.
wmedrano 5mo ago
We've moved on from Git5. Although it was a pain, I kind of liked that Git5 made the monorepo less monolithic to my editor.
- ncruces 5mo ago
  Do you mean local checkouts? There's a similar workflow with at least mercurial? Dunno about jujutsu.
  - nostrademons 5mo ago
    You don't really need it anymore - CitC let you do views (mapping just part of the monorepo into your filesystem via FUSE) since about 2013, and then that functionality just got built into Piper. When I returned in 2020 you'd have a file at the top of your source tree that included all the relevant file mappings as well as any Blaze flags needed to build the project, and you could just point your IDE at that and it'd map in just what you need. The history of Google's relationship to version control is even more interesting than editors - it went from CVS in 1998 to Perforce (P4) in 2000, then gcheckout and g4 in ~2006, then OverlayFS was invented in 2008, git5 came out in 2009, CitC obsoleted OverlayFS in ~2012, Piper built this all into the VCS in ~2013-2014, while I was gone from 2014-2020 apparently we got hg and jujutsu frameworks, and then when I got back in 2020 you'd just check out a .blazeproject from your IDE and everything would magically work. Many of these started as 20% projects (I used to have lunch with the guy who invented OverlayFS; interesting character and one of the best programmers I knew) and then got folded into the "official" way of doing things once grassroot adoption showed the execs that this was how people really wanted to work.
    - ncruces 5mo ago
      Haven't tried. But there are "IDEs" (Unity) that are really hostile to the idea that your project directory is not a fast local disk that they can both fill with garbage and use fsnotify on everything.
    - aiuto 5mo ago
      +1. CitC and it's relationship to version control (perforce/piper) is central to all development. Let's say there were 100M file in the monorepo (an underestimate). You obviously never want to do a git clone of that. But what if clone and checkout were free? That's what CITC did. Creating a new workspace took less than 1 second and got you a FUSE filesystem that looked like it had everything in the repo. But nothing was actually downloaded until you opened files. And your local changes were also stored in the service. And this was available to the CI machines. See where I am going. CI did not have to clone the repo and apply your branch. CI just had your changes available. If you were just testing your 10 files there was no cost to having 100M other source files that were unrelated to your project. This solves many problems that git worktrees attempt to address, but 10 years earlier, at less local disk cost, and more performant. I miss that.
BoredPositron 5mo ago
I still have nightmares about eclipse sometimes.
ngd 5mo ago
I started a touch before this in London. I recall before Blaze and git5 - every morning we had a ritual of checking out google3 and making sure we could get some sort of build working for the day so that we could then attempt to write some software on top of it. The builds in play were “Mach” and “quickie” or something like that. It was so painful we used to agree that we wouldn’t grab food or coffee or anything until we’d worked out what CL we should sync to for the day to do some work on. Pair programming was very in vogue and I used to get in a little later than some which was a great excuse to just hop on someone else’s machine who’d already gone through that pain
materielle 5mo ago
One important piece of context that might make all these stories less confusing for non-googlers: Code references are less important inside Google editors, because we have a code viewer tool inside the web browser. Most people read, explore, follow references, and share permalinks to the view-only tool. It’s a lot better than viewing code in GitHub. It’s super fast, is connected to language servers and can actually trace referenced, and overall has a million little features optimized for reading code. We also have a code reviewer tool, and a separate tool to run and view CI runs. So what’s left for the editor? Syntax highlighting? I would tend to view code, run tests and CI, and review in separate tools specialized for their specific use case. The code editor was just a place where I would type in my changes. I’d imagine this workflow feels weird to people who learned in one-stop-shop IntelliJ and GitHub world. But I can’t emphasize how much better these other tools were compared to GitHib. So a code editor that also lets me read, review, and test code didn’t really matter for me when I had a collection of smaller tools specialized for each individual task.
- richardboegli 5mo ago
  What tools available to the public would you say is similar to this workflow?
  - spwa4 5mo ago
    The only thing remotely close is a monorepo checkout ... with all the problems that come with that.
  - hollowcelery 5mo ago
    Sourcegraph is the closest external thing I've found to Google's internal web tool for viewing code.
  - steveklabnik 5mo ago
    The startup I'm at (ersc.io) is working in this space (version control more than the IDE side of things), because, in my opinion, there just plain isn't any.
benoau 5mo ago
> Engineers being unhappy when things change and when the slightest thing works differently is the least surprising thing I've ever heard. Gold.
dotwaffle 5mo ago
Cider (and p4/g4c etc) was amazing when I left back in 2020, I loved it so much, and truly miss it. I rejoined Google last year, and they'd replaced it with a VSCode clone that truly was just a glorified text editor and most were all-in on mercurial as a piper/citc shim -- I was only there for 5 months before I decided not to stay, and I never managed to get Go type definition hints working.
- aix1 5mo ago
  Google has also been systematically destaffing various language infra teams. https://www.linkedin.com/pulse/google-fires-entire-python-team-ahead-developer-conference-snpac https://www.linkedin.com/pulse/google-fires-entire-python-te... https://www.airs.com/blog/archives/670 https://www.airs.com/blog/archives/670 https://en.wikipedia.org/wiki/Google_Kythe https://en.wikipedia.org/wiki/Google_Kythe etc
  - hanwenn 5mo ago
    That is not quite the right word. For Python, the headcount was moved from the Bay Area (the most expensive place in the world to hire software engineers) to Munich (the most expensive place in Germany to hire SWEs.), for cost saving reasons.
    - dotwaffle 5mo ago
      If you're the author of https://github.com/hanwen/go-fuse/ https://github.com/hanwen/go-fuse/ -- thank you :D
    - VirusNewbie 5mo ago
      The problem is most SWEs in Germany are not as good as most SWEs in the Bay Area. :(
      - jbstjohn 5mo ago
        Most of the engineers making most of the tools being praised in this thread are in Germany, so I don't think that generalization quite holds. Even if the best SWEs are better in the Bay area, there's also a lot more competition for them, so Google in Germany might be able to get top 1% there (and in neighboring countries) but Google in the Bay Area is probably having a tough time getting even top 10%.
paul_h 5mo ago
I recall a couple of JetBrains staff visiting in 2008 to see IntelliJ's struggle with the size of checkout for AdWords. Soon after the reindexing front/center dialog moved from blocking and center of window, to a status line message that was non-bloking of edits at the bottom of the window. It may have been due anyway but was shown as problematic in the same moment.

by cletus 5mo ago

ncruces 5mo ago

There are mercurial and jujutsu frontends now.

wmedrano 5mo ago

We've moved on from Git5. Although it was a pain, I kind of liked that Git5 made the monorepo less monolithic to my editor.

ncruces 5mo ago
Do you mean local checkouts? There's a similar workflow with at least mercurial? Dunno about jujutsu.
- nostrademons 5mo ago
  You don't really need it anymore - CitC let you do views (mapping just part of the monorepo into your filesystem via FUSE) since about 2013, and then that functionality just got built into Piper. When I returned in 2020 you'd have a file at the top of your source tree that included all the relevant file mappings as well as any Blaze flags needed to build the project, and you could just point your IDE at that and it'd map in just what you need. The history of Google's relationship to version control is even more interesting than editors - it went from CVS in 1998 to Perforce (P4) in 2000, then gcheckout and g4 in ~2006, then OverlayFS was invented in 2008, git5 came out in 2009, CitC obsoleted OverlayFS in ~2012, Piper built this all into the VCS in ~2013-2014, while I was gone from 2014-2020 apparently we got hg and jujutsu frameworks, and then when I got back in 2020 you'd just check out a .blazeproject from your IDE and everything would magically work. Many of these started as 20% projects (I used to have lunch with the guy who invented OverlayFS; interesting character and one of the best programmers I knew) and then got folded into the "official" way of doing things once grassroot adoption showed the execs that this was how people really wanted to work.
  - ncruces 5mo ago
    Haven't tried. But there are "IDEs" (Unity) that are really hostile to the idea that your project directory is not a fast local disk that they can both fill with garbage and use fsnotify on everything.
  - aiuto 5mo ago
    +1. CitC and it's relationship to version control (perforce/piper) is central to all development. Let's say there were 100M file in the monorepo (an underestimate). You obviously never want to do a git clone of that. But what if clone and checkout were free? That's what CITC did. Creating a new workspace took less than 1 second and got you a FUSE filesystem that looked like it had everything in the repo. But nothing was actually downloaded until you opened files. And your local changes were also stored in the service. And this was available to the CI machines. See where I am going. CI did not have to clone the repo and apply your branch. CI just had your changes available. If you were just testing your 10 files there was no cost to having 100M other source files that were unrelated to your project. This solves many problems that git worktrees attempt to address, but 10 years earlier, at less local disk cost, and more performant. I miss that.

BoredPositron 5mo ago

I still have nightmares about eclipse sometimes.

ngd 5mo ago

I started a touch before this in London. I recall before Blaze and git5 - every morning we had a ritual of checking out google3 and making sure we could get some sort of build working for the day so that we could then attempt to write some software on top of it. The builds in play were “Mach” and “quickie” or something like that. It was so painful we used to agree that we wouldn’t grab food or coffee or anything until we’d worked out what CL we should sync to for the day to do some work on. Pair programming was very in vogue and I used to get in a little later than some which was a great excuse to just hop on someone else’s machine who’d already gone through that pain

materielle 5mo ago

One important piece of context that might make all these stories less confusing for non-googlers: Code references are less important inside Google editors, because we have a code viewer tool inside the web browser. Most people read, explore, follow references, and share permalinks to the view-only tool. It’s a lot better than viewing code in GitHub. It’s super fast, is connected to language servers and can actually trace referenced, and overall has a million little features optimized for reading code. We also have a code reviewer tool, and a separate tool to run and view CI runs. So what’s left for the editor? Syntax highlighting? I would tend to view code, run tests and CI, and review in separate tools specialized for their specific use case. The code editor was just a place where I would type in my changes. I’d imagine this workflow feels weird to people who learned in one-stop-shop IntelliJ and GitHub world. But I can’t emphasize how much better these other tools were compared to GitHib. So a code editor that also lets me read, review, and test code didn’t really matter for me when I had a collection of smaller tools specialized for each individual task.

richardboegli 5mo ago
What tools available to the public would you say is similar to this workflow?
- spwa4 5mo ago
  The only thing remotely close is a monorepo checkout ... with all the problems that come with that.
- hollowcelery 5mo ago
  Sourcegraph is the closest external thing I've found to Google's internal web tool for viewing code.
- steveklabnik 5mo ago
  The startup I'm at (ersc.io) is working in this space (version control more than the IDE side of things), because, in my opinion, there just plain isn't any.

benoau 5mo ago

> Engineers being unhappy when things change and when the slightest thing works differently is the least surprising thing I've ever heard. Gold.

dotwaffle 5mo ago

Cider (and p4/g4c etc) was amazing when I left back in 2020, I loved it so much, and truly miss it. I rejoined Google last year, and they'd replaced it with a VSCode clone that truly was just a glorified text editor and most were all-in on mercurial as a piper/citc shim -- I was only there for 5 months before I decided not to stay, and I never managed to get Go type definition hints working.

aix1 5mo ago
Google has also been systematically destaffing various language infra teams. https://www.linkedin.com/pulse/google-fires-entire-python-team-ahead-developer-conference-snpac https://www.linkedin.com/pulse/google-fires-entire-python-te... https://www.airs.com/blog/archives/670 https://www.airs.com/blog/archives/670 https://en.wikipedia.org/wiki/Google_Kythe https://en.wikipedia.org/wiki/Google_Kythe etc
- hanwenn 5mo ago
  That is not quite the right word. For Python, the headcount was moved from the Bay Area (the most expensive place in the world to hire software engineers) to Munich (the most expensive place in Germany to hire SWEs.), for cost saving reasons.
  - dotwaffle 5mo ago
    If you're the author of https://github.com/hanwen/go-fuse/ https://github.com/hanwen/go-fuse/ -- thank you :D
  - VirusNewbie 5mo ago
    The problem is most SWEs in Germany are not as good as most SWEs in the Bay Area. :(
    - jbstjohn 5mo ago
      Most of the engineers making most of the tools being praised in this thread are in Germany, so I don't think that generalization quite holds. Even if the best SWEs are better in the Bay area, there's also a lot more competition for them, so Google in Germany might be able to get top 1% there (and in neighboring countries) but Google in the Bay Area is probably having a tough time getting even top 10%.

paul_h 5mo ago

I recall a couple of JetBrains staff visiting in 2008 to see IntelliJ's struggle with the size of checkout for AdWords. Soon after the reindexing front/center dialog moved from blocking and center of window, to a status line message that was non-bloking of edits at the bottom of the window. It may have been due anyway but was shown as problematic in the same moment.