5 ms·

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open source projects have access to and it's way better than Amazon internal tooling (is that 'industry standard' enough?). The sapling workflow is better than git. I was skeptical when I joined, but between no branches and excellent stack and merge support, it's just a better, more intuitive workflow. Eden is fast. Crazy fast, in fact, if you look at the size of the monorepo. The comment about small files is odd to me. Source code are small files. The entire monorepo is just small files. Now if we are going to complain about something, it would be the custom Android environment we've got. Android Studio integration is actually clunky, bordering on non-existent. Compared to writing vanilla Android code, which I have done for years before joining Meta, Meta folks wrapped almost every API, so all your pre-existing Android knowledge is useless. I've never been so unproductive when writing Android code.

PreachSoup 3y ago
I agree. The tooling is not as good as Google's(expected, Google's internal ides are incredible). But it's definitely better than the rest
MikeTheRocker 3y ago
Having also written Android code at Meta, I 100% agree.
- Solvency 3y ago
  What's the short version of why Meta did this, wrapping so much up in this internal nonstandard format?
  - nsonha 3y ago
    I'm guessing same to why everyone else does that: they want to abstract away the platform. To some people platform intricacies are uninteresting comparing to the problem they are solving, so their abstractions aim to express their problem domain, and hide everything else. For example for apps you may like to deal with abstractions that can express navigation and pages. They are the same across web, iOS, Android. Unlike Activity, Fragment, View (is that still a thing, I stopped doing android 10 years ago lol) and some slightly different set of abstrations over iOS.
  - billjings 3y ago
    Often, performance. When I was at Meta, if you could measure it, you could ship it and mark that as a win in your performance review. So a lot of projects got shipped off of A/B tested metric wins.
calvinmorrison 3y ago
Fastmail had great developer tooling. Give yourself about two minutes from a issued command via slack or CLI and you'd have a brand new tagged developer box the live system that was a self contained infrastructure in a box of everything from the imap server to the user interface. Another command you could get as many copies for whatever development branch, etc. The entire system replicated the real system (not user data but the software) and you could spin up test addresses and accounts and all sorts trivially. Beautiful!
- mandeepj 3y ago
  > get as many copies for whatever development branch Why would someone need a more than one copy of a dev/feature branch?
  - calvinmorrison 3y ago
    Because its free? You could have a beta server spun up on the fly to do some QA while your dev server is still kicking. It's just spinning servers off of git tags
kevan 3y ago
>and it's way better than Amazon internal tooling (is that 'industry standard' enough?). It's slowly changing but I wouldn't consider Amazon's tooling to be industry standard by any definition.
zug_zug 3y ago
>> Amazon internal tooling (is that 'industry standard' enough?). I'm not sure, maybe? When I say industry-standard tools I mean "Best you can purchase." So I mean a UI as good as github, a platform as good as AWS (meta's doesn't hold a candle to AWS), full-text log searching as good as splunk, chat as fast/searchable as slack (workplace doesn't hold a candle), video chat as fast/clear/clean as Zoom (workplace doesn't hold a candle) Some problems are harder at scale (feature toggles interactions just get harder with size), but some of them are just a mess of their own making (restarting taking 30m, butterfly rules having >20m+ delays, eden not being optimized to work with buck, which needs to read tons of files out of tons of directories)
- vlovich123 3y ago
  GitHub's developer workflow is a joke compared with the ticketing, CI, and code review tooling at Meta. I found the tools at Meta made it a lot smoother and faster to write and review code. Also, their automated flaky test detection, suppression, & ticket issuance/resolution is no joke. One of my favorite features. Their feature flagging system was also on point. Also, Workplace is a fantastic replacement of Wiki that surfaces relevant & interesting content to you. Their chat tools are way better than Google Chat / Microsoft Teams (not sure how it stacks up against Slack since I'm not a huge user of them). I agree the observability tools I interfaced within Meta was subpar a few years ago, but I think you're being ungenerous to take that and extend it to the coding tooling specifically.
  - erik_seaberg 3y ago
    Suppressing flaky tests is a little terrifying. Google would just retry them a few times, which is slow but only delays requesting a code review.
    - vlovich123 3y ago
      It determines flakiness in aggregate across all builds. It also does automatic retries but it does so for the purposes of marking a run as flaky which it then remembers. And it's your team's responsibility for making your tests stable. If they're not stable, they're not going to fail builds which is the correct response (a ticket is filed & it's your team's job to keep those tickets under control). The neat thing is also that if you fix the test and it's no longer flaky, it gets automatically added back in & the ticket is closed (in case you forgot). https://engineering.fb.com/2020/12/10/developer-tools/probabilistic-flakiness/ https://engineering.fb.com/2020/12/10/developer-tools/probab...
wilde 3y ago
> Eden is fast. Crazy fast, in fact, if you look at the size of the monorepo. You’re measuring the wrong thing. The user actions are slow and take several seconds to complete usually. No one gives a shit how big the repo is. They care how long their operation takes.
- lanza 3y ago
  There isn't a tool that can do this job faster. git takes a few seconds once you're at half a million commits. Facebook has many repos checked in in their entirety that are bigger than that.
  - wilde 3y ago
    That’s like saying cargo ships are fast because they carry lots of stuff. It’s still slow in absolute terms to get my stuff from China.
- optymizer 3y ago
  Compared to what? Not having a monorepo? Sure, but you run into different issues with that approach. Eden is the best solution there is for a monorepo of this size.
  - wilde 3y ago
    Yeah compared to not having a monorepo and buying more SSDs. The separate repos are much nicer to work in, but harder for the infra folks to manage. I miss having dataswarm as a separate repo. I will miss IG’s. I understand the tradeoff. That doesn’t mean it’s “fast”. It’s slow but the other alternatives are slower.
nappy-doo 3y ago
I've worked at Google and Meta (née FB). They are inferior.

PreachSoup 3y ago

I agree. The tooling is not as good as Google's(expected, Google's internal ides are incredible). But it's definitely better than the rest

MikeTheRocker 3y ago

Having also written Android code at Meta, I 100% agree.

Solvency 3y ago
What's the short version of why Meta did this, wrapping so much up in this internal nonstandard format?
- nsonha 3y ago
  I'm guessing same to why everyone else does that: they want to abstract away the platform. To some people platform intricacies are uninteresting comparing to the problem they are solving, so their abstractions aim to express their problem domain, and hide everything else. For example for apps you may like to deal with abstractions that can express navigation and pages. They are the same across web, iOS, Android. Unlike Activity, Fragment, View (is that still a thing, I stopped doing android 10 years ago lol) and some slightly different set of abstrations over iOS.
- billjings 3y ago
  Often, performance. When I was at Meta, if you could measure it, you could ship it and mark that as a win in your performance review. So a lot of projects got shipped off of A/B tested metric wins.

calvinmorrison 3y ago

Fastmail had great developer tooling. Give yourself about two minutes from a issued command via slack or CLI and you'd have a brand new tagged developer box the live system that was a self contained infrastructure in a box of everything from the imap server to the user interface. Another command you could get as many copies for whatever development branch, etc. The entire system replicated the real system (not user data but the software) and you could spin up test addresses and accounts and all sorts trivially. Beautiful!

mandeepj 3y ago
> get as many copies for whatever development branch Why would someone need a more than one copy of a dev/feature branch?
- calvinmorrison 3y ago
  Because its free? You could have a beta server spun up on the fly to do some QA while your dev server is still kicking. It's just spinning servers off of git tags

kevan 3y ago

>and it's way better than Amazon internal tooling (is that 'industry standard' enough?). It's slowly changing but I wouldn't consider Amazon's tooling to be industry standard by any definition.

zug_zug 3y ago

>> Amazon internal tooling (is that 'industry standard' enough?). I'm not sure, maybe? When I say industry-standard tools I mean "Best you can purchase." So I mean a UI as good as github, a platform as good as AWS (meta's doesn't hold a candle to AWS), full-text log searching as good as splunk, chat as fast/searchable as slack (workplace doesn't hold a candle), video chat as fast/clear/clean as Zoom (workplace doesn't hold a candle) Some problems are harder at scale (feature toggles interactions just get harder with size), but some of them are just a mess of their own making (restarting taking 30m, butterfly rules having >20m+ delays, eden not being optimized to work with buck, which needs to read tons of files out of tons of directories)

vlovich123 3y ago
GitHub's developer workflow is a joke compared with the ticketing, CI, and code review tooling at Meta. I found the tools at Meta made it a lot smoother and faster to write and review code. Also, their automated flaky test detection, suppression, & ticket issuance/resolution is no joke. One of my favorite features. Their feature flagging system was also on point. Also, Workplace is a fantastic replacement of Wiki that surfaces relevant & interesting content to you. Their chat tools are way better than Google Chat / Microsoft Teams (not sure how it stacks up against Slack since I'm not a huge user of them). I agree the observability tools I interfaced within Meta was subpar a few years ago, but I think you're being ungenerous to take that and extend it to the coding tooling specifically.
- erik_seaberg 3y ago
  Suppressing flaky tests is a little terrifying. Google would just retry them a few times, which is slow but only delays requesting a code review.
  - vlovich123 3y ago
    It determines flakiness in aggregate across all builds. It also does automatic retries but it does so for the purposes of marking a run as flaky which it then remembers. And it's your team's responsibility for making your tests stable. If they're not stable, they're not going to fail builds which is the correct response (a ticket is filed & it's your team's job to keep those tickets under control). The neat thing is also that if you fix the test and it's no longer flaky, it gets automatically added back in & the ticket is closed (in case you forgot). https://engineering.fb.com/2020/12/10/developer-tools/probabilistic-flakiness/ https://engineering.fb.com/2020/12/10/developer-tools/probab...

wilde 3y ago

> Eden is fast. Crazy fast, in fact, if you look at the size of the monorepo. You’re measuring the wrong thing. The user actions are slow and take several seconds to complete usually. No one gives a shit how big the repo is. They care how long their operation takes.

lanza 3y ago
There isn't a tool that can do this job faster. git takes a few seconds once you're at half a million commits. Facebook has many repos checked in in their entirety that are bigger than that.
- wilde 3y ago
  That’s like saying cargo ships are fast because they carry lots of stuff. It’s still slow in absolute terms to get my stuff from China.
optymizer 3y ago
Compared to what? Not having a monorepo? Sure, but you run into different issues with that approach. Eden is the best solution there is for a monorepo of this size.
- wilde 3y ago
  Yeah compared to not having a monorepo and buying more SSDs. The separate repos are much nicer to work in, but harder for the infra folks to manage. I miss having dataswarm as a separate repo. I will miss IG’s. I understand the tradeoff. That doesn’t mean it’s “fast”. It’s slow but the other alternatives are slower.

nappy-doo 3y ago

I've worked at Google and Meta (née FB). They are inferior.