5 ms·

Polar opposite of my experience. To achieve the technical equivalent of changing a lightbulb, spend the entire day wrangling a dozen tools which are broken in d

Polar opposite of my experience. To achieve the technical equivalent of changing a lightbulb, spend the entire day wrangling a dozen tools which are broken in different ways, maintained by teams that no longer exist or have completely rolled over, only to arrive at the finish line and discover we don't use those lightbulbs anymore. Move things and break fast.

extr 2y ago
Yeah 100%. I found it immensely frustrating to be using tools with no community (except internally), so-so documentation, and features that were clearly broken in a way that would be unacceptable for a regular consumer product. If you have a question or error not covered by an internal search or documentation, good luck, you'll need it. Literally part of the reason I left the company.
- landedgentry 2y ago
  Well, you're supposed to read the code and figure it out. And if you can't, you're not good enough an engineer. According to people at Meta.
  - extr 2y ago
    People probably think you’re exaggerating but it’s true. Sometimes when I would get blocked the suggestion was to “read the source code” or “submit a fix” on some far flung internal project. Huge fucking waste of time and effort, completely unserious.
    - hnav 2y ago
      Doesn't sound like your type of company tbh, the flipside is that a "serious" company will often have broken bs too except now nobody is going to look at your contribution/fix.
      - KaiserPro 2y ago
        Pfft. "your type of company" sod off. Meta is only like this because its got a massive advertising revenue stream. the sheer amount of engineering time wasted because we don't document stuff is astounding. For example, how many message queue systems do we have? how many half arsed message queues have been created because they didn't know about FOQS?
        extr 2y ago
        Yes lmao, the number of times I would start off on some nominally useful task only to find out 3 weeks later that there is actually already a solution to that created by team XYZ that nobody in my reporting chain has ever heard of…(3 weeks was optimistic case, I remember my team member getting like 2 months in to some new data pipeline before finding out some tables already existed that did what he needed…)
        lclarkmichalek 2y ago
        I think a fair few of them were created because they knew a bit too much about FOQS
    - tru3_power 2y ago
      No matter what, tools will be broken. Having access to the source and being able to land a diff to fix the issue is awesome imo.
      - extr 2y ago
        That’s how open source already works by default. The difference is if an OSS tool is broken my boss doesn’t imply landing a fix is my responsibility on top of my regular job duties.
        majormajor 2y ago
        Working around it is somehow is. A huge part of my work has been plumbing and hacking around limitations in mediocre-at-best OSS tools. Lots of nonserious companies that take those issues as enough of a reason to move slowly. Many fewer serious ones where bad tooling is expected to be fixed, smoothed over, or replaced entirely in the interest of future dev time.
      - KaiserPro 2y ago
        > Having access to the source Yes, thats great. > being able to land a diff to fix the issue is awesome imo. yes, if its a one off. but for my last project that would involve spinning up many "XFNs" (multi-team chat fests) to argue that actually they don't want to have that change because of reason x,y and z. At which point you just give up and make a stupid fucking hack. So much is not about engineering excellence, its about trying to get people to accept change.
  - moandcompany 2y ago
    Same as Google. Many internal tools have painful interfaces and poor or documentation because the hiring bar was high and it was acceptable to assume that the user's skill level is high enough to figure it out. That attitude becomes a bigger problem when trying to sell tools to the public (e.g. Google Cloud Platform).
    - yodsanklai 2y ago
      As an outsider, I was always under the impression that Google had a tradition of engineering excellence (robust tools, clean and while tested code following strict guidelines), while Meta has more of a Hacker culture (move fast and break things).
      - moandcompany 2y ago
        Google also has traditions that created Broccoli Man: https://www.youtube.com/watch?v=3t6L-FlfeaI https://www.youtube.com/watch?v=3t6L-FlfeaI
  - fsociety 2y ago
    Or you know, go chat with the tool maintainers because they want people using them for impact.
  - KaiserPro 2y ago
    Welcome to meta! where everything is a murder mystery. Except you're not really sure if there has been a murder, or sometimes you wonder if you're the murderer, because at every turn you're told that you've been a bad dev for trying x,y and z
- zer0zzz 2y ago
  Agreed. I often get my work done using open source build instructions and tools and then when everything works I port it to internal infra. Other people are the opposite though, which for open source based code bases has a nasty side effect of the work having no upstream able tests!
loeg 2y ago
IMO there's a mix of a few really good, widely used, well-supported tools as well as a long tail of random tiny tools where the original team is gone that are cruftier.
uuddlrlrbaba 2y ago
Mmm breakfast
- grantsucceeded 2y ago
  haha the reason I stayed as long as i did
  - deleted 2y ago
    [deleted]
bozhark 2y ago
Move Smooth and Fix Things (tm) is our nonprofit corporation’s version of this atrocious motto.
ElonChrist 2y ago
[dead]
- ec109685 2y ago
  Large checkouts is a solved problem now https://github.com/facebook/sapling/blob/main/eden/fs/docs/Overview.md https://github.com/facebook/sapling/blob/main/eden/fs/docs/O...
aprilthird2021 2y ago
But you're both talking about different things. The tools are both often left in disuse, lacking documentation, etc. But they also have a really tight integration with each other that allows for unparalleled visibility and ability over enormous systems with many moving parts.

extr 2y ago

Yeah 100%. I found it immensely frustrating to be using tools with no community (except internally), so-so documentation, and features that were clearly broken in a way that would be unacceptable for a regular consumer product. If you have a question or error not covered by an internal search or documentation, good luck, you'll need it. Literally part of the reason I left the company.

landedgentry 2y ago
Well, you're supposed to read the code and figure it out. And if you can't, you're not good enough an engineer. According to people at Meta.
- extr 2y ago
  People probably think you’re exaggerating but it’s true. Sometimes when I would get blocked the suggestion was to “read the source code” or “submit a fix” on some far flung internal project. Huge fucking waste of time and effort, completely unserious.
  - hnav 2y ago
    Doesn't sound like your type of company tbh, the flipside is that a "serious" company will often have broken bs too except now nobody is going to look at your contribution/fix.
    - KaiserPro 2y ago
      Pfft. "your type of company" sod off. Meta is only like this because its got a massive advertising revenue stream. the sheer amount of engineering time wasted because we don't document stuff is astounding. For example, how many message queue systems do we have? how many half arsed message queues have been created because they didn't know about FOQS?
      - extr 2y ago
        Yes lmao, the number of times I would start off on some nominally useful task only to find out 3 weeks later that there is actually already a solution to that created by team XYZ that nobody in my reporting chain has ever heard of…(3 weeks was optimistic case, I remember my team member getting like 2 months in to some new data pipeline before finding out some tables already existed that did what he needed…)
      - lclarkmichalek 2y ago
        I think a fair few of them were created because they knew a bit too much about FOQS
  - tru3_power 2y ago
    No matter what, tools will be broken. Having access to the source and being able to land a diff to fix the issue is awesome imo.
    - extr 2y ago
      That’s how open source already works by default. The difference is if an OSS tool is broken my boss doesn’t imply landing a fix is my responsibility on top of my regular job duties.
      - majormajor 2y ago
        Working around it is somehow is. A huge part of my work has been plumbing and hacking around limitations in mediocre-at-best OSS tools. Lots of nonserious companies that take those issues as enough of a reason to move slowly. Many fewer serious ones where bad tooling is expected to be fixed, smoothed over, or replaced entirely in the interest of future dev time.
    - KaiserPro 2y ago
      > Having access to the source Yes, thats great. > being able to land a diff to fix the issue is awesome imo. yes, if its a one off. but for my last project that would involve spinning up many "XFNs" (multi-team chat fests) to argue that actually they don't want to have that change because of reason x,y and z. At which point you just give up and make a stupid fucking hack. So much is not about engineering excellence, its about trying to get people to accept change.
- moandcompany 2y ago
  Same as Google. Many internal tools have painful interfaces and poor or documentation because the hiring bar was high and it was acceptable to assume that the user's skill level is high enough to figure it out. That attitude becomes a bigger problem when trying to sell tools to the public (e.g. Google Cloud Platform).
  - yodsanklai 2y ago
    As an outsider, I was always under the impression that Google had a tradition of engineering excellence (robust tools, clean and while tested code following strict guidelines), while Meta has more of a Hacker culture (move fast and break things).
    - moandcompany 2y ago
      Google also has traditions that created Broccoli Man: https://www.youtube.com/watch?v=3t6L-FlfeaI https://www.youtube.com/watch?v=3t6L-FlfeaI
- fsociety 2y ago
  Or you know, go chat with the tool maintainers because they want people using them for impact.
- KaiserPro 2y ago
  Welcome to meta! where everything is a murder mystery. Except you're not really sure if there has been a murder, or sometimes you wonder if you're the murderer, because at every turn you're told that you've been a bad dev for trying x,y and z
zer0zzz 2y ago
Agreed. I often get my work done using open source build instructions and tools and then when everything works I port it to internal infra. Other people are the opposite though, which for open source based code bases has a nasty side effect of the work having no upstream able tests!

loeg 2y ago

IMO there's a mix of a few really good, widely used, well-supported tools as well as a long tail of random tiny tools where the original team is gone that are cruftier.

uuddlrlrbaba 2y ago

Mmm breakfast

grantsucceeded 2y ago
haha the reason I stayed as long as i did
- deleted 2y ago
  [deleted]

bozhark 2y ago

Move Smooth and Fix Things (tm) is our nonprofit corporation’s version of this atrocious motto.

ElonChrist 2y ago

[dead]

ec109685 2y ago
Large checkouts is a solved problem now https://github.com/facebook/sapling/blob/main/eden/fs/docs/Overview.md https://github.com/facebook/sapling/blob/main/eden/fs/docs/O...

aprilthird2021 2y ago

But you're both talking about different things. The tools are both often left in disuse, lacking documentation, etc. But they also have a really tight integration with each other that allows for unparalleled visibility and ability over enormous systems with many moving parts.