5 ms·

I'm surprised that they published the more recent paper discussed in part 2 [0] since, IIRC, TAO has supported atomic transactions since at least when I was the

I'm surprised that they published the more recent paper discussed in part 2 [0] since, IIRC, TAO has supported atomic transactions since at least when I was there a few years ago, but better late than never I suppose. I wonder if that means there's still hope that someone at the company will write a blog post about frameworks like: - EntSchema/EntPattern - EntQL - EntAction - PrivacyPolicy - Ent Deletion framework - Ent consistency checkers/fixers (I forget the name of the framework/tool) It's sad to think that people's only glimpse into these tools is through https://entgo.io https://entgo.io which is a really crude approximation of the real thing due to Go's lack of generics and expressiveness. Imagine if such amazing tools could be used at actually ethical companies. [0] https://www.micahlerner.com/2021/10/23/ramp-tao-layering-atomic-transactions-on-facebooks-online-tao-data-store.html https://www.micahlerner.com/2021/10/23/ramp-tao-layering-ato...

colesantiago 5y ago
why do tools like this and others always get invented at unethical companies?
- baby 5y ago
  Unethical from your point of view
  - colesantiago 5y ago
    still unethical no matter how you slice it.
    - polio 5y ago
      Pack it in, guys, colesantiago has solved morality.
- koolba 5y ago
  Developing tools costs money and unethical companies will always be more profitable than ethical ones as they have access to both ethical and unethical revenue streams. More profits means more money to throw at NIH projects.
  - nefitty 5y ago
    The less barriers the people in an organization perceive, the more likely they are to innovate, whether for the greater good or not.
    - int0x2e 5y ago
      I agree with your point, but I've personally also experienced the reverse as well - having very difficult constraints while being extremely motivated and well funded has often resulted in extremely innovative solutions. All of my examples are classified, sadly.
      - nefitty 5y ago
        I can see how more constraints can lead to more focused problem solving for sure.
- sizediterable 5y ago
  Unethical companies get rewarded for their unethical behavior instead of being held accountable in any meaningful way. They then use their vast resources to grow their engineering force at an incredible rate. Moving fast means they are also accruing technical debt at an astounding rate, which then leads to throwing more of their resources at people to focus solely on solving these scaling problems.
- jitl 5y ago
  Are there another kind of company that has 2 billion plus users? Can you store 500 edges * 2 billion users in a off the shelf database system? That was available in 2009? It is from scale and necessity that this kind of software is created.
cletus 5y ago
Technically those are unrelated to TAO. Those are all part of layers that sit on top of TAO. The TAO part is optional too. For example, you can put the Ent data access layer on top of MySQL directly.
- sizediterable 5y ago
  Right, I wasn't trying to imply a relationship. I only meant to say that if they're willing to write about old news for one piece of tech, then maybe they'd do it for another.
  - tehlike 5y ago
    Isn't ent pretty much an orm like n/hibernate
    - sizediterable 5y ago
      It is an ORM, but I wouldn't call it just an ORM. I don't have experience with NHibernate, but most ORMs I have experience with feel like just a way progammatically build queries. I found EntMutator to provide nice abstractions to couple data changes with business logic like mutation triggers and observers. Again, all the additional tooling built around the Ent framework adds to the value.
      - tehlike 5y ago
        Things like observers are fairly available in the orm land too. Same with mutators and unit of work and so on.
mhlakhani 5y ago
There’s now a paper on the deletion framework at least. [1] is a high level summary that links to the paper. [1] https://engineering.fb.com/2020/08/12/security/delf/ https://engineering.fb.com/2020/08/12/security/delf/
- sizediterable 5y ago
  Oh cool, I didn't know about this. I guess they tend to publish more infrastructure-oriented papers, which is understandable, but I wish there was more in general about the developer APIs that they enable and how they benefit productivity from a product engineer's point of view.
  - katrielalex 5y ago
    Lead author on the deletion paper here. Thanks for the feedback! I agree it would be cool to talk about the Ent APIs some more; they have a lot of obvious and less-obvious benefits. (For example, we are only able to add deletion constraints to Ents as described in the paper because the Ent specification language is expressive enough to do it.)
numair 5y ago
> I was there a few years ago > Imagine if such amazing tools could be used at actually ethical companies. Congratulations. How long did it take you to realize things are, in fact, as bad as the "haters" (to use a favorite term of the FB inner circle) said they were? On a more on-topic note, is entGo actually worth implementing as an ORM for Go? I've always been suspicious of most ORMs for all of the various reasons we all know about letting other people manage your database connections, in terms of performance/etc...
- sizediterable 5y ago
  I didn't leave due to the realization that the company was unethical, but there is a correlation there. I left because I got burnt out by the culture built around performance incentives. I didn't encounter anyone at the company who had inherently malicious or exploitative intent, but the culture drives the individual behaviors that in aggregate leads to all the harm caused by the company. That being said, everyone there, including myself should feel somewhat accountable, but it's really mostly on the leadership for having the power to change the culture but neglecting to do so. > is entGo actually worth implementing as an ORM for Go I can't speak to how valuable it would be when judged on its own. I just think that it currently does not and probably will not come close to delivering the same level of value as the FB Ent framework.
  - numair 5y ago
    I appreciate this reply. The sentiment you’ve shared is the same that a lot of current/former Facebook employees have privately emailed / called / etc and told me. The interesting thing is, it’s a lot of the seriously talented engineers and designers who care about these things the most, so there is hope for our future in what all of you will do based on these bad experiences. EntGo seems to be like it would be useful if used in combination with a bunch of other stuff that was built to take advantage of it, but that’s just my quick impression after looking through the docs. If entity modeling is your thing, I would be interested to know what you thought of Ecto in the Phoenix/Elixir ecosystem.
southerntofu 5y ago
> ethical companies What a cute oxymoron!
ahupp 5y ago
> IIRC, TAO has supported atomic transactions since at least when I was there a few years ago TAO has supported single-shard atomic updates forever but this work is about cross-shared transactions. I left a few years ago as well and this didn't exist at the time. Yeah I think a lot of people would enjoy writeups of those frameworks. There's also this very nice Ent framework, written by ex-FB engineer: https://ent.dev/ https://ent.dev/
- sizediterable 5y ago
  Ah, that clears things up. That's sounds huge! > There's also this very nice Ent framework, written by ex-FB engineer: https://ent.dev/ https://ent.dev/ Awesome, hope it succeeds!

colesantiago 5y ago

why do tools like this and others always get invented at unethical companies?

baby 5y ago
Unethical from your point of view
- colesantiago 5y ago
  still unethical no matter how you slice it.
  - polio 5y ago
    Pack it in, guys, colesantiago has solved morality.
koolba 5y ago
Developing tools costs money and unethical companies will always be more profitable than ethical ones as they have access to both ethical and unethical revenue streams. More profits means more money to throw at NIH projects.
- nefitty 5y ago
  The less barriers the people in an organization perceive, the more likely they are to innovate, whether for the greater good or not.
  - int0x2e 5y ago
    I agree with your point, but I've personally also experienced the reverse as well - having very difficult constraints while being extremely motivated and well funded has often resulted in extremely innovative solutions. All of my examples are classified, sadly.
    - nefitty 5y ago
      I can see how more constraints can lead to more focused problem solving for sure.
sizediterable 5y ago
Unethical companies get rewarded for their unethical behavior instead of being held accountable in any meaningful way. They then use their vast resources to grow their engineering force at an incredible rate. Moving fast means they are also accruing technical debt at an astounding rate, which then leads to throwing more of their resources at people to focus solely on solving these scaling problems.
jitl 5y ago
Are there another kind of company that has 2 billion plus users? Can you store 500 edges * 2 billion users in a off the shelf database system? That was available in 2009? It is from scale and necessity that this kind of software is created.

cletus 5y ago

Technically those are unrelated to TAO. Those are all part of layers that sit on top of TAO. The TAO part is optional too. For example, you can put the Ent data access layer on top of MySQL directly.

sizediterable 5y ago
Right, I wasn't trying to imply a relationship. I only meant to say that if they're willing to write about old news for one piece of tech, then maybe they'd do it for another.
- tehlike 5y ago
  Isn't ent pretty much an orm like n/hibernate
  - sizediterable 5y ago
    It is an ORM, but I wouldn't call it just an ORM. I don't have experience with NHibernate, but most ORMs I have experience with feel like just a way progammatically build queries. I found EntMutator to provide nice abstractions to couple data changes with business logic like mutation triggers and observers. Again, all the additional tooling built around the Ent framework adds to the value.
    - tehlike 5y ago
      Things like observers are fairly available in the orm land too. Same with mutators and unit of work and so on.

mhlakhani 5y ago

There’s now a paper on the deletion framework at least. [1] is a high level summary that links to the paper. [1] https://engineering.fb.com/2020/08/12/security/delf/ https://engineering.fb.com/2020/08/12/security/delf/

sizediterable 5y ago
Oh cool, I didn't know about this. I guess they tend to publish more infrastructure-oriented papers, which is understandable, but I wish there was more in general about the developer APIs that they enable and how they benefit productivity from a product engineer's point of view.
- katrielalex 5y ago
  Lead author on the deletion paper here. Thanks for the feedback! I agree it would be cool to talk about the Ent APIs some more; they have a lot of obvious and less-obvious benefits. (For example, we are only able to add deletion constraints to Ents as described in the paper because the Ent specification language is expressive enough to do it.)

numair 5y ago

> I was there a few years ago > Imagine if such amazing tools could be used at actually ethical companies. Congratulations. How long did it take you to realize things are, in fact, as bad as the "haters" (to use a favorite term of the FB inner circle) said they were? On a more on-topic note, is entGo actually worth implementing as an ORM for Go? I've always been suspicious of most ORMs for all of the various reasons we all know about letting other people manage your database connections, in terms of performance/etc...

sizediterable 5y ago
I didn't leave due to the realization that the company was unethical, but there is a correlation there. I left because I got burnt out by the culture built around performance incentives. I didn't encounter anyone at the company who had inherently malicious or exploitative intent, but the culture drives the individual behaviors that in aggregate leads to all the harm caused by the company. That being said, everyone there, including myself should feel somewhat accountable, but it's really mostly on the leadership for having the power to change the culture but neglecting to do so. > is entGo actually worth implementing as an ORM for Go I can't speak to how valuable it would be when judged on its own. I just think that it currently does not and probably will not come close to delivering the same level of value as the FB Ent framework.
- numair 5y ago
  I appreciate this reply. The sentiment you’ve shared is the same that a lot of current/former Facebook employees have privately emailed / called / etc and told me. The interesting thing is, it’s a lot of the seriously talented engineers and designers who care about these things the most, so there is hope for our future in what all of you will do based on these bad experiences. EntGo seems to be like it would be useful if used in combination with a bunch of other stuff that was built to take advantage of it, but that’s just my quick impression after looking through the docs. If entity modeling is your thing, I would be interested to know what you thought of Ecto in the Phoenix/Elixir ecosystem.

southerntofu 5y ago

> ethical companies What a cute oxymoron!

ahupp 5y ago

> IIRC, TAO has supported atomic transactions since at least when I was there a few years ago TAO has supported single-shard atomic updates forever but this work is about cross-shared transactions. I left a few years ago as well and this didn't exist at the time. Yeah I think a lot of people would enjoy writeups of those frameworks. There's also this very nice Ent framework, written by ex-FB engineer: https://ent.dev/ https://ent.dev/

sizediterable 5y ago
Ah, that clears things up. That's sounds huge! > There's also this very nice Ent framework, written by ex-FB engineer: https://ent.dev/ https://ent.dev/ Awesome, hope it succeeds!