3 ms·

I don't have a read on this - do we know that Figma isn't doing difficult stuff that warrants proprietary solutions?

jiggawatts 3y ago
I just read through several thousand lines of code re-implementing the concept of a distributed queue from the ground up... for an application that has maybe a few hundred users. And doesn't need queues, at all. This issue is so pervasive that we've all just assumed that it must be necessary.
- djtango 3y ago
  I couldn't get the context of this response - is this application you read unrelated to the featured article? I just read the article and from what I can tell the Figma team made a somewhat reasonable sounding decision
  - jiggawatts 3y ago
    Yes, unrelated. My point was that wheel-reinvention is a curse of the software industry because it's just so easy to reinvent every wheel on a whim. DevOps and is no different. How many large orgs have their own build tooling, or some special sauce around large repos?
winrid 3y ago
Absolutely not. Citus would have solved this problem. Or move to MySQL and use PlanetScale etc. Second best option is ability to easily create prod environments, and then give those to your biggest customers (bigname.figma.com) etc. No single figma customer will go beyond an i3.metal for the DB, or the app.
- djtango 3y ago
  So I just read the article - they were on RDS so Citus wasn't an option. They also stated it was too risky to migrate data stores on the timeline they were working within Those all seem like measured engineering decisions AFAICT
  - winrid 3y ago
    That doesn't sound right. Do data dump from prod for initial sync and then setup replication from RDS to new cluster. Once synced do switch. Then you're off RDS and can shard on Citus.
    - raverbashing 3y ago
      Yes but they probably wouldn't want to migrate off RDS.
    - nathanappere 3y ago
      This doesn't work with the constraint of "staying on AWS" though.
      - winrid 3y ago
        Yes it does.
    - djtango 3y ago
      a) they didn't want to move off RDS b) this is a pretty big hand wave over migrating your persistence store on a moving product and moving engineering teams The coordination alone usually takes months
      - winrid 3y ago
        Yes, it takes months. They also spent months building this custom solution that they now need to maintain.
  - thibaut_barrere 3y ago
    I do agree here. The choice to prefer X over Y for hosting (no matter X and Y) often makes sense, changing hosting providers can take a bit of time & it's hard to fully assess the reality of the quality of support / security (again, not specifically writing about RDS or Citus, both are very good teams) beforehand, so it usually is safer to have a long probing period to move safely, which takes time, something they visibly didn't have much.

I just read through several thousand lines of code re-implementing the concept of a distributed queue from the ground up... for an application that has maybe a few hundred users. And doesn't need queues, at all. This issue is so pervasive that we've all just assumed that it must be necessary.

djtango 3y ago
I couldn't get the context of this response - is this application you read unrelated to the featured article? I just read the article and from what I can tell the Figma team made a somewhat reasonable sounding decision
- jiggawatts 3y ago
  Yes, unrelated. My point was that wheel-reinvention is a curse of the software industry because it's just so easy to reinvent every wheel on a whim. DevOps and is no different. How many large orgs have their own build tooling, or some special sauce around large repos?

winrid 3y ago

Absolutely not. Citus would have solved this problem. Or move to MySQL and use PlanetScale etc. Second best option is ability to easily create prod environments, and then give those to your biggest customers (bigname.figma.com) etc. No single figma customer will go beyond an i3.metal for the DB, or the app.

djtango 3y ago
So I just read the article - they were on RDS so Citus wasn't an option. They also stated it was too risky to migrate data stores on the timeline they were working within Those all seem like measured engineering decisions AFAICT
- winrid 3y ago
  That doesn't sound right. Do data dump from prod for initial sync and then setup replication from RDS to new cluster. Once synced do switch. Then you're off RDS and can shard on Citus.
  - raverbashing 3y ago
    Yes but they probably wouldn't want to migrate off RDS.
  - nathanappere 3y ago
    This doesn't work with the constraint of "staying on AWS" though.
    - winrid 3y ago
      Yes it does.
  - djtango 3y ago
    a) they didn't want to move off RDS b) this is a pretty big hand wave over migrating your persistence store on a moving product and moving engineering teams The coordination alone usually takes months
    - winrid 3y ago
      Yes, it takes months. They also spent months building this custom solution that they now need to maintain.
- thibaut_barrere 3y ago
  I do agree here. The choice to prefer X over Y for hosting (no matter X and Y) often makes sense, changing hosting providers can take a bit of time & it's hard to fully assess the reality of the quality of support / security (again, not specifically writing about RDS or Citus, both are very good teams) beforehand, so it usually is safer to have a long probing period to move safely, which takes time, something they visibly didn't have much.