3 ms·

Twitter's core operation is running a massively distributed database, as I understand it, where concurrency issues are pretty important to get right. See Data-I

Twitter's core operation is running a massively distributed database, as I understand it, where concurrency issues are pretty important to get right. See Data-Intensive Applications (2017), Part II: Distributed Systems (Replication, Partitioning, Transactions, etc.) for an overview. https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/ https://www.oreilly.com/library/view/designing-data-intensiv... Just guessing, you'd think Starlink would be more experienced in that area than either Tesla or SpaceX? I suppose there are things like remote software updates and internal working databases to manage at SpaceX and Tesla, but it seems managing satellite traffic is a closer match. Who knows, they might be, I really don't trust WaPo reporting anymore.

ed25519FUUU 4y ago
Everyone thinks that these Tesla engineers are going there to “evaluate” but nobody is really considering them going there to “steal” code to bring to Tesla.
- stingraycharles 4y ago
  This makes no sense, they wouldn’t even know what to look for and likely there is little that would be of use to them.
bpodgursky 4y ago
The suggestion algorithms are more of a technical challenge than concurrency, at Twitter scale. Tesla does a lot more ML than SpaceX. (and let's be honest, Twitter hasn't solved the concurrency issues, so I'm not sure there's much to review)
- permo-w 4y ago
  twitter is one of the slowest of all the big modern websites. half the time all it has to load is text and it still takes up to 30 seconds
  - stingraycharles 4y ago
    That’s a site design issue, not a backend / data storage / concurrency algorithms issue.
    - permo-w 4y ago
      how do you know but also yes it probably is just shitty overuse of worthless js
      - stingraycharles 4y ago
        Because the requests that render the actual content are very fast, it’s just overhead that makes things slow.
  - bpodgursky 4y ago
    My unread notifications always load 15-60 seconds after loading my timeline. Definitely does not seem like a pageload issue, it's some kind of BE processing latency.
jpgvm 4y ago
Tesla stores an absolutely silly amount of captured training data. They are very familiar with large distributed databases. What makes Manhattan special (Twitters internal database) isn't it's scale (FB, Google, etc operate DBs which much higher scale) but it's multi-engine paradigm and other cool features. The Tesla folk that are there probably aren't even DB specialists though, they are likely just there to evaluate the lay of the land and start the process of working out which teams are pulling their weight and which aren't.
- plazmatic 4y ago

ed25519FUUU 4y ago

Everyone thinks that these Tesla engineers are going there to “evaluate” but nobody is really considering them going there to “steal” code to bring to Tesla.

stingraycharles 4y ago
This makes no sense, they wouldn’t even know what to look for and likely there is little that would be of use to them.

bpodgursky 4y ago

The suggestion algorithms are more of a technical challenge than concurrency, at Twitter scale. Tesla does a lot more ML than SpaceX. (and let's be honest, Twitter hasn't solved the concurrency issues, so I'm not sure there's much to review)

permo-w 4y ago
twitter is one of the slowest of all the big modern websites. half the time all it has to load is text and it still takes up to 30 seconds
- stingraycharles 4y ago
  That’s a site design issue, not a backend / data storage / concurrency algorithms issue.
  - permo-w 4y ago
    how do you know but also yes it probably is just shitty overuse of worthless js
    - stingraycharles 4y ago
      Because the requests that render the actual content are very fast, it’s just overhead that makes things slow.
- bpodgursky 4y ago
  My unread notifications always load 15-60 seconds after loading my timeline. Definitely does not seem like a pageload issue, it's some kind of BE processing latency.

jpgvm 4y ago

Tesla stores an absolutely silly amount of captured training data. They are very familiar with large distributed databases. What makes Manhattan special (Twitters internal database) isn't it's scale (FB, Google, etc operate DBs which much higher scale) but it's multi-engine paradigm and other cool features. The Tesla folk that are there probably aren't even DB specialists though, they are likely just there to evaluate the lay of the land and start the process of working out which teams are pulling their weight and which aren't.

plazmatic 4y ago