8 ms·

John Carmack tweeted something that made me noodle on this too: >It is amusing to consider how much of the world you could serve something like Twitter to from

by sethev 4y ago

John Carmack tweeted something that made me noodle on this too: >It is amusing to consider how much of the world you could serve something like Twitter to from a single beefy server if it really was just shuffling tweet sized buffers to network offload cards. Smart clients instead of web pages could make a very large difference. [1] Very interesting to see the idea worked out in more detail. [1] https://twitter.com/id_aa_carmack/status/1350672098029694998 https://twitter.com/id_aa_carmack/status/1350672098029694998

varjag 4y ago
Isn't that what an OPA sorta kinda does.
threeseed 4y ago
> just shuffling tweet sized buffers to network offload cards Except that's not what it is doing at all. It assembles all the Tweets internally, applies an ML model to produce a finalised response to the user.
- seritools 4y ago
  > if it really was [which it isn't]
  - threeseed 4y ago
    But many people including the OP think it is. It’s like me running a web crawler on my phone and saying I can replace Google.
    - Dylan16807 4y ago
      If you were able to index the same amount of content you'd have a damn good alternative. And that's the level of this experiment.
      - threeseed 4y ago
        It is trivial to index the same amount of content as Google as the web is mostly static. The hard part is in being able to translate a search query into a list of pages. And that requires a level of sophistication that far exceeds a laptop.
        Dylan16807 4y ago
        It is not trivial at all. That is a huge amount of scraping. Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
        threeseed 4y ago
        It is trivial compared to the task of translating a user query into search results. It's akin to saying the magic behind OpenGPT is the dataset.
        Dylan16807 4y ago
        Keyword search is not that hard. Pagerank is not that hard. The comparable goal to the article is to be a search engine, not to fight google for best results.
        threeseed 4y ago
        > Keyword search is not that hard. Pagerank is not that hard. George Hotz, is that you ?
        Dylan16807 4y ago
        Keyword search and pagerank had working solutions decades ago. Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw. I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
    - seritools 4y ago
      What? The OP covers much more than "just shuffling tweet sized buffers to network offload cards"
  - to11mtm 4y ago
    Maybe it should be, though? I sometimes wonder how much value ML provides vs a proper sort function for anything but advertising.
    - threeseed 4y ago
      It's an easy thought experiment. Do you think that the most successful web companies in the world with arguably the best people i.e. Amazon, Facebook, Instagram, TikTok, LinkedIn, Pinterest, Youtube, Netflix, Snapchat etc. have no idea what they are doing. That the highly complex, expensive and latency impacting recommendation systems could be replaced by trivial sorting. Or maybe they do work, do translate to increased usage and do significantly impact revenue.
      - fzingle 4y ago
        Maybe they are considering that we shouldn't build and optimize our society solely for the purposes of maximizing revenue. Would it be better to live in a world where Twitter (for example) existed because it is a useful thing and not because it might make lots of money?
        afrodc_ 4y ago
        Doesn't it lose a lot of money and its usefulness is directly correlated with its current massive usage.
        saagarjha 4y ago
        It ran about break even until very recently.
      - brazzledazzle 4y ago
        A lot of people don't realize that Dunning-Kruger can catch any of us unaware. It's easy to look at a problem, think about the surface level challenges you'd have building it and come to the conclusion that you could do it better or simpler.
- PaulDavisThe1st 4y ago
  which arrives at a browser running Tweak New Twitter as a browser extension, and that strips the response to just what the user actually wants to see. Effficiency!
- nimish 4y ago
  Great, staple a few ML accelerators to your NIC. Nvidia sells them! You could build an entire supercomputer style setup 100% optimized for Twitter data movement and computation with COTS hardware IMO. I strongly doubt that entire datacenters need to be used if and only if Twitter obsessively optimized for hardware usage efficiency over everything else. In reality they don't and make some pretty big compromises to actually get stuff built. Hardware is cheap, people are not.
  - threeseed 4y ago
    a) No one has said that Twitter doesn't use ML accelerators. b) No one has said Twitter operates entire data centres. c) You need more than just NICs and ML accelerators to built a Twitter timeline. You need to rank the content, determine appropriate ads and combine them together. You can't do that in your network card.
    - pg314 4y ago
      > No one has said Twitter operates entire data centres. They do though. 3 in the US alone (https://www.datacenterdynamics.com/en/news/report-elon-musk-considers-shutting-twitters-sacramento-data-center-leaving-just-two/ https://www.datacenterdynamics.com/en/news/report-elon-musk-...).
    - nimish 4y ago
      Good news, Nvidia will now sell you a GPU+NIC+CPU with competitive performance: https://developer.nvidia.com/blog/accelerating-data-center-ai-with-the-nvidia-converged-accelerator-developer-kit/ https://developer.nvidia.com/blog/accelerating-data-center-a... Twitter operates a handful of datacenters because their scale is such that it makes sense.

by sethev 4y ago

varjag 4y ago

Isn't that what an OPA sorta kinda does.

threeseed 4y ago

> just shuffling tweet sized buffers to network offload cards Except that's not what it is doing at all. It assembles all the Tweets internally, applies an ML model to produce a finalised response to the user.

seritools 4y ago
> if it really was [which it isn't]
- threeseed 4y ago
  But many people including the OP think it is. It’s like me running a web crawler on my phone and saying I can replace Google.
  - Dylan16807 4y ago
    If you were able to index the same amount of content you'd have a damn good alternative. And that's the level of this experiment.
    - threeseed 4y ago
      It is trivial to index the same amount of content as Google as the web is mostly static. The hard part is in being able to translate a search query into a list of pages. And that requires a level of sophistication that far exceeds a laptop.
      - Dylan16807 4y ago
        It is not trivial at all. That is a huge amount of scraping. Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
        threeseed 4y ago
        It is trivial compared to the task of translating a user query into search results. It's akin to saying the magic behind OpenGPT is the dataset.
        Dylan16807 4y ago
        Keyword search is not that hard. Pagerank is not that hard. The comparable goal to the article is to be a search engine, not to fight google for best results.
        threeseed 4y ago
        > Keyword search is not that hard. Pagerank is not that hard. George Hotz, is that you ?
        Dylan16807 4y ago
        Keyword search and pagerank had working solutions decades ago. Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw. I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
  - seritools 4y ago
    What? The OP covers much more than "just shuffling tweet sized buffers to network offload cards"
- to11mtm 4y ago
  Maybe it should be, though? I sometimes wonder how much value ML provides vs a proper sort function for anything but advertising.
  - threeseed 4y ago
    It's an easy thought experiment. Do you think that the most successful web companies in the world with arguably the best people i.e. Amazon, Facebook, Instagram, TikTok, LinkedIn, Pinterest, Youtube, Netflix, Snapchat etc. have no idea what they are doing. That the highly complex, expensive and latency impacting recommendation systems could be replaced by trivial sorting. Or maybe they do work, do translate to increased usage and do significantly impact revenue.
    - fzingle 4y ago
      Maybe they are considering that we shouldn't build and optimize our society solely for the purposes of maximizing revenue. Would it be better to live in a world where Twitter (for example) existed because it is a useful thing and not because it might make lots of money?
      - afrodc_ 4y ago
        Doesn't it lose a lot of money and its usefulness is directly correlated with its current massive usage.
        saagarjha 4y ago
        It ran about break even until very recently.
    - brazzledazzle 4y ago
      A lot of people don't realize that Dunning-Kruger can catch any of us unaware. It's easy to look at a problem, think about the surface level challenges you'd have building it and come to the conclusion that you could do it better or simpler.
PaulDavisThe1st 4y ago
which arrives at a browser running Tweak New Twitter as a browser extension, and that strips the response to just what the user actually wants to see. Effficiency!
nimish 4y ago
Great, staple a few ML accelerators to your NIC. Nvidia sells them! You could build an entire supercomputer style setup 100% optimized for Twitter data movement and computation with COTS hardware IMO. I strongly doubt that entire datacenters need to be used if and only if Twitter obsessively optimized for hardware usage efficiency over everything else. In reality they don't and make some pretty big compromises to actually get stuff built. Hardware is cheap, people are not.
- threeseed 4y ago
  a) No one has said that Twitter doesn't use ML accelerators. b) No one has said Twitter operates entire data centres. c) You need more than just NICs and ML accelerators to built a Twitter timeline. You need to rank the content, determine appropriate ads and combine them together. You can't do that in your network card.
  - pg314 4y ago
    > No one has said Twitter operates entire data centres. They do though. 3 in the US alone (https://www.datacenterdynamics.com/en/news/report-elon-musk-considers-shutting-twitters-sacramento-data-center-leaving-just-two/ https://www.datacenterdynamics.com/en/news/report-elon-musk-...).
  - nimish 4y ago
    Good news, Nvidia will now sell you a GPU+NIC+CPU with competitive performance: https://developer.nvidia.com/blog/accelerating-data-center-ai-with-the-nvidia-converged-accelerator-developer-kit/ https://developer.nvidia.com/blog/accelerating-data-center-a... Twitter operates a handful of datacenters because their scale is such that it makes sense.