3 ms·

>The best fresh graduates from the top colleges went to Google and Facebook in the last decade -- because they paid significantly more -- and Microsoft picked u

>The best fresh graduates from the top colleges went to Google and Facebook in the last decade -- because they paid significantly more -- and Microsoft picked up the rest. That seems shaky as a justification. I don’t have any data behind it, but in a world of eight billion people, it’s a hard sell that there’s only enough top talent for 3 companies. Plus, feel free to correct me but I don’t feel Meta has brought anything technically brilliant to the table lately. Google might be a better sell with alphafold and some other projects but none of it seems mainstream tech either. Maybe pixels if you want to stretch it, but search, maps, Gmail, YouTube, android, and even photos were there a decade ago already.

gxs 1y ago
I hear you, I might give people the benefit of the doubt too much, but I interpreted what he said more like while Facebook and Google were going out of their way to make sure they hired only the best, hiring was just not a point of emphasis for Microsoft and as a result it didn’t hire as many good people I think that point is fair - at the very least, Microsoft wasn’t in the conversation as far as where the typical CS grad wanted to work - FANG doesn’t even include Microsoft in the acronym (I kid, I realize this doesn’t actually mean anything)
ycombinatrix 1y ago
>I don’t feel Meta has brought anything technically brilliant to the table lately I was pleasantly surprised by Llama.
- klooney 1y ago
  I always hear bad things about its standings in the frontier model rankings, although I don't follow that close enough to know if that's real or not. What's cool about Llama?
  - ycombinatrix 1y ago
    Basically after years of raising money as a non-profit & training on everyone's data, "Open"AI rug-pulled the world and decided to keep their secret models secret to become a for-profit venture. I was pissed. Then Meta came out of nowhere to release Llama, a model with performance comparable to ChatGPT. I was able to download it & run it for free on my computer. Thank you Zucc, very cool.
alexey-salmin 1y ago
> Plus, feel free to correct me but I don’t feel Meta has brought anything technically brilliant to the table lately PyTorch? > Maybe pixels if you want to stretch it, but search, maps, Gmail, YouTube, android, and even photos were there a decade ago already. I don't disagree really, in the end all three suck: G, MS, FB. Google did in fact hire all the top talent but only to make sure they don't work on something good for someone else.
- rafram 1y ago
  PyTorch was released nine years ago.
singhrac 1y ago
Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question. A few projects just in ML: DeepMind has plenty enough for Google alone but Jax and the TPU project are technically ambitious and very strategically important for Google; they hired Adam Paszke away from Meta. Besides Gemini they have Gemma and Veo and they have a reputation in the industry of having extremely high MFU averages. From Facebook there's PyTorch but that's a whole cluster of projects (compiler, abstractions for many accelerators, torchao, torchtitan). They're also famous for DINO and SAM models, as well as many others by FAIR (e.g. Mask R-CNN).
- godelski 1y ago
  It's worth noting that many of those projects were the famous versions of papers, ones that were successful through scale rather than through innovation. I'll give a good example, here's essentially the same architecture as the 16x16 words ViT paper but a year earlier[0]. It's not even the first, they even mention two other works that used transformers on images. I'm all for scaling and it is important, but there's tons of papers like this that are greatly overshadowed because someone just scaled up and got a better benchmark. It's been getting out of hand... Fuck... I'm starting to sound like Schmidhuber [0] https://arxiv.org/abs/1911.03584 https://arxiv.org/abs/1911.03584
- high_na_euv 1y ago
  >Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question. You mean small startups founded by experienced engineers?
- deleted 1y ago
  [deleted]
mindwok 1y ago
You often don’t see what Google and Meta bring to the table because they aren’t trying to sell you products, they’re trying to glue your eyeballs to your screen and they’ve done extremely well at that.
- akho 1y ago
  Have they? Meta's main product, Facebook, seems to be much less dominant than it was a decade ago. I, in particular, used to use Facebook, Instagram, Google Search, and Gmail a decade ago. Now I don't, because the products have become unbearably worse.
  - mcny 1y ago
    We probably can't even name all their products. I doubt even anyone at these companies, even the CEOs can name all their products. One conspiracy theory I've heard is that because they were making so much money, they hired or at least recruited enough people so they would not have any serious competition. Also, a lot of things that look straightforward is probably a lot of work internally. For example, renaming Facebook to meta was likely not a simple one point story, right? Also, I bring this up every chance I get because it bothers me so much but making a worse product probably takes more effort in terms of man hours than making a good one. Look at this... .https://developer.chrome.com/blog/autoplay#media_engagement_index https://developer.chrome.com/blog/autoplay#media_engagement_...
    - akho 1y ago
      The comment that I was commenting said > they’re trying to glue your eyeballs to your screen and they’ve done extremely well at that I was saying that they haven't, in my experience. The number of products and the volume of work involved is important, but not relevant.

gxs 1y ago

I hear you, I might give people the benefit of the doubt too much, but I interpreted what he said more like while Facebook and Google were going out of their way to make sure they hired only the best, hiring was just not a point of emphasis for Microsoft and as a result it didn’t hire as many good people I think that point is fair - at the very least, Microsoft wasn’t in the conversation as far as where the typical CS grad wanted to work - FANG doesn’t even include Microsoft in the acronym (I kid, I realize this doesn’t actually mean anything)

ycombinatrix 1y ago

>I don’t feel Meta has brought anything technically brilliant to the table lately I was pleasantly surprised by Llama.

klooney 1y ago
I always hear bad things about its standings in the frontier model rankings, although I don't follow that close enough to know if that's real or not. What's cool about Llama?
- ycombinatrix 1y ago
  Basically after years of raising money as a non-profit & training on everyone's data, "Open"AI rug-pulled the world and decided to keep their secret models secret to become a for-profit venture. I was pissed. Then Meta came out of nowhere to release Llama, a model with performance comparable to ChatGPT. I was able to download it & run it for free on my computer. Thank you Zucc, very cool.

alexey-salmin 1y ago

> Plus, feel free to correct me but I don’t feel Meta has brought anything technically brilliant to the table lately PyTorch? > Maybe pixels if you want to stretch it, but search, maps, Gmail, YouTube, android, and even photos were there a decade ago already. I don't disagree really, in the end all three suck: G, MS, FB. Google did in fact hire all the top talent but only to make sure they don't work on something good for someone else.

rafram 1y ago
PyTorch was released nine years ago.

singhrac 1y ago

Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question. A few projects just in ML: DeepMind has plenty enough for Google alone but Jax and the TPU project are technically ambitious and very strategically important for Google; they hired Adam Paszke away from Meta. Besides Gemini they have Gemma and Veo and they have a reputation in the industry of having extremely high MFU averages. From Facebook there's PyTorch but that's a whole cluster of projects (compiler, abstractions for many accelerators, torchao, torchtitan). They're also famous for DINO and SAM models, as well as many others by FAIR (e.g. Mask R-CNN).

godelski 1y ago
It's worth noting that many of those projects were the famous versions of papers, ones that were successful through scale rather than through innovation. I'll give a good example, here's essentially the same architecture as the 16x16 words ViT paper but a year earlier[0]. It's not even the first, they even mention two other works that used transformers on images. I'm all for scaling and it is important, but there's tons of papers like this that are greatly overshadowed because someone just scaled up and got a better benchmark. It's been getting out of hand... Fuck... I'm starting to sound like Schmidhuber [0] https://arxiv.org/abs/1911.03584 https://arxiv.org/abs/1911.03584
high_na_euv 1y ago
>Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question. You mean small startups founded by experienced engineers?
deleted 1y ago
[deleted]

mindwok 1y ago

You often don’t see what Google and Meta bring to the table because they aren’t trying to sell you products, they’re trying to glue your eyeballs to your screen and they’ve done extremely well at that.

akho 1y ago
Have they? Meta's main product, Facebook, seems to be much less dominant than it was a decade ago. I, in particular, used to use Facebook, Instagram, Google Search, and Gmail a decade ago. Now I don't, because the products have become unbearably worse.
- mcny 1y ago
  We probably can't even name all their products. I doubt even anyone at these companies, even the CEOs can name all their products. One conspiracy theory I've heard is that because they were making so much money, they hired or at least recruited enough people so they would not have any serious competition. Also, a lot of things that look straightforward is probably a lot of work internally. For example, renaming Facebook to meta was likely not a simple one point story, right? Also, I bring this up every chance I get because it bothers me so much but making a worse product probably takes more effort in terms of man hours than making a good one. Look at this... .https://developer.chrome.com/blog/autoplay#media_engagement_index https://developer.chrome.com/blog/autoplay#media_engagement_...
  - akho 1y ago
    The comment that I was commenting said > they’re trying to glue your eyeballs to your screen and they’ve done extremely well at that I was saying that they haven't, in my experience. The number of products and the volume of work involved is important, but not relevant.