4 ms·

I am at a complete loss as to how you get from "Of course Amazon's chip advantage is their own chip consumption and scale in AWS" to "being dismissive of Amazon

by jerf 2y ago

I am at a complete loss as to how you get from "Of course Amazon's chip advantage is their own chip consumption and scale in AWS" to "being dismissive of Amazon's developer advocacy". If they're doing better than the others, good for them. It still blindingly obvious that having the biggest cloud is a huge advantage for chip design and to successfully exploit savings because of that scale, not some sort of super amazing secret just now being revealed by IEEE.

alephnerd 2y ago
Both Google and MS had advantages that Amazon did not have in the mid-2010s in the ML space. Google had the advantage of owning the entire ML and Infra stack (TensorFlow, K8s, BERT, CNCF) and Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research. At the time, the Annapurna Labs acquisition was seen as a massive coin-toss because IBM went down a similar path a decade before and failed.
- dh2022 2y ago
  "Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research" I thought for a few minutes and I could not come up with an example of an ML technology that originated at MS Research and then spread outside MSFT. Care to give some examples? Thanks!
  - alephnerd 2y ago
    > Care to give some examples In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567 https://arxiv.org/abs/1803.05567 MT-DNN (2019) - https://arxiv.org/abs/1901.11504 https://arxiv.org/abs/1901.11504 MASS (2019) - https://arxiv.org/abs/1905.02450 https://arxiv.org/abs/1905.02450 VALL-E (2023) - https://arxiv.org/abs/2301.02111 https://arxiv.org/abs/2301.02111 VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370 https://arxiv.org/abs/2406.05370 While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language modeling, machine translation, text-to-speech, etc. It's in this segment that Microsoft Research shines moreso than even Google Research let alone other organizations because of their strong NLP background in Chinese (Microsoft Research Asia), South Asian languages (Microsoft Research India), Arabic (Microsoft Research's older work during the Iraq War), etc.

by jerf 2y ago

alephnerd 2y ago

Both Google and MS had advantages that Amazon did not have in the mid-2010s in the ML space. Google had the advantage of owning the entire ML and Infra stack (TensorFlow, K8s, BERT, CNCF) and Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research. At the time, the Annapurna Labs acquisition was seen as a massive coin-toss because IBM went down a similar path a decade before and failed.

dh2022 2y ago
"Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research" I thought for a few minutes and I could not come up with an example of an ML technology that originated at MS Research and then spread outside MSFT. Care to give some examples? Thanks!
- alephnerd 2y ago
  > Care to give some examples In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567 https://arxiv.org/abs/1803.05567 MT-DNN (2019) - https://arxiv.org/abs/1901.11504 https://arxiv.org/abs/1901.11504 MASS (2019) - https://arxiv.org/abs/1905.02450 https://arxiv.org/abs/1905.02450 VALL-E (2023) - https://arxiv.org/abs/2301.02111 https://arxiv.org/abs/2301.02111 VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370 https://arxiv.org/abs/2406.05370 While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language modeling, machine translation, text-to-speech, etc. It's in this segment that Microsoft Research shines moreso than even Google Research let alone other organizations because of their strong NLP background in Chinese (Microsoft Research Asia), South Asian languages (Microsoft Research India), Arabic (Microsoft Research's older work during the Iraq War), etc.