3 ms·

There was no market reaction when Deepseek r1 was released. There was no market reaction when Deepseek v3 was released. There was a massive market reaction wh

There was no market reaction when Deepseek r1 was released. There was no market reaction when Deepseek v3 was released. There was a massive market reaction when a Deepseek app was released. There is too much damn irrationality in the AI and AI adjacent market right now, and pop articles like this are only making it worse. Jevons Paradox absolutely holds for model development, and Deepseek should (and is - all my cybersecurity startup peers are investigating the feasibility of building their own models now) be viewed as an opportunity for smaller or medium stage companies to build their own competitive domain specific models, while not having to pay what is essentially protection money to OpenAI or Anthropic. The only people at risk with the democratization of model development are the investors in foundational model companies --------- Also, I hope Deepseek FINALLY reprioritizes distributed systems education in CS. There are too many "ML Engineers" who do not know the basics of OS or Hardware Engineering (eg. Cannot optimize Mellanox hardware, really dig into workload optimization, etc) and are thus burning resources inefficiently. Most MLEs I meet now just import a package and glue code together WITHOUT also understanding the performance capabilities that their compute has. Most universities in the US either don't require OS or CompArch classes for CS majors (eg. Harvard) or dumb them down significantly (eg. Not going deep into Linux kernel implementation, scheduling, etc). This is why the entire cybersecurity industry has largely shifted to Israel and India, and the skillset overlaps significantly with MLOps and MLEng (if you know how to implement spinlocks in C you can easily be taught CUDA) <"Old Man yelling at clouds" rant over>

high_na_euv 2y ago
Or people in charge cannot properly evaluate things :) It is very possible, when things require advanced technical knowledge and experience ><"Old Man yelling at clouds" rant over> They are too busy deploying ruby apps and juggling jsons over https to mess with ugly cpp, segfaults, kernel details, hardware intrinsics and semiconductors
- alephnerd 2y ago
  At the scale that the selloffs happened, that meant there were some shifts by institutional investors. These are the kinds of managers that would have technical people on retainer to consult (eg. Guidepoint). > They are too busy deploying ruby apps and juggling jsons over https Ruby is "legacy" as well nowadays. Most younger devs I meet tend to really only understand JS and Python, and that too while heavily relying on outside packages or dependencies. Not a bad thing per say, but if a VC like me has deeper knowledge about OS internals or Mellanox tuning than some of the (American) MLEs in companies they've done due diligence on, something's very wrong with the talent pipeline.
- highcountess 2y ago
  Humans are generally not very good at properly evaluating things in general. They are exceptionally good at believing themselves to be very good at properly evaluating things though. It is why empires collapse from miscalculations about war by their experts, it is the foundational mental model underlying communism that simply cannot work but keeps being attempted, and it is how countries can be destroyed in every which way while the experts maintain that everything is just fine. Never underestimate the powers of the ego, the destroyer of worlds. I mean they don’t know what they are doing, but we have advanced technical knowledge and experience, so we should clearly be in charge of all things, including those beyond our narrow scope of advanced technical knowledge and experience.
  - dagss 2y ago
    To be fair, capitalism doesn't really work either, you get a very unstable systems with cycles (which people try to stabilize in ad hoc ways), you optimize for the wrong things long term (right now producing much more goods than we need, without factoring in devastating costs that climate change will bring in a few decades, and destruction of nature and eradication of species) and when things crash many people are hurt. And it practically needs wars and destruction to reset wealth distribution every 50 to 100 years. No communism doesn't work either. I wish there was more alternatives. And some of the usual examples of "socialism doesn't work" might have worked if US hadn't interfered. E.g. in Chile, no system would have survived US actively supporting Pinochet. We will never know how socialism would have played out there if US (and Soviet too) left it alone.
mhog_hn 2y ago
What resources are suggested to get started in this area? Assuming a distributed systems web dev background
- alephnerd 2y ago
  UIUC CS241 - https://pkgamma.github.io/cs241-site-clone-fa19/ https://pkgamma.github.io/cs241-site-clone-fa19/ UCB's CS162 - https://cs162.org/ https://cs162.org/ MIT 6.1810 - https://kaashoek.github.io/65810-2023/ https://kaashoek.github.io/65810-2023/ These were tablestake courses that teach you the basics of OS and Systems Development (eg. Synchronization, Scheduling, etc). After you understand that, then you'd start digging into HPC classes AND then ideally go into ML. At least, this was the workflow and mental model I followed a decade+ ago.
  - mhog_hn 2y ago
    Thanks!
InkCanon 2y ago
Jevons paradox has been quoted to death here - it really needs pointing out that it points to a commodity in general, not Nvidia. Nvidia is not selling just coal, it's selling a super purified ultra powerful coal only it can make. Jevons paradox just says coal (compute) usage will increase. Not that this specific type of coal will. And this is quite separate from the business of it. For example Jevons paradox is quite apparent in the airline industry, but airlines are notorious for possibly never having net made a profit because of high capex and aggressive pricing competition. Jevons paradox is not any reason at all for a runaway valuation of a company.
- alephnerd 2y ago
  I agree, but the existing moat that Nvidia has due to CUDA and the entire HPC ecosystem still holds. And this is why it's relevant to treat Nvidia as the majority of the GPU space segment. Furthermore, it's Nvidia that owns Mellanox - which is the owner of the Infiniband IP which is used in just about every DC or cluster, because no other vendor came as close for interconnect performance
  - InkCanon 2y ago
    You're on the dot here. Most people are looking at it wrong. Someone here once put it succinctly - Nvidia isn't selling GPUs, it's selling Infiniband clusters. The whole NVLink/CUDA ecosystem is the key, and a mass of separate players (Cerebras, Google TPUs for inference, Mojo for CUDA, AMD/Intel/Huawei etc for GPUs, etc) are very unlikely to break that. In fact NVLink is so powerful, based on the DeepSeek paper it appears to be nerfed (classic Nvidia market segmentation strategy), and a big part of squeezing their compute was hacking around that. That said, the reason I don't invest Nvidia is that they're selling one more thing - AI legitimacy. Every tech shop is buying Nvidia because all their CEOs need to go to the board and say, we've got so and so AI strategy. Sure as hell Meta isn't spending 50 billion a year because AI makes them so much damn money (supposedly ML saved their ass when Apple put on App Tracking Transparency, but 99% suds it's not the ML that needs a gigawatt GPU cluster to run). And the very real calculus for investors/CEOs is that if DeepSeek made this on a thousandth of the budget (conflicting reports but high probably DS has a fraction of the compute), whYs stopping Zuckerberg from taking 10B of that capex spend and offering every name on that DeepSeek paper a 10 million salary?
samvher 2y ago
I don't think the part about the market reaction is too surprising, for COVID it took a while for the market to react as well. Seeing that DeepSeek-R1 has benchmark scores like o1 is not the same as seeing that people actually like it, that it's being adopted, and that stated training costs are getting accepted as credible. (But I agree that there is a lot of irrationality in these markets.)
xnorswap 2y ago
> There was no market reaction when Deepseek r1 was released. > There was no market reaction when Deepseek v3 was released. > There was a massive market reaction when a Deepseek app was released. This signals that there was money on the table by paying attention to the underlying tech.
TechDebtDevin 2y ago
Everything I know about ML is self taught. My local Uni has a ML Engineering degree that's relatively new. Every once in a while I look at the curriculum, outside of some higher lvl math, 90% of that four year degree consists of topics I "mastered" in the first 6-12 months of self study trying to build prediction models for various sports.
- alephnerd 2y ago
  Good for you. Do you know how to optimize the MTU of an Infiniband interconnect? Do you understand how to schedule multiple models being trained concurrently? Do you know why you cannot directly leverage bare metal compute within a Docker image? This is important Infra knowledge you need to take full advantage of any model you are training on your own hardware. And this is why Deepseek was successful - they understood the ins-and-outs of systems programming and the H800 architecture to maximize the compute performance they needed to train their model.
  - beardedwizard 2y ago
    What about Linux namespaces makes you think it's not running on bare metal? Maybe you are thinking of versions of docker that introduce a Linux hyper visor like macos?
    - alephnerd 2y ago
      > What about Linux namespaces makes you think it's not running on bare metal Nothing! That's my point! I've met a number of "MLEs" who couldn't push back like that with my very basic "fizzbuzz" question
- blackkat 2y ago
  Would you mind giving some pointers of where to start? There is a wasteland of blogspam around the topic of getting started with ML and it’s hard to know what is useful at the beginning.
  - 2y ago

high_na_euv 2y ago

Or people in charge cannot properly evaluate things :) It is very possible, when things require advanced technical knowledge and experience ><"Old Man yelling at clouds" rant over> They are too busy deploying ruby apps and juggling jsons over https to mess with ugly cpp, segfaults, kernel details, hardware intrinsics and semiconductors

alephnerd 2y ago
At the scale that the selloffs happened, that meant there were some shifts by institutional investors. These are the kinds of managers that would have technical people on retainer to consult (eg. Guidepoint). > They are too busy deploying ruby apps and juggling jsons over https Ruby is "legacy" as well nowadays. Most younger devs I meet tend to really only understand JS and Python, and that too while heavily relying on outside packages or dependencies. Not a bad thing per say, but if a VC like me has deeper knowledge about OS internals or Mellanox tuning than some of the (American) MLEs in companies they've done due diligence on, something's very wrong with the talent pipeline.
highcountess 2y ago
Humans are generally not very good at properly evaluating things in general. They are exceptionally good at believing themselves to be very good at properly evaluating things though. It is why empires collapse from miscalculations about war by their experts, it is the foundational mental model underlying communism that simply cannot work but keeps being attempted, and it is how countries can be destroyed in every which way while the experts maintain that everything is just fine. Never underestimate the powers of the ego, the destroyer of worlds. I mean they don’t know what they are doing, but we have advanced technical knowledge and experience, so we should clearly be in charge of all things, including those beyond our narrow scope of advanced technical knowledge and experience.
- dagss 2y ago
  To be fair, capitalism doesn't really work either, you get a very unstable systems with cycles (which people try to stabilize in ad hoc ways), you optimize for the wrong things long term (right now producing much more goods than we need, without factoring in devastating costs that climate change will bring in a few decades, and destruction of nature and eradication of species) and when things crash many people are hurt. And it practically needs wars and destruction to reset wealth distribution every 50 to 100 years. No communism doesn't work either. I wish there was more alternatives. And some of the usual examples of "socialism doesn't work" might have worked if US hadn't interfered. E.g. in Chile, no system would have survived US actively supporting Pinochet. We will never know how socialism would have played out there if US (and Soviet too) left it alone.

mhog_hn 2y ago

What resources are suggested to get started in this area? Assuming a distributed systems web dev background

alephnerd 2y ago
UIUC CS241 - https://pkgamma.github.io/cs241-site-clone-fa19/ https://pkgamma.github.io/cs241-site-clone-fa19/ UCB's CS162 - https://cs162.org/ https://cs162.org/ MIT 6.1810 - https://kaashoek.github.io/65810-2023/ https://kaashoek.github.io/65810-2023/ These were tablestake courses that teach you the basics of OS and Systems Development (eg. Synchronization, Scheduling, etc). After you understand that, then you'd start digging into HPC classes AND then ideally go into ML. At least, this was the workflow and mental model I followed a decade+ ago.
- mhog_hn 2y ago
  Thanks!

InkCanon 2y ago

Jevons paradox has been quoted to death here - it really needs pointing out that it points to a commodity in general, not Nvidia. Nvidia is not selling just coal, it's selling a super purified ultra powerful coal only it can make. Jevons paradox just says coal (compute) usage will increase. Not that this specific type of coal will. And this is quite separate from the business of it. For example Jevons paradox is quite apparent in the airline industry, but airlines are notorious for possibly never having net made a profit because of high capex and aggressive pricing competition. Jevons paradox is not any reason at all for a runaway valuation of a company.

alephnerd 2y ago
I agree, but the existing moat that Nvidia has due to CUDA and the entire HPC ecosystem still holds. And this is why it's relevant to treat Nvidia as the majority of the GPU space segment. Furthermore, it's Nvidia that owns Mellanox - which is the owner of the Infiniband IP which is used in just about every DC or cluster, because no other vendor came as close for interconnect performance
- InkCanon 2y ago
  You're on the dot here. Most people are looking at it wrong. Someone here once put it succinctly - Nvidia isn't selling GPUs, it's selling Infiniband clusters. The whole NVLink/CUDA ecosystem is the key, and a mass of separate players (Cerebras, Google TPUs for inference, Mojo for CUDA, AMD/Intel/Huawei etc for GPUs, etc) are very unlikely to break that. In fact NVLink is so powerful, based on the DeepSeek paper it appears to be nerfed (classic Nvidia market segmentation strategy), and a big part of squeezing their compute was hacking around that. That said, the reason I don't invest Nvidia is that they're selling one more thing - AI legitimacy. Every tech shop is buying Nvidia because all their CEOs need to go to the board and say, we've got so and so AI strategy. Sure as hell Meta isn't spending 50 billion a year because AI makes them so much damn money (supposedly ML saved their ass when Apple put on App Tracking Transparency, but 99% suds it's not the ML that needs a gigawatt GPU cluster to run). And the very real calculus for investors/CEOs is that if DeepSeek made this on a thousandth of the budget (conflicting reports but high probably DS has a fraction of the compute), whYs stopping Zuckerberg from taking 10B of that capex spend and offering every name on that DeepSeek paper a 10 million salary?

samvher 2y ago

I don't think the part about the market reaction is too surprising, for COVID it took a while for the market to react as well. Seeing that DeepSeek-R1 has benchmark scores like o1 is not the same as seeing that people actually like it, that it's being adopted, and that stated training costs are getting accepted as credible. (But I agree that there is a lot of irrationality in these markets.)

xnorswap 2y ago

> There was no market reaction when Deepseek r1 was released. > There was no market reaction when Deepseek v3 was released. > There was a massive market reaction when a Deepseek app was released. This signals that there was money on the table by paying attention to the underlying tech.

TechDebtDevin 2y ago

Everything I know about ML is self taught. My local Uni has a ML Engineering degree that's relatively new. Every once in a while I look at the curriculum, outside of some higher lvl math, 90% of that four year degree consists of topics I "mastered" in the first 6-12 months of self study trying to build prediction models for various sports.

alephnerd 2y ago
Good for you. Do you know how to optimize the MTU of an Infiniband interconnect? Do you understand how to schedule multiple models being trained concurrently? Do you know why you cannot directly leverage bare metal compute within a Docker image? This is important Infra knowledge you need to take full advantage of any model you are training on your own hardware. And this is why Deepseek was successful - they understood the ins-and-outs of systems programming and the H800 architecture to maximize the compute performance they needed to train their model.
- beardedwizard 2y ago
  What about Linux namespaces makes you think it's not running on bare metal? Maybe you are thinking of versions of docker that introduce a Linux hyper visor like macos?
  - alephnerd 2y ago
    > What about Linux namespaces makes you think it's not running on bare metal Nothing! That's my point! I've met a number of "MLEs" who couldn't push back like that with my very basic "fizzbuzz" question
blackkat 2y ago
Would you mind giving some pointers of where to start? There is a wasteland of blogspam around the topic of getting started with ML and it’s hard to know what is useful at the beginning.
- 2y ago