11 ms·

The description of DeepSeek reminds me of my experience in networking in the late 80s - early 90s. Back then a really big motivator for Asynchronous Transfer M

The description of DeepSeek reminds me of my experience in networking in the late 80s - early 90s. Back then a really big motivator for Asynchronous Transfer Mode (ATM) and fiber-to-the-home was the promise of video on demand, which was a huge market in comparison to the Internet of the day. Just about all the work in this area ignored the potential of advanced video coding algorithms, and assumed that broadcast TV-quality video would require about 50x more bandwidth than today's SD Netflix videos, and 6x more than 4K. What made video on the Internet possible wasn't a faster Internet, although the 10-20x increase every decade certainly helped - it was smarter algorithms that used orders of magnitude less bandwidth. In the case of AI, GPUs keep getting faster, but it's going to take a hell of a long time to achieve a 10x improvement in performance per cm^2 of silicon. Vastly improved training/inference algorithms may or may not be possible (DeepSeek seems to indicate the answer is "may") but there's no physical limit preventing them from being discovered, and the disruption when someone invents a new algorithm can be nearly immediate.

TMWNN 2y ago
>but there's no physical limit preventing them from being discovered, and the disruption when someone invents a new algorithm can be nearly immediate. The rise of the net is Jevons paradox fulfilled. The orders of magnitude less bandwidth needed per cat video drove much more than that in overall growth in demand for said videos. During the dotcom bubble's collapse, bandwidth use kept going up. Even if there is a near-term bear case for NVDA (dotcom bubble/bust), history indicates a bull case for the sector overall and related investments such as utilities (the entire history of the tech sector from 1995 to today).
accra4rx 2y ago
Love those analogies . This is one of main reason I love hacker news / reddit . Honest golden experiences
AlanYx 2y ago
Another aspect that reinforces your point is that the ATM push (and subsequent downfall) was not just bandwidth-motivated but also motivated by a belief that ATM's QoS guarantees were necessary. But it turned out that software improvements, notably MPLS to handle QoS, were all that was needed.
- pjdesno 2y ago
  Nah, it's mostly just buffering :-) Plus the cell phone industry paved the way for VOIP by getting everyone used to really, really crappy voice quality. Generations of Bell Labs and Bellcore engineers would rather have resigned than be subjected to what's considered acceptable voice quality nowadays...
  - hedgehog 2y ago
    Yes, I think most video on the Internet is HLS and similar approaches which are about as far from the ATM circuit-switching approach as it gets. For those unfamiliar HLS is pretty much breaking the video into chunks to download over plain HTTP.
    - matt-attack 2y ago
      Yes, but that's entirely orthogonal to the "coding" algorithms being used and which are specifically responsible for the improvement that GP was describing. HLS is really just a way to empower the client with the ownership of the playback logic. Let the client handle forward buffering, retries, stream selection, etc.
  - nyarlathotep_ 2y ago
    >> Plus the cell phone industry paved the way for VOIP by getting everyone used to really, really crappy voice quality What accounts for this difference? Is there something inherently worse about the nature of cell phone infrastructure over land-line use? I'm totally naive on such subjects. I'm just old enough to remember landlines being widespread, but nearly all of my phone calls have been via cell since the mid 00s, so I can't judge quality differences given the time that's passed.
    - hnuser123456 2y ago
      Because at some point, someone decided that 8 kbps makes for an acceptable audio stream per subscriber. And at first, the novelty of being able to call anyone anywhere, even with this awful quality, was novel enough that people would accept it. And most people did until the carriers decided they could allocate a little more with VoLTE, if it works on your phone in your area.
aurareturn 2y ago
Doesn’t your point about video compression tech support Nvidia’s bull case? Better video compression led to an explosion in video consumption on the Internet, leading to much more revenue for companies like Comcast, Google, T-Mobile, Verizon, etc. More efficient LLMs lead to much more AI usage. Nvidia, TSMC, etc will benefit.
- vFunct 2y ago
  I agree that advancements like DeepSeek, like transformer models before it, is just going to end up increasing demand. It’s very shortsighted to think we’re going to need fewer chips because the algorithms got better. The system became more efficient, which causes induced demand.
  - floatrock 2y ago
    obligatory https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
  - diamond559 2y ago
    More demand for what, chatbots? ai slop? buggy code?
  - eru 2y ago
    It will increase the total volume demanded, but not necessarily the amount of value that companies like NVidia can capture. Most likely, consumer surplus has gone up.
- snailmailstare 2y ago
  It improves TSMC' case.. Paying Nvidia would be like paying Cray for every smartphone that is faster than a supercomputer of old.
- 9rx 2y ago
  Yes, over the long haul, probably. But as far as individual investors go they might not like that Nvidia. Anyone currently invested is presumably in because they like the insanely high profit margin, and this is apt to quash that. There is now much less reason to give your first born to get your hands on their wares. Comcast, Google, T-Mobile, Verizon, etc., and especially those not named Google, have nothingburger margins in comparison. If you are interested in what they can do with volume, then there is still a lot of potential. They may even be more profitable on that end than a margin play could ever hope for. But that interest is probably not from the same person who currently owns the stock, it being a change in territory, and there is apt to be a lot of instability as stock changes hands from the one group to the next.
vFunct 2y ago
I worked on a network that used a protocol very similar to ATM (actually it was the first Iridium satellite network). An internet based on ATM would have been amazing. You’re basically guaranteeing a virtual switched circuit, instead of the packets we have today. The horror of packet switching is all the buffering it needs, since it doesn’t guarantee circuits. Bandwidth is one thing, but the real benefit is that ATM also guaranteed minimal latencies. You could now shave off another 20-100ms of latency for your FaceTime calls, which is subtle but game changing. Just instant-on high def video communications, as if it were on closed circuits to the next room. For the same reasons, the AI analogy could benefit from both huge processing as well as stronger algorithms.
- lxgr 2y ago
  > You’re basically guaranteeing a virtual switched circuit Which means you need state (and the overhead that goes with it) for each connection within the network. That's horribly inefficient, and precisely the reason packet-switching won. > An internet based on ATM would have been amazing. No, we'd most likely be paying by the socket connection (as somebody has to pay for that state keeping overhead), which sounds horrible. > You could now shave off another 20-100ms of latency for your FaceTime calls, which is subtle but game changing. Maybe on congested Wi-Fi (where even circuit switching would struggle) or poorly managed networks (including shitty ISP-supplied routers suffering from horrendous bufferbloat). Definitely not on the majority of networks I've used in the past years. > The horror of packet switching is all the buffering it needs [...] The ideal buffer size is exactly the bandwidth-delay product. That's really not a concern these days anymore. If anything, buffers are much too large, causing unnecessary latency; that's where bufferbloat-aware scheduling comes in.
  - vFunct 2y ago
    The cost for interactive video would be a requirement of 10x bandwidth, basically to cover idle time. Not efficient but not impossible, and definitely wouldn’t change ISP business models. The latency benefit would outweigh the cost. Just absolutely instant video interaction.
paulddraper 2y ago
I love algorithms as much the next guy, but not really. DCT was developed in 1972 and has a compression ratio of 100:1. H.264 compresses 2000:1. And standard resolution (480p) is ~1/30th the resolution of 4k. --- I.e. Standard resolution with DCT is smaller than 4k with H.264. Even high-definition (720p) with DCT is only twice the bandwidth of 4k H.264. Modern compression has allowed us to add a bunch more pixels, but it was hardly a requirement for internet video.
- wtallis 2y ago
  The web didn't go from streaming 480p straight to 4k. There were a couple of intermediate jumps in pixel count that were enabled in large part by better compression. Notably, there was a time period where it was important to ensure your computer had hardware support for H.264 decode, because it was taxing on low-power CPUs to do at 1080p and you weren't going to get streamed 1080p content in any simpler, less efficient codec.
  - paulddraper 2y ago
    Right. Modern compression algorithms were developed but not even computationally available for some of the time.
- WhitneyLand 2y ago
  DCT is not an algorithm at all, it’s a mathematical transform. It doesn’t have a compression ratio.
  - paulddraper 2y ago
    > DCT compression, also known as block compression, compresses data in sets of discrete DCT blocks.[3] DCT blocks sizes including 8x8 pixels for the standard DCT, and varied integer DCT sizes between 4x4 and 32x32 pixels.[1][4] The DCT has a strong energy compaction property,[5][6] capable of achieving high quality at high data compression ratios.[7][8] However, blocky compression artifacts can appear when heavy DCT compression is applied. https://en.wikipedia.org/wiki/Discrete_cosine_transform https://en.wikipedia.org/wiki/Discrete_cosine_transform
eigenvalue 2y ago
Yes, that is a very apt analogy!
tuna74 2y ago
I always like the "look" of high bit rate Mpeg2 video. Download HD japanese TV content from 2005-2010 and it still looks really good.
TheCondor 2y ago
It seems more stark even. The energy costs that are current and then projected for AI are staggering. At the same time, I think it has been MS that has been publishing papers on LLMs that are smaller (so called small language models) but more targeted and still achieving a fairly high "accuracy rate." Didn't TMSC say that SamA came for a visit and said they needed $7T in investment to keep up with the pending demand needs. This stuff is all super cool and fun to play with, I'm not a nay sayer but it almost feels like these current models are "bubble sort" and who knows how it will look if "quicksort" for them becomes invented.
lokar 2y ago
Another example: people like to cite how the people who really made money in the CA gold rush were selling picks and shovels. That only lasted so long. Then it was heavy machinery (hydraulics, excavators, etc)

TMWNN 2y ago

>but there's no physical limit preventing them from being discovered, and the disruption when someone invents a new algorithm can be nearly immediate. The rise of the net is Jevons paradox fulfilled. The orders of magnitude less bandwidth needed per cat video drove much more than that in overall growth in demand for said videos. During the dotcom bubble's collapse, bandwidth use kept going up. Even if there is a near-term bear case for NVDA (dotcom bubble/bust), history indicates a bull case for the sector overall and related investments such as utilities (the entire history of the tech sector from 1995 to today).

accra4rx 2y ago

Love those analogies . This is one of main reason I love hacker news / reddit . Honest golden experiences

AlanYx 2y ago

Another aspect that reinforces your point is that the ATM push (and subsequent downfall) was not just bandwidth-motivated but also motivated by a belief that ATM's QoS guarantees were necessary. But it turned out that software improvements, notably MPLS to handle QoS, were all that was needed.

pjdesno 2y ago
Nah, it's mostly just buffering :-) Plus the cell phone industry paved the way for VOIP by getting everyone used to really, really crappy voice quality. Generations of Bell Labs and Bellcore engineers would rather have resigned than be subjected to what's considered acceptable voice quality nowadays...
- hedgehog 2y ago
  Yes, I think most video on the Internet is HLS and similar approaches which are about as far from the ATM circuit-switching approach as it gets. For those unfamiliar HLS is pretty much breaking the video into chunks to download over plain HTTP.
  - matt-attack 2y ago
    Yes, but that's entirely orthogonal to the "coding" algorithms being used and which are specifically responsible for the improvement that GP was describing. HLS is really just a way to empower the client with the ownership of the playback logic. Let the client handle forward buffering, retries, stream selection, etc.
- nyarlathotep_ 2y ago
  >> Plus the cell phone industry paved the way for VOIP by getting everyone used to really, really crappy voice quality What accounts for this difference? Is there something inherently worse about the nature of cell phone infrastructure over land-line use? I'm totally naive on such subjects. I'm just old enough to remember landlines being widespread, but nearly all of my phone calls have been via cell since the mid 00s, so I can't judge quality differences given the time that's passed.
  - hnuser123456 2y ago
    Because at some point, someone decided that 8 kbps makes for an acceptable audio stream per subscriber. And at first, the novelty of being able to call anyone anywhere, even with this awful quality, was novel enough that people would accept it. And most people did until the carriers decided they could allocate a little more with VoLTE, if it works on your phone in your area.

aurareturn 2y ago

Doesn’t your point about video compression tech support Nvidia’s bull case? Better video compression led to an explosion in video consumption on the Internet, leading to much more revenue for companies like Comcast, Google, T-Mobile, Verizon, etc. More efficient LLMs lead to much more AI usage. Nvidia, TSMC, etc will benefit.

vFunct 2y ago
I agree that advancements like DeepSeek, like transformer models before it, is just going to end up increasing demand. It’s very shortsighted to think we’re going to need fewer chips because the algorithms got better. The system became more efficient, which causes induced demand.
- floatrock 2y ago
  obligatory https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- diamond559 2y ago
  More demand for what, chatbots? ai slop? buggy code?
- eru 2y ago
  It will increase the total volume demanded, but not necessarily the amount of value that companies like NVidia can capture. Most likely, consumer surplus has gone up.
snailmailstare 2y ago
It improves TSMC' case.. Paying Nvidia would be like paying Cray for every smartphone that is faster than a supercomputer of old.
9rx 2y ago
Yes, over the long haul, probably. But as far as individual investors go they might not like that Nvidia. Anyone currently invested is presumably in because they like the insanely high profit margin, and this is apt to quash that. There is now much less reason to give your first born to get your hands on their wares. Comcast, Google, T-Mobile, Verizon, etc., and especially those not named Google, have nothingburger margins in comparison. If you are interested in what they can do with volume, then there is still a lot of potential. They may even be more profitable on that end than a margin play could ever hope for. But that interest is probably not from the same person who currently owns the stock, it being a change in territory, and there is apt to be a lot of instability as stock changes hands from the one group to the next.

vFunct 2y ago

I worked on a network that used a protocol very similar to ATM (actually it was the first Iridium satellite network). An internet based on ATM would have been amazing. You’re basically guaranteeing a virtual switched circuit, instead of the packets we have today. The horror of packet switching is all the buffering it needs, since it doesn’t guarantee circuits. Bandwidth is one thing, but the real benefit is that ATM also guaranteed minimal latencies. You could now shave off another 20-100ms of latency for your FaceTime calls, which is subtle but game changing. Just instant-on high def video communications, as if it were on closed circuits to the next room. For the same reasons, the AI analogy could benefit from both huge processing as well as stronger algorithms.

lxgr 2y ago
> You’re basically guaranteeing a virtual switched circuit Which means you need state (and the overhead that goes with it) for each connection within the network. That's horribly inefficient, and precisely the reason packet-switching won. > An internet based on ATM would have been amazing. No, we'd most likely be paying by the socket connection (as somebody has to pay for that state keeping overhead), which sounds horrible. > You could now shave off another 20-100ms of latency for your FaceTime calls, which is subtle but game changing. Maybe on congested Wi-Fi (where even circuit switching would struggle) or poorly managed networks (including shitty ISP-supplied routers suffering from horrendous bufferbloat). Definitely not on the majority of networks I've used in the past years. > The horror of packet switching is all the buffering it needs [...] The ideal buffer size is exactly the bandwidth-delay product. That's really not a concern these days anymore. If anything, buffers are much too large, causing unnecessary latency; that's where bufferbloat-aware scheduling comes in.
- vFunct 2y ago
  The cost for interactive video would be a requirement of 10x bandwidth, basically to cover idle time. Not efficient but not impossible, and definitely wouldn’t change ISP business models. The latency benefit would outweigh the cost. Just absolutely instant video interaction.

paulddraper 2y ago

I love algorithms as much the next guy, but not really. DCT was developed in 1972 and has a compression ratio of 100:1. H.264 compresses 2000:1. And standard resolution (480p) is ~1/30th the resolution of 4k. --- I.e. Standard resolution with DCT is smaller than 4k with H.264. Even high-definition (720p) with DCT is only twice the bandwidth of 4k H.264. Modern compression has allowed us to add a bunch more pixels, but it was hardly a requirement for internet video.

wtallis 2y ago
The web didn't go from streaming 480p straight to 4k. There were a couple of intermediate jumps in pixel count that were enabled in large part by better compression. Notably, there was a time period where it was important to ensure your computer had hardware support for H.264 decode, because it was taxing on low-power CPUs to do at 1080p and you weren't going to get streamed 1080p content in any simpler, less efficient codec.
- paulddraper 2y ago
  Right. Modern compression algorithms were developed but not even computationally available for some of the time.
WhitneyLand 2y ago
DCT is not an algorithm at all, it’s a mathematical transform. It doesn’t have a compression ratio.
- paulddraper 2y ago
  > DCT compression, also known as block compression, compresses data in sets of discrete DCT blocks.[3] DCT blocks sizes including 8x8 pixels for the standard DCT, and varied integer DCT sizes between 4x4 and 32x32 pixels.[1][4] The DCT has a strong energy compaction property,[5][6] capable of achieving high quality at high data compression ratios.[7][8] However, blocky compression artifacts can appear when heavy DCT compression is applied. https://en.wikipedia.org/wiki/Discrete_cosine_transform https://en.wikipedia.org/wiki/Discrete_cosine_transform

eigenvalue 2y ago

Yes, that is a very apt analogy!

tuna74 2y ago

I always like the "look" of high bit rate Mpeg2 video. Download HD japanese TV content from 2005-2010 and it still looks really good.

TheCondor 2y ago

It seems more stark even. The energy costs that are current and then projected for AI are staggering. At the same time, I think it has been MS that has been publishing papers on LLMs that are smaller (so called small language models) but more targeted and still achieving a fairly high "accuracy rate." Didn't TMSC say that SamA came for a visit and said they needed $7T in investment to keep up with the pending demand needs. This stuff is all super cool and fun to play with, I'm not a nay sayer but it almost feels like these current models are "bubble sort" and who knows how it will look if "quicksort" for them becomes invented.

lokar 2y ago

Another example: people like to cite how the people who really made money in the CA gold rush were selling picks and shovels. That only lasted so long. Then it was heavy machinery (hydraulics, excavators, etc)