5 ms·

I still cannot comprehend the significance of Moore's law. Although the hardware keeps getting more sophisticated, I cannot perceive that today's computer is 10

I still cannot comprehend the significance of Moore's law. Although the hardware keeps getting more sophisticated, I cannot perceive that today's computer is 100x faster or more than 20-30 years ago. Perhaps the software has been getting bloated to counteract or the speed increase is not real. For AI chip, will we fall into the same situation? will AI algorithm still focus on efficiency?

bobthepanda 6y ago
For games, which many are pretty run entirely locally, things are noticeably faster and higher quality. For networked applications, I feel like the much bigger limit is the network. The Tidal streaming service ran into this with its business model; streaming lossless, high quality audio is a bit of a moot point if the network doesn't allow you to download all of it without noticeable buffering when listening to music.
throw_m239339 6y ago
While processors have become more powerful, we're now coding our apps in Javascript, which negates any progress described by Moore's Law. Of course I'm being hyperbolic but you get my point.
- richajak 6y ago
  Thanks, finally someone agree with my observation. :-) I got deduction point for saying that statement. For me, it is not right to use 100x speed increase for eye candy feature or using bloated Javascript libraries.
afterburner 6y ago
> I cannot perceive that today's computer is 100x faster or more than 20-30 years ago Seems pretty obvious when comparing, say, Duke Nukem 3D with Far Cry 5. Or Wolfenstein 3D with Assassin's Creed Odyssey.
- ncmncm 6y ago
  In certain limited domains, increased hardware capability shows up in proportionally more useful or meaningful results. In most uses the difference is squandered on eye candy or abstraction overhead. We see it in video game rendering and weather prediction, but not in web page presentation, where it is squandered on pointless animation.
  - afterburner 6y ago
    It's not web pages that have driven the demand for more and more processing power. This is a very web-programming-centric way of looking at things.
    - ncmncm 6y ago
      I don't understand this remark. It seemed to me weather-prediction-centric.
tlb 6y ago
The usefulness of a computer isn't linear with speed. Usefulness is application-specific, of course, but in most applications usefulness is somewhere between log(speed) and sqrt(speed). For example: If you're solving NP problems (common in optimization), the size of the problem you can handle in a given amount of time is proportional to log(speed). If you're adding features to software, then any given feature requires some amount of compute power. Experience shows these to be power-law distributed. Early software versions started with the cheapest features, so now we're adding things on the right side of the distribution. So each new feature is likely to require additional compute power increasing like a power law. For a concrete example: when OSX added transparency and shadows to windows, it required every pixel in a window to be operated on twice. A doubling of a major component of the screen update time for a feature which only slightly increases user happiness/productivity. It had to wait until GPUs were fast enough to support it. In machine learning, the accuracy of results increases much less than linearly, somewhere between log(# parameters) and sqrt(# parameters). And the inference time is linear in the number of parameters. So it's a good thing that compute power is increasing exponentially or else progress would be really slow.
jayd16 6y ago
Even trivially a 4k@120hz monitor is 108x the pixels as a 630x480@30hz monitor and you can clearly render much more complex scenes per pixel. It seems trivial to me to see 100x the speed without relying on benchmarks. That said, speed as a derivative of how long the user waits has not changed simply because humans have not changed what they consider acceptable wait times.
mceachen 6y ago
For a concrete example of what 100x feels like, importing digital camera imagery into a gallery in 1999-2000 was on the order of 10-15 seconds per image (for a ~2 MP image, which was current state-of-the-art for resolution). Multi-core and hyperthreading was still years away. PhotoStructure, a self-hosted photo and video manager, uses libvips (a wonderfully performant image library) and is multi-processed and multi-threaded. Typical images are >10x the resolution (24-40MP images are standard from DSLR) yet several are processed in parallel every second.

bobthepanda 6y ago

For games, which many are pretty run entirely locally, things are noticeably faster and higher quality. For networked applications, I feel like the much bigger limit is the network. The Tidal streaming service ran into this with its business model; streaming lossless, high quality audio is a bit of a moot point if the network doesn't allow you to download all of it without noticeable buffering when listening to music.

throw_m239339 6y ago

While processors have become more powerful, we're now coding our apps in Javascript, which negates any progress described by Moore's Law. Of course I'm being hyperbolic but you get my point.

richajak 6y ago
Thanks, finally someone agree with my observation. :-) I got deduction point for saying that statement. For me, it is not right to use 100x speed increase for eye candy feature or using bloated Javascript libraries.

afterburner 6y ago

> I cannot perceive that today's computer is 100x faster or more than 20-30 years ago Seems pretty obvious when comparing, say, Duke Nukem 3D with Far Cry 5. Or Wolfenstein 3D with Assassin's Creed Odyssey.

ncmncm 6y ago
In certain limited domains, increased hardware capability shows up in proportionally more useful or meaningful results. In most uses the difference is squandered on eye candy or abstraction overhead. We see it in video game rendering and weather prediction, but not in web page presentation, where it is squandered on pointless animation.
- afterburner 6y ago
  It's not web pages that have driven the demand for more and more processing power. This is a very web-programming-centric way of looking at things.
  - ncmncm 6y ago
    I don't understand this remark. It seemed to me weather-prediction-centric.

tlb 6y ago

The usefulness of a computer isn't linear with speed. Usefulness is application-specific, of course, but in most applications usefulness is somewhere between log(speed) and sqrt(speed). For example: If you're solving NP problems (common in optimization), the size of the problem you can handle in a given amount of time is proportional to log(speed). If you're adding features to software, then any given feature requires some amount of compute power. Experience shows these to be power-law distributed. Early software versions started with the cheapest features, so now we're adding things on the right side of the distribution. So each new feature is likely to require additional compute power increasing like a power law. For a concrete example: when OSX added transparency and shadows to windows, it required every pixel in a window to be operated on twice. A doubling of a major component of the screen update time for a feature which only slightly increases user happiness/productivity. It had to wait until GPUs were fast enough to support it. In machine learning, the accuracy of results increases much less than linearly, somewhere between log(# parameters) and sqrt(# parameters). And the inference time is linear in the number of parameters. So it's a good thing that compute power is increasing exponentially or else progress would be really slow.

jayd16 6y ago

Even trivially a 4k@120hz monitor is 108x the pixels as a 630x480@30hz monitor and you can clearly render much more complex scenes per pixel. It seems trivial to me to see 100x the speed without relying on benchmarks. That said, speed as a derivative of how long the user waits has not changed simply because humans have not changed what they consider acceptable wait times.

mceachen 6y ago

For a concrete example of what 100x feels like, importing digital camera imagery into a gallery in 1999-2000 was on the order of 10-15 seconds per image (for a ~2 MP image, which was current state-of-the-art for resolution). Multi-core and hyperthreading was still years away. PhotoStructure, a self-hosted photo and video manager, uses libvips (a wonderfully performant image library) and is multi-processed and multi-threaded. Typical images are >10x the resolution (24-40MP images are standard from DSLR) yet several are processed in parallel every second.