5 ms·

Great list! To quote Bob Colwell (former Intel fellow and DARPA director), it's hard to replace an exponential. There will be non-uniform improvements and "one

by akuma73 10y ago

Great list! To quote Bob Colwell (former Intel fellow and DARPA director), it's hard to replace an exponential. There will be non-uniform improvements and "one-off" type of breakthroughs but a sustained 2x every 2x years is just not going to happen ever. As a thought experiment, what if Moore's Law scaling had ended in 2000? No smartphones. I agree that the value of software engineering and computer architecture in general will increase as the transistor-scaling free ride ends. We have to make more efficient use of existing transistors that will no longer scale going forward.

api 10y ago
Quibble: I disagree about 'no smartphones.' Look at what is possible on a 64K/8-bit machine in the 1980s: https://en.wikipedia.org/wiki/GEOS_(8-bit_operating_system) https://en.wikipedia.org/wiki/GEOS_(8-bit_operating_system) An iPhone could be done on a single core 100mhz system if code is efficient. That would mean forfeiting some amount of eye candy and developing more efficient GUI layers. I do agree that some smart phone functionality like super-high-end 3d maps apps might not exist, but a basic map with basic turn-by-turn directions could be built for the Commodore 64 by programmers who actually think about performance. This is why I think a "Software Moore's Law" and performance is going to matter a lot. Premature optimization might no longer be the root of all evil. It might be a skill programmers need to have.
- berntb 10y ago
  An interesting argument. Well, a phone with Emacs as the OS/gui? Given a good chording keyboard, I could have lived happily ever after with that. (But premature optimization is still evil. First you make it work, then you make it fast. Preferably using better algorithms. There is a lot of experience with this.)
- akuma73 10y ago
  I agree here with the word "possible". However, progress on this front would be slow due to the increased cost of software development. Software would be much more difficult to write leading to longer lead times for applications. I don't think the app stores would scale at the same rate that it did for iPhone or Android. One analogy is parallel programming. This is generally very difficult, so people avoid it, even when hardware vendors give you get a bounty of cores. Another example is GPGPU - very slow progress on mapping data-parallel apps to the GPU (image processing etc.). We may not know what future devices and applications are not possible due to Moore's Law's end. Another example is the recent explosion in deep learning - this was only possible due to the increase in FLOP density as a direct consequence of Moore's Law. These algorithms were around in the 80s.
  - api 10y ago
    "Software would be much more difficult to write leading to longer lead times for applications." This is a problem we will need to solve. So far we've increased programmer productivity by sacrificing code efficiency because Moore's Law gives us more speed for free. We'll have to find ways to reduce this trade-off in the future by thinking harder and deeper about software since the hardware's not going to save us anymore.
vardump 10y ago
> As a thought experiment, what if Moore's Law scaling had ended in 2000? No smartphones. 2000? No, we would still have had smartphones. We were not in stone age anymore. Mass market smartphones would have eventually been built with that era process technology. By year 2000, we had full color displays and StrongARM CPUs 206 MHz+, similar process could stretch to 400-500 MHz. We'd have our beloved smartphones. Just with lower resolution screens (at most 640x480), slower CPUs (200-500 MHz single core, 8 kB L1 cache), 64-128 MB RAM and less storage (up to 1 GB). Packet data connections would have likely been limited way below 1 Mbit/s. (A)GPS would exist, but be much less sensitive (needs a lot of processing power) and take a lot longer to lock. Web browsers and other apps would have eventually been developed that work smoothly with such limited HW. I think I could have lived with such a moore's-law-stopped-year-2000-device. See what roughly 2000 StrongARM can do in a mobile device (PocketPC Tomb Raider, just like the original Playstation version): https://www.youtube.com/watch?v=TujJ4nf31rI https://www.youtube.com/watch?v=TujJ4nf31rI Or these demos (run nicely on 2000 PocketPC HW): https://www.youtube.com/watch?v=bjZupXHhflA https://www.youtube.com/watch?v=bjZupXHhflA https://www.youtube.com/watch?v=E2CDOtHswR0 https://www.youtube.com/watch?v=E2CDOtHswR0
- gpderetta 10y ago
  "Web browsers and other apps would have eventually been developed that work smoothly with such limited HW" and they would probably feel as fast as today's phones for many applications.
  - vardump 10y ago
    Absolutely, when it comes to CPU side of things. Of course, network latency would likely be higher than 13-20 ms we currently enjoy on 4G+ networks. Probably something like 50 ms -- main reason for high latency, circuit switched network, would have still been eventually replaced with purely packet switched system.
ghaff 10y ago
Colwell's presentation from Hot Chips two or three years ago really nails it. There are a lot of potential paths to increased performance/efficiency but you don't just replace the sort of gains that came from CMOS process scaling.
T-A 10y ago
I drooled over this in 2000: https://en.wikipedia.org/wiki/Nokia_9210_Communicator https://en.wikipedia.org/wiki/Nokia_9210_Communicator

by akuma73 10y ago

api 10y ago

Quibble: I disagree about 'no smartphones.' Look at what is possible on a 64K/8-bit machine in the 1980s: https://en.wikipedia.org/wiki/GEOS_(8-bit_operating_system) https://en.wikipedia.org/wiki/GEOS_(8-bit_operating_system) An iPhone could be done on a single core 100mhz system if code is efficient. That would mean forfeiting some amount of eye candy and developing more efficient GUI layers. I do agree that some smart phone functionality like super-high-end 3d maps apps might not exist, but a basic map with basic turn-by-turn directions could be built for the Commodore 64 by programmers who actually think about performance. This is why I think a "Software Moore's Law" and performance is going to matter a lot. Premature optimization might no longer be the root of all evil. It might be a skill programmers need to have.

berntb 10y ago
An interesting argument. Well, a phone with Emacs as the OS/gui? Given a good chording keyboard, I could have lived happily ever after with that. (But premature optimization is still evil. First you make it work, then you make it fast. Preferably using better algorithms. There is a lot of experience with this.)
akuma73 10y ago
I agree here with the word "possible". However, progress on this front would be slow due to the increased cost of software development. Software would be much more difficult to write leading to longer lead times for applications. I don't think the app stores would scale at the same rate that it did for iPhone or Android. One analogy is parallel programming. This is generally very difficult, so people avoid it, even when hardware vendors give you get a bounty of cores. Another example is GPGPU - very slow progress on mapping data-parallel apps to the GPU (image processing etc.). We may not know what future devices and applications are not possible due to Moore's Law's end. Another example is the recent explosion in deep learning - this was only possible due to the increase in FLOP density as a direct consequence of Moore's Law. These algorithms were around in the 80s.
- api 10y ago
  "Software would be much more difficult to write leading to longer lead times for applications." This is a problem we will need to solve. So far we've increased programmer productivity by sacrificing code efficiency because Moore's Law gives us more speed for free. We'll have to find ways to reduce this trade-off in the future by thinking harder and deeper about software since the hardware's not going to save us anymore.

vardump 10y ago

> As a thought experiment, what if Moore's Law scaling had ended in 2000? No smartphones. 2000? No, we would still have had smartphones. We were not in stone age anymore. Mass market smartphones would have eventually been built with that era process technology. By year 2000, we had full color displays and StrongARM CPUs 206 MHz+, similar process could stretch to 400-500 MHz. We'd have our beloved smartphones. Just with lower resolution screens (at most 640x480), slower CPUs (200-500 MHz single core, 8 kB L1 cache), 64-128 MB RAM and less storage (up to 1 GB). Packet data connections would have likely been limited way below 1 Mbit/s. (A)GPS would exist, but be much less sensitive (needs a lot of processing power) and take a lot longer to lock. Web browsers and other apps would have eventually been developed that work smoothly with such limited HW. I think I could have lived with such a moore's-law-stopped-year-2000-device. See what roughly 2000 StrongARM can do in a mobile device (PocketPC Tomb Raider, just like the original Playstation version): https://www.youtube.com/watch?v=TujJ4nf31rI https://www.youtube.com/watch?v=TujJ4nf31rI Or these demos (run nicely on 2000 PocketPC HW): https://www.youtube.com/watch?v=bjZupXHhflA https://www.youtube.com/watch?v=bjZupXHhflA https://www.youtube.com/watch?v=E2CDOtHswR0 https://www.youtube.com/watch?v=E2CDOtHswR0

gpderetta 10y ago
"Web browsers and other apps would have eventually been developed that work smoothly with such limited HW" and they would probably feel as fast as today's phones for many applications.
- vardump 10y ago
  Absolutely, when it comes to CPU side of things. Of course, network latency would likely be higher than 13-20 ms we currently enjoy on 4G+ networks. Probably something like 50 ms -- main reason for high latency, circuit switched network, would have still been eventually replaced with purely packet switched system.

ghaff 10y ago

Colwell's presentation from Hot Chips two or three years ago really nails it. There are a lot of potential paths to increased performance/efficiency but you don't just replace the sort of gains that came from CMOS process scaling.

T-A 10y ago

I drooled over this in 2000: https://en.wikipedia.org/wiki/Nokia_9210_Communicator https://en.wikipedia.org/wiki/Nokia_9210_Communicator