5 ms·

The fifth epoch of distributed computing

58 points by simonpure 3y ago

superkuh 3y ago
It's not distributed if it's all owned by the same corporation. If anything distributed computing has gone backwards the last decade. This switch to centralization was mostly forced by people using low functionality dumb terminals (smart phones) that users don't control and that literally can't hold a TCP connection open.
- cmrx64 3y ago
  don’t conflate decentralized with distributed
- toast0 3y ago
  > This switch to centralization was mostly forced by people using low functionality dumb terminals (smart phones) that users don't control and that literally can't hold a TCP connection open. Not being able to hold a TCP connection open is an implementation choice of the OS developer (and some choices at the network level too). At WhatsApp, we would find Nokia Symbian devices with 45 day old TCP connections to our servers anytime we looked. Nokia S40 isn't a smart phone platform, but it could hold long connections too. Old versions of Android could do it, although old versions of Android had trouble switching between wifi and cellular. You can probably do it on current Android if you turn off all the Doze or whatever stuff (which isn't always easy to turn off). There's an efficiency argument for having a single long TCP connection for platform push, rather than one per app that needs it, of course. But in that case, platform push better be reliable.
  - superkuh 3y ago
    I didn't mean it was infeasible. I meant holding open TCP connections are avoided by choice to save battery power because transmitting wirelessly is a power drain. But also for all wireless there's a random round trip time and so exponential backoff. The longer you hold open the more likely. Almost all smart phones and their software are gimped in this way; the outliers you mention are interesting but not significant.
swozey 3y ago
I think that the engineers who began their careers in the first epoch are going to turn out to be some of the most productive and technically skilled having gone through each epoch which, frankly, grew easier to deal with every time. They call that point out, basically, but I had no idea it'd be a 100x improvement. > While we cannot predict the breakthroughs that will be delivered in this fifth epoch of computing, we do know that each previous epoch has been characterized by a factor of 100x improvement in scale, efficiency, and cost-performance, all while improving security and reliability. I'm worried about 5th gen epoch engineers. On one side there is a massive amount of smart rust programmers who are super young in all the rust communities just as an example and on the other a lot of people who don't know anything beyond a gui operating system and have never owned a computer and do everything by phone. I've been happily teaching linux quite a bit lately.
- zitterbewegung 3y ago
  I can imagine fifth epoch engineers only doing code review and prompt engineering or handling major incidence once the LLMs handle the code writing.
  - ActionHank 3y ago
    My son is the only one in his class that has a PC. There is going to be an astounding gap between those in the know and those who are not.
    - dagw 3y ago
      I've noticed the same. I'm convinced that the generation that is under 16 today will on average know a lot less about computers compared to 20 years ago.
      - ActionHank 3y ago
        Not my son, he's counting binary and hex, built his PC with me, ran network cables with me, is coding his own games for himself and his friends. Assuming some form of ubiquity for AI tooling people like him are going to build amazing things.
anonymousDan 3y ago
A lot of words to say not much frankly.
thundergolfer 3y ago
> Epoch five: from information to insights Eugh, this article is full of MBA-speak, not something I'd expect from a highly technical Google Fellow.
- JSR_FDED 3y ago
  Geez I only just wrapped my head around the “fourth Industrial Revolution” we are apparently in… https://www.salesforce.com/eu/blog/industry-4-0-where-are-we-going/#:~:text=What%20is%20the%20Fourth%20Industrial%20Revolution%3F,Things%20and%20cyber%2Dphysical%20systems https://www.salesforce.com/eu/blog/industry-4-0-where-are-we....
  - reactordev 3y ago
    I somehow feel like these are cover pieces so that the execs can show to their board that they are “thinking about AI”.
    - dirtyhippiefree 3y ago
      You should read the article.
      - reactordev 3y ago
        I have, nothing in it had any substance whatsoever. Reinforcing my point that it’s just a cover piece for whatever boardroom argument is to be made for or against AI (or to quote the article: the “digital future”)
        dirtyhippiefree 3y ago
        The fact •you• don’t see substance doesn’t mean it’s not there…
  - dirtyhippiefree 3y ago
    3-D printers in stores was the third…
stonogo 3y ago
It's amazing how every few years we get another article claiming Von Neumann architecture can't cut it, we're not measuring computers right, and we're going to have to give up general-purpose computing any day now. Sometimes they change the order, but it's always the same claims, and it's always the 'unique' circumstances of (insert date here) that make everything different this time. I wonder why they bother?
geodel 3y ago
As per this 5th epoch seems to an answering machine for users whereas all compute is owned out cloud vendors. All efficiency gains is to be accrued by cloud companies and users can pay less(due to competition) or more (due to rise in costs) depends on which way wind blows.
tracerbulletx 3y ago
Can we please stop taking everything Google says seriously? This is a white paper for bad CTOs with made up terminology and self serving predictions.
GMoromisato 3y ago
Here's my History of Computing, which I arrogantly think is better, though more focused on program architectures: https://medium.com/@gridwhale/gridwhale-and-a-brief-history-of-computing-be24e5263328 https://medium.com/@gridwhale/gridwhale-and-a-brief-history-... I wrote that before the AI revolution, and I honestly don't know what's going to happen. The future could go one of two ways: 1. AI just becomes a smarter compiler. We programmers still "write" code, though maybe at a higher level. AI helps us deal with complexity, but we're still the ones designing UX, architecture, etc. 2. AI becomes the front-end layer for everything, so that users only interact with an AI and that AI carries out tasks as needed. All other software is just an implementation detail (the way we software people think about hardware). Most likely we will start with #1 and evolve into #2.
- 0xcafefood 3y ago
  I'm not so sure about either of these, but especially #2. Engineers need to control the behavior of their software very precisely. The fantasy of firing all your engineers and just letting product folks write something extremely high-level like "Write me an app for reviewing movies" and letting a language model fill in all the details seems like a nonstarter. First, the details it imagines likely won't be what you actually want. So they'll have to start getting more and more precise. So precise in fact that it's back to "programming" instead of something like actual natural language. Second, this seems unlikely to provide stability over time. Trying to evolve your FE, BE and storage to remain compatible between one prompt and the next or one language model and the next likely will involve specifying those details fully as well. Natural language just isn't precise enough for this. There is way too much inference involved. This situation is fundamentally different from abstractions that allow one group of engineers to encapsulate implementation details within a system.
  - uticus 3y ago
    > the fantasy...seems like a nonstarter i see what you mean. but I wonder if this is because we have difficulty imagining a totally possible thing. i mean, surely at one point it must have seemed like less machine code engineers, and more script monkeys writing something extremely high level, was a nonstarter. or perhaps firing system software engineers and just letting FE devs write something extremely high level must have seemed like a nonstarter at one point. but here we are. > natural language just isn't precise enough for this i often make the mistake of thinking business needs something more precise than what it is actually asking for (or what it truly needs). of course precise dollars and cents truly matter in some situations, but in other situations a crud app or report has a great deal of flexibility in its requirements. not willing to die on this hill, but i wonder if precise software and precise computer modelling is because the tools require it to be precise? and if so, a less precise interface tool may well lead to less precise inputs that still meet the underlying business requirements.
Aerbil313 3y ago
> Complexity will only increase in the years ahead, essentially requiring new declarative programming models focused on intent, the user, and business logic. This line of thought has never occurred to me before. The idea that complexity will be so unmanageable that we’ll need to rewrite the computing stack in a more easily debuggable, functional way. Modern computing seems simple: The kernel and your application on top of it, but the existence of solutions like Antithesis suggests something different.
Aerbil313 3y ago
> Complexity will only increase in the years ahead, essentially requiring new declarative programming models focused on intent, the user, and business logic. This line of thought has never occurred to me before. The idea that complexity will be so unmanageable that we’ll need to rewrite the computing stack in a more easily debuggable, functional way. Modern computing seems simple: The kernel and your application on top of it, but the existence of solutions like Antithesis suggests something different.
Ecstatify 3y ago
keynote in 2023 at the University of Washington for The Allen School's Distinguished Lecture Series. https://www.youtube.com/watch?v=9lBbqH_1KS4 https://www.youtube.com/watch?v=9lBbqH_1KS4
hulitu 3y ago
> The fifth epoch of distributed computing It this the 5th generation of computing, Japanese were talking about in the 90's ? It took some time... /s
- defrost 3y ago
  Talking about in the early 1980s actually - I recall as I wrote an meta essay on all that in ~ 1985 that was a waste of time in hindsight. https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Systems https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Syst... Long memories I guess :-)

58 points by simonpure 3y ago

It's not distributed if it's all owned by the same corporation. If anything distributed computing has gone backwards the last decade. This switch to centralization was mostly forced by people using low functionality dumb terminals (smart phones) that users don't control and that literally can't hold a TCP connection open.

cmrx64 3y ago
don’t conflate decentralized with distributed
toast0 3y ago
> This switch to centralization was mostly forced by people using low functionality dumb terminals (smart phones) that users don't control and that literally can't hold a TCP connection open. Not being able to hold a TCP connection open is an implementation choice of the OS developer (and some choices at the network level too). At WhatsApp, we would find Nokia Symbian devices with 45 day old TCP connections to our servers anytime we looked. Nokia S40 isn't a smart phone platform, but it could hold long connections too. Old versions of Android could do it, although old versions of Android had trouble switching between wifi and cellular. You can probably do it on current Android if you turn off all the Doze or whatever stuff (which isn't always easy to turn off). There's an efficiency argument for having a single long TCP connection for platform push, rather than one per app that needs it, of course. But in that case, platform push better be reliable.
- superkuh 3y ago
  I didn't mean it was infeasible. I meant holding open TCP connections are avoided by choice to save battery power because transmitting wirelessly is a power drain. But also for all wireless there's a random round trip time and so exponential backoff. The longer you hold open the more likely. Almost all smart phones and their software are gimped in this way; the outliers you mention are interesting but not significant.

swozey 3y ago

I think that the engineers who began their careers in the first epoch are going to turn out to be some of the most productive and technically skilled having gone through each epoch which, frankly, grew easier to deal with every time. They call that point out, basically, but I had no idea it'd be a 100x improvement. > While we cannot predict the breakthroughs that will be delivered in this fifth epoch of computing, we do know that each previous epoch has been characterized by a factor of 100x improvement in scale, efficiency, and cost-performance, all while improving security and reliability. I'm worried about 5th gen epoch engineers. On one side there is a massive amount of smart rust programmers who are super young in all the rust communities just as an example and on the other a lot of people who don't know anything beyond a gui operating system and have never owned a computer and do everything by phone. I've been happily teaching linux quite a bit lately.

zitterbewegung 3y ago
I can imagine fifth epoch engineers only doing code review and prompt engineering or handling major incidence once the LLMs handle the code writing.
- ActionHank 3y ago
  My son is the only one in his class that has a PC. There is going to be an astounding gap between those in the know and those who are not.
  - dagw 3y ago
    I've noticed the same. I'm convinced that the generation that is under 16 today will on average know a lot less about computers compared to 20 years ago.
    - ActionHank 3y ago
      Not my son, he's counting binary and hex, built his PC with me, ran network cables with me, is coding his own games for himself and his friends. Assuming some form of ubiquity for AI tooling people like him are going to build amazing things.

anonymousDan 3y ago

A lot of words to say not much frankly.

thundergolfer 3y ago

> Epoch five: from information to insights Eugh, this article is full of MBA-speak, not something I'd expect from a highly technical Google Fellow.

JSR_FDED 3y ago
Geez I only just wrapped my head around the “fourth Industrial Revolution” we are apparently in… https://www.salesforce.com/eu/blog/industry-4-0-where-are-we-going/#:~:text=What%20is%20the%20Fourth%20Industrial%20Revolution%3F,Things%20and%20cyber%2Dphysical%20systems https://www.salesforce.com/eu/blog/industry-4-0-where-are-we....
- reactordev 3y ago
  I somehow feel like these are cover pieces so that the execs can show to their board that they are “thinking about AI”.
  - dirtyhippiefree 3y ago
    You should read the article.
    - reactordev 3y ago
      I have, nothing in it had any substance whatsoever. Reinforcing my point that it’s just a cover piece for whatever boardroom argument is to be made for or against AI (or to quote the article: the “digital future”)
      - dirtyhippiefree 3y ago
        The fact •you• don’t see substance doesn’t mean it’s not there…
- dirtyhippiefree 3y ago
  3-D printers in stores was the third…

stonogo 3y ago

It's amazing how every few years we get another article claiming Von Neumann architecture can't cut it, we're not measuring computers right, and we're going to have to give up general-purpose computing any day now. Sometimes they change the order, but it's always the same claims, and it's always the 'unique' circumstances of (insert date here) that make everything different this time. I wonder why they bother?

geodel 3y ago

As per this 5th epoch seems to an answering machine for users whereas all compute is owned out cloud vendors. All efficiency gains is to be accrued by cloud companies and users can pay less(due to competition) or more (due to rise in costs) depends on which way wind blows.

tracerbulletx 3y ago

Can we please stop taking everything Google says seriously? This is a white paper for bad CTOs with made up terminology and self serving predictions.

GMoromisato 3y ago

Here's my History of Computing, which I arrogantly think is better, though more focused on program architectures: https://medium.com/@gridwhale/gridwhale-and-a-brief-history-of-computing-be24e5263328 https://medium.com/@gridwhale/gridwhale-and-a-brief-history-... I wrote that before the AI revolution, and I honestly don't know what's going to happen. The future could go one of two ways: 1. AI just becomes a smarter compiler. We programmers still "write" code, though maybe at a higher level. AI helps us deal with complexity, but we're still the ones designing UX, architecture, etc. 2. AI becomes the front-end layer for everything, so that users only interact with an AI and that AI carries out tasks as needed. All other software is just an implementation detail (the way we software people think about hardware). Most likely we will start with #1 and evolve into #2.

0xcafefood 3y ago
I'm not so sure about either of these, but especially #2. Engineers need to control the behavior of their software very precisely. The fantasy of firing all your engineers and just letting product folks write something extremely high-level like "Write me an app for reviewing movies" and letting a language model fill in all the details seems like a nonstarter. First, the details it imagines likely won't be what you actually want. So they'll have to start getting more and more precise. So precise in fact that it's back to "programming" instead of something like actual natural language. Second, this seems unlikely to provide stability over time. Trying to evolve your FE, BE and storage to remain compatible between one prompt and the next or one language model and the next likely will involve specifying those details fully as well. Natural language just isn't precise enough for this. There is way too much inference involved. This situation is fundamentally different from abstractions that allow one group of engineers to encapsulate implementation details within a system.
- uticus 3y ago
  > the fantasy...seems like a nonstarter i see what you mean. but I wonder if this is because we have difficulty imagining a totally possible thing. i mean, surely at one point it must have seemed like less machine code engineers, and more script monkeys writing something extremely high level, was a nonstarter. or perhaps firing system software engineers and just letting FE devs write something extremely high level must have seemed like a nonstarter at one point. but here we are. > natural language just isn't precise enough for this i often make the mistake of thinking business needs something more precise than what it is actually asking for (or what it truly needs). of course precise dollars and cents truly matter in some situations, but in other situations a crud app or report has a great deal of flexibility in its requirements. not willing to die on this hill, but i wonder if precise software and precise computer modelling is because the tools require it to be precise? and if so, a less precise interface tool may well lead to less precise inputs that still meet the underlying business requirements.

Aerbil313 3y ago

> Complexity will only increase in the years ahead, essentially requiring new declarative programming models focused on intent, the user, and business logic. This line of thought has never occurred to me before. The idea that complexity will be so unmanageable that we’ll need to rewrite the computing stack in a more easily debuggable, functional way. Modern computing seems simple: The kernel and your application on top of it, but the existence of solutions like Antithesis suggests something different.

Aerbil313 3y ago

Ecstatify 3y ago

keynote in 2023 at the University of Washington for The Allen School's Distinguished Lecture Series. https://www.youtube.com/watch?v=9lBbqH_1KS4 https://www.youtube.com/watch?v=9lBbqH_1KS4

hulitu 3y ago

> The fifth epoch of distributed computing It this the 5th generation of computing, Japanese were talking about in the 90's ? It took some time... /s

defrost 3y ago
Talking about in the early 1980s actually - I recall as I wrote an meta essay on all that in ~ 1985 that was a waste of time in hindsight. https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Systems https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Syst... Long memories I guess :-)