5 ms·

I am pretty concerned about AGI. I love computer science and programming. I would love to get into research after I finish my Masters in a couple of years. I t

I am pretty concerned about AGI. I love computer science and programming. I would love to get into research after I finish my Masters in a couple of years. I think it's fair to say that _some day_ we will achieve AGI. It might not be in 10 years, or 1000, but eventually we'll get there. When we do have AGI, what would that mean for someone like me? Would there be any reason to have programmers (or similar) at all? Would computer science research (or scientific research at all!) exist, or is that something a computer could do? I think humans might start pursuing more creative/leisure activities, but... I like learning and working! --- The anxiety of the above has made me feel like it's not worth being ambitious. What's the point of research if some AI might do it better than me in 10-20 years? What's the point of joining a startup or... doing anything? I realize this sounds a bit absurd, but it is something that I've really feared since ChatGPT came about.

99_00 3y ago
>What's the point of research if some AI might do it better than me in 10-20 years? You said you enjoy learning and working.
- apsurd 3y ago
  This comment is beautiful in its ring of truth.
- mitthrowaway2 3y ago
  There's research of the lab variety, and research of the library variety. It's valid to feel a sense of loss if all that remains for us is the library kind.
eslaught 3y ago
The original article gets into this at the end. It's easy to get into a cyclic argument here, where you posit a tool that can do "anything" and then try to derive consequences from that. What we don't know is what its actual capabilities will be, or how transformative those will be. There's a prayer about knowing the difference between what you can do and what you can't, and setting aside the things you can't do to focus on what you can. To me, this seems like the right way to approach this.
apsurd 3y ago
10 vs 1,000 years before AGI is a lifetime of difference. I liken it to "not knowing the future". All humans face the same anxiety of an unknown future. In this way, I don't think AGI is a _special_ concern to worry about any more than when your death date will be. And if the concern is "I want to do something meaningful" well... that's a whole other realm of philosophy of meaning. TLDR: I don't think meaningful work is reserved to "influences the thousandth generation after non nonexistence".
- shepherdjerred 3y ago
  > TLDR: I don't think meaningful work is reserved to "influences the thousandth generation after non nonexistence". Sorry, I put that 10 vs 1000 year generation just to focus the conversation on AGI and not whether or not it's possible in some timeframe. I personally feel like we could be _very_ close in 10-20 years, which is within my lifetime. > All humans face the same anxiety of an unknown future. In this way, I don't think AGI is a _special_ concern to worry about any more than when your death date will be. This is actually very reassuring, and I think a really good way for me to frame it for myself.
Barrin92 3y ago
Chess engines overtook human chess players more than a decade or two ago and human chess is thriving. Activities don't become less rewarding for you simply because someone else is better at it, even if that's a machine. A robotic arm can probably lift more than I do, doesn't take my enjoyment from working out away. And people enjoy the human aspect and narrative in any activity in its own right. And that's the worst case in a way, because a chess computer is a really stupid machine. Getting beaten by minimax is kind of annoying. AGIs on the other hand if they ever exist will be very fascinating entities indeed and I wouldn't mind if they're better at anything than I am. And the same goes for scientific research. I haven't stopped doing math because Terence Tao is better than I'll ever be. How many people can seriously claim to be at the frontiers of science? The value in learning isn't in being the best guy or girl there is.
- MicropenisMike 3y ago
  Chess is a leisure activity, not work, you cannot compare the impact of automation of the two. Also, chess engines did kill chess. Chess used to be a spectacle on the national stage, from 2000 up until Queen's Gambit and the following streamer boom, chess was largely dead. https://trends.google.com/trends/explore?date=all&q=Chess&hl=en https://trends.google.com/trends/explore?date=all&q=Chess&hl...
  - joshuahedlund 3y ago
    Human chess is still an activity that earns people money tho
    - MicropenisMike 3y ago
      Even today, there's a handful of people who make a full living off chess. The owners of chess.com, a few streamers, a handful of super GMs, and chess teachers, and the latter will be automated away. And prior to the explosion in popularity? Barely anyone.
  - sanderjd 3y ago
    I'm extremely skeptical of the claim that engines killed mainstream interest in chess. The trend seems to only go back to 2004, which is not long enough. My sense growing up in the 90s was that chess was a niche thing for nerds. I think it seems somewhat more mainstream now than it was then, while remaining fundamentally a niche thing for nerds :) But it seems very clear to me that it never underwent the broad collapse that I remember being broadly predicted after Kasparov vs Deep Blue.
raincole 3y ago
I ensure you no one actually cares about what happens 1000 years later. If someone actually cares about that, we'll call them mentally disordered. By actually I mean to expend a significant part of the resource (time & money) you own to prepare for it.
barrysteve 3y ago
There are many ideas that have not been covered by programming or predictive AI. AGI sounds like Skynet, the Holodeck, the super information highway and all the buzzwords. Wake me up in 20yrs, cause I will probably be using c++ and COBOL to sticky tape bank mainframes back together, when the boomers have finally given up on keeping those beasts alive and staving off the thirty seventh crypto revolution.
cm277 3y ago
I agree with the article; GPT is not AGI, it's ABSI (automated BS intelligence). That's certainly useful and it will automate some jobs, but a) it's just an enabling technology for new applications, b) we haven't seen these applications yet, so we have no real idea what the impact will be. To put it another way: the enabling tech for iPhone was (mainly) multitouch capacitive displays. I dont think anyone predicted Uber or taxi drivers being at risk because of multitouch when it was unveiled (late 90s?), but here we are.
- koheripbal 3y ago
  We are all BS intelligence. People who think little of AI forget how bad the rest of us are also
thatsadude 3y ago
I'm concerned about who AGI will serve? Rich people surely!
- koheripbal 3y ago
  Maybe at first, but they will likely not keep any humans around for long.
  - moffkalast 3y ago
    Luckily capitalism solves that bit for them. Just gradually automate jobs so people can't make a living and they'll slowly die off in the gutter bit by bit. The slower it, is the less chance of a revolt. To paraphrase that old quote: >First they automated the artists, and I did not speak out because I was not an artist. >Then they automated the writers, and I did not speak out because I was not a writer. >Then they automated the office workers, and I did not speak out because I was not an office worker. >Then they automated me and there was no one left to speak for me.
  - candiddevmike 3y ago
    Who does they refer to, the AGI or rich people?
- brightlancer 3y ago
  > I'm concerned about who AGI will serve? Rich people surely! Almost everything invented in the past 200 years was originally sold to rich people, because they were the only ones who could afford it -- then the price came down and middle-class folks could buy it, then the price came down more and virtually everyone was able to afford it. Thanks capitalism!
giuscri 3y ago
If we have AGI we won’t have to work anymore so why worrying about the future? In the meantime enjoy and try to partecipate in this new paradigm. (Unless the AGI will come up capitalism is the best of the possible systems and they will rule them out of the equation)
- thumbuddy 3y ago
  I hope you watch more scifi movies and read more books. What people do with technology is basically never utopian or utopia adjacent.
  - giuscri 3y ago
    I get a decent amount of that and dystopian scenarios are just our horror fantasies. I like them too but I like to have faith in people.
Shorel 3y ago
What's the point of research if some new student can do it better than me, right now? What's the point of playing chess if Magnus Carlsen will beat me every time we play? This seems a bit absurd, but it's something I have feared since I learned I am not the best at everything.
- jiggawatts 3y ago
  There's only one Magnus Carlsen. Now ask yourself what's the point of playing chess if every chessplayer except you is a copy of Magnus Carlsen.
  - sanderjd 3y ago
    But that's just it: there are as we speak an infinite number of chess "players" who are better than Magnus Carlsen. And chess has never been more popular.
- shepherdjerred 3y ago
  Once we have one AGI machine wouldn't we quickly have as many as we wanted?
sanderjd 3y ago
The thing that makes me more optimistic about this specific question is chess. Chess has been, from a human perspective, fully solved by AI. But it has never been more popular. Why? I find it both hard to explain and kind of intuitive. Even though we can't beat computers anymore, we can still learn from them, but remain human, and so human play remains its own unique thing, but infused with insights from the super human computer players. And on net, human chess is now just better. I think science and math could be the same, which is really exciting! That is, maybe humans would be learning from what the super-human computers are doing, but using that to do different, more human, things.
- poochkoishi728 3y ago
  Love this take! Each contributing and sharing our own style in combination with others.

99_00 3y ago

>What's the point of research if some AI might do it better than me in 10-20 years? You said you enjoy learning and working.

apsurd 3y ago
This comment is beautiful in its ring of truth.
mitthrowaway2 3y ago
There's research of the lab variety, and research of the library variety. It's valid to feel a sense of loss if all that remains for us is the library kind.

eslaught 3y ago

The original article gets into this at the end. It's easy to get into a cyclic argument here, where you posit a tool that can do "anything" and then try to derive consequences from that. What we don't know is what its actual capabilities will be, or how transformative those will be. There's a prayer about knowing the difference between what you can do and what you can't, and setting aside the things you can't do to focus on what you can. To me, this seems like the right way to approach this.

apsurd 3y ago

10 vs 1,000 years before AGI is a lifetime of difference. I liken it to "not knowing the future". All humans face the same anxiety of an unknown future. In this way, I don't think AGI is a _special_ concern to worry about any more than when your death date will be. And if the concern is "I want to do something meaningful" well... that's a whole other realm of philosophy of meaning. TLDR: I don't think meaningful work is reserved to "influences the thousandth generation after non nonexistence".

shepherdjerred 3y ago
> TLDR: I don't think meaningful work is reserved to "influences the thousandth generation after non nonexistence". Sorry, I put that 10 vs 1000 year generation just to focus the conversation on AGI and not whether or not it's possible in some timeframe. I personally feel like we could be _very_ close in 10-20 years, which is within my lifetime. > All humans face the same anxiety of an unknown future. In this way, I don't think AGI is a _special_ concern to worry about any more than when your death date will be. This is actually very reassuring, and I think a really good way for me to frame it for myself.

Barrin92 3y ago

Chess engines overtook human chess players more than a decade or two ago and human chess is thriving. Activities don't become less rewarding for you simply because someone else is better at it, even if that's a machine. A robotic arm can probably lift more than I do, doesn't take my enjoyment from working out away. And people enjoy the human aspect and narrative in any activity in its own right. And that's the worst case in a way, because a chess computer is a really stupid machine. Getting beaten by minimax is kind of annoying. AGIs on the other hand if they ever exist will be very fascinating entities indeed and I wouldn't mind if they're better at anything than I am. And the same goes for scientific research. I haven't stopped doing math because Terence Tao is better than I'll ever be. How many people can seriously claim to be at the frontiers of science? The value in learning isn't in being the best guy or girl there is.

MicropenisMike 3y ago
Chess is a leisure activity, not work, you cannot compare the impact of automation of the two. Also, chess engines did kill chess. Chess used to be a spectacle on the national stage, from 2000 up until Queen's Gambit and the following streamer boom, chess was largely dead. https://trends.google.com/trends/explore?date=all&q=Chess&hl=en https://trends.google.com/trends/explore?date=all&q=Chess&hl...
- joshuahedlund 3y ago
  Human chess is still an activity that earns people money tho
  - MicropenisMike 3y ago
    Even today, there's a handful of people who make a full living off chess. The owners of chess.com, a few streamers, a handful of super GMs, and chess teachers, and the latter will be automated away. And prior to the explosion in popularity? Barely anyone.
- sanderjd 3y ago
  I'm extremely skeptical of the claim that engines killed mainstream interest in chess. The trend seems to only go back to 2004, which is not long enough. My sense growing up in the 90s was that chess was a niche thing for nerds. I think it seems somewhat more mainstream now than it was then, while remaining fundamentally a niche thing for nerds :) But it seems very clear to me that it never underwent the broad collapse that I remember being broadly predicted after Kasparov vs Deep Blue.

raincole 3y ago

I ensure you no one actually cares about what happens 1000 years later. If someone actually cares about that, we'll call them mentally disordered. By actually I mean to expend a significant part of the resource (time & money) you own to prepare for it.

barrysteve 3y ago

There are many ideas that have not been covered by programming or predictive AI. AGI sounds like Skynet, the Holodeck, the super information highway and all the buzzwords. Wake me up in 20yrs, cause I will probably be using c++ and COBOL to sticky tape bank mainframes back together, when the boomers have finally given up on keeping those beasts alive and staving off the thirty seventh crypto revolution.

cm277 3y ago

I agree with the article; GPT is not AGI, it's ABSI (automated BS intelligence). That's certainly useful and it will automate some jobs, but a) it's just an enabling technology for new applications, b) we haven't seen these applications yet, so we have no real idea what the impact will be. To put it another way: the enabling tech for iPhone was (mainly) multitouch capacitive displays. I dont think anyone predicted Uber or taxi drivers being at risk because of multitouch when it was unveiled (late 90s?), but here we are.

koheripbal 3y ago
We are all BS intelligence. People who think little of AI forget how bad the rest of us are also

thatsadude 3y ago

I'm concerned about who AGI will serve? Rich people surely!

koheripbal 3y ago
Maybe at first, but they will likely not keep any humans around for long.
- moffkalast 3y ago
  Luckily capitalism solves that bit for them. Just gradually automate jobs so people can't make a living and they'll slowly die off in the gutter bit by bit. The slower it, is the less chance of a revolt. To paraphrase that old quote: >First they automated the artists, and I did not speak out because I was not an artist. >Then they automated the writers, and I did not speak out because I was not a writer. >Then they automated the office workers, and I did not speak out because I was not an office worker. >Then they automated me and there was no one left to speak for me.
- candiddevmike 3y ago
  Who does they refer to, the AGI or rich people?
brightlancer 3y ago
> I'm concerned about who AGI will serve? Rich people surely! Almost everything invented in the past 200 years was originally sold to rich people, because they were the only ones who could afford it -- then the price came down and middle-class folks could buy it, then the price came down more and virtually everyone was able to afford it. Thanks capitalism!

giuscri 3y ago

If we have AGI we won’t have to work anymore so why worrying about the future? In the meantime enjoy and try to partecipate in this new paradigm. (Unless the AGI will come up capitalism is the best of the possible systems and they will rule them out of the equation)

thumbuddy 3y ago
I hope you watch more scifi movies and read more books. What people do with technology is basically never utopian or utopia adjacent.
- giuscri 3y ago
  I get a decent amount of that and dystopian scenarios are just our horror fantasies. I like them too but I like to have faith in people.

Shorel 3y ago

What's the point of research if some new student can do it better than me, right now? What's the point of playing chess if Magnus Carlsen will beat me every time we play? This seems a bit absurd, but it's something I have feared since I learned I am not the best at everything.

jiggawatts 3y ago
There's only one Magnus Carlsen. Now ask yourself what's the point of playing chess if every chessplayer except you is a copy of Magnus Carlsen.
- sanderjd 3y ago
  But that's just it: there are as we speak an infinite number of chess "players" who are better than Magnus Carlsen. And chess has never been more popular.
shepherdjerred 3y ago
Once we have one AGI machine wouldn't we quickly have as many as we wanted?

sanderjd 3y ago

The thing that makes me more optimistic about this specific question is chess. Chess has been, from a human perspective, fully solved by AI. But it has never been more popular. Why? I find it both hard to explain and kind of intuitive. Even though we can't beat computers anymore, we can still learn from them, but remain human, and so human play remains its own unique thing, but infused with insights from the super human computer players. And on net, human chess is now just better. I think science and math could be the same, which is really exciting! That is, maybe humans would be learning from what the super-human computers are doing, but using that to do different, more human, things.

poochkoishi728 3y ago
Love this take! Each contributing and sharing our own style in combination with others.