3 ms·

Let's say you, a human, were given access to a ridiculously-large trove of almost-working software; do you believe you would be unable to learn to program corre

by saurik 4y ago

Let's say you, a human, were given access to a ridiculously-large trove of almost-working software; do you believe you would be unable to learn to program correctly? (Related: would you even need to look at much of that software before you were able to code well?) I am extremely confident that I am better than almost all of the code I learned to program with. If nothing else, someone out there must have written the best version of some particular function, and they didn't get to see a better version beforehand. When I look at intro programming books now, I consider many of the examples sufficiently flawed that I tell people I am teaching who are using these books "well, don't do that... I guess the author doesn't understand why that's a problem :/". And yet, somehow, despite learning from a bunch of bad examples, humans learn to become good. Hell: a human can then go off and work alone in the woods improving their craft and become better--even amazing!--given no further examples as training data. To me, that is why I have such little fear of these models. People look at them and are all "omg they are so intelligent" and yet they generate an average of what they are given rather than the best of what they are given: this tech is, thereby, seemingly, a dead end for actual intelligence. If these models were ever to become truly intelligent, they should--easily!--be able to output something much better than what they were given, and it doesn't even seem like that's on the roadmap given how much fear people have over contamination of the training data set. If you actually believe that we'll be able to see truly intelligent AI any time in the near future, I will thereby claim it just won't matter how much of the data out there is bullshit, because an actually-intelligent being can still learn and improve under such conditions.

alchemist1e9 4y ago
You are in a serious case of denial right now. Edit: Only took a few hours before in real life what I was trying to imply the denial was about is already happening: https://news.ycombinator.com/item?id=33855416 https://news.ycombinator.com/item?id=33855416
- saurik 4y ago
  I am claiming there are two paths: one where this (specific) branch of tech is a dead end, and one where it doesn't matter how much bullshit exists in the training set (and so we shouldn't be too concerned about that). I claim this is the case because a truly intelligent system will still be able to learn despite the bullshit. Do you believe this is wrong? That I should simultaneously be concerned that some iteration of this tech--not some different concept but this current lineage of large models--is intelligent and yet ALSO that it isn't going to work because the training set is full of garbage? The version of this tech that works--and maybe someone is working on it right now--isn't going to care about bullshit in the training set. That simply doesn't seem to be a mere scale-up of this tech to run in more computers (or, of course, using more training data): that seems like it requires a fundamentally different algorithm concept.
  - anon7725 4y ago
    You can interact with the system and see that it is working on some level today. It’s not hard to extrapolate where its capabilities will be a few years from now, since these are changes of degree not of kind. We have witnessed the change of kind with this model. Is it intelligent? A great question for science, and one that could be investigated while entire industries are upended by this thing.
    - saurik 4y ago
      Oh yeah, it totally works! I have even had quite a bit of fun with Stable Diffusion. I'd probably also be playing with something like Copilot if it were open source. But like, the person I am responding to is concerned--as are many people--that we are going to flood the world with shitty training data and then no longer be able to build these models... and that's either not the case and no one should concern themselves with that or, alternatively, these models need some fundamental improvement before they don't have the property of only being as good as average inputs.
      - alchemist1e9 4y ago
        Edit: Already happening https://news.ycombinator.com/item?id=33855416 https://news.ycombinator.com/item?id=33855416 So I’ll respond here instead as the conversation progressed. I would say the quality of the input data is likely very important component and I think you are wrong overall in your opinion. I would say the quality of input training data is so important that I’ve personally been thinking I should probably start data hoarding myself, specifically around my skillsets. Additionally when you understand that embedding like word2vec are perhaps a significant part of the improvement, not just the transformers, it occurs that adding symbolic capabilities, like classic symbolic reasoning and for example symbolic computing, like Mathematica, then maybe provide also true computational power, floating point, so it can write, debug, and execute it’s own output… it must be getting closer and closer to AGI. when you play with the system, which I feel most of the commenters on HN which are being dismissive likely have not personally spent enough time exploring it’s current capabilities, then there is no way any well trained and experienced SWE isn’t blown away. This is why I said you are in denial. I happen to thing AGI will be very beneficial for humanity and I also think this is a positive for SWE by humans, including myself. I will very likely be a paying customer when the trial ends in a few weeks.
mk_stjames 4y ago
I kinda agree... and I would really not want to be a very young person right now, as I feel the world will be much harder to navigate and learn from. It takes so much more energy to refute bullshit than to make it, and if this starts creeping into computer science then christ I wouldn't want to be a part of it. I can imagine a sci-fi like story in the near future where CS students are searching out for 'coveted' copies of K&R, and reading human-written Python documentation, all pre-2023-vintage, because that was 'the last of the good stuff'. Hell, I could see far future stories about youth who join religions around the 'old ways' seeking the wisdom that comes with learning from actual books and docs written by actual people, instead of regurgitated teachings from an inbred, malformed, super-AI.
- ricardobayes 4y ago
  We are experiencing the same as our forefathers who worked on steam engines or wrote calligraphy by hand. Or like the ancient Egyptian accountants using abacus. Jobs change, yes, we might undergo a major change, but we will do just fine.
- deleted 4y ago
  [deleted]
- rogual 4y ago
  One day it'll be hard to know what the old ways even were. Which of the thousands of slightly-different PDFs claiming to be the original K&R is the real one? Which of the million history texts? Which Bible? Which maps?

by saurik 4y ago

alchemist1e9 4y ago

You are in a serious case of denial right now. Edit: Only took a few hours before in real life what I was trying to imply the denial was about is already happening: https://news.ycombinator.com/item?id=33855416 https://news.ycombinator.com/item?id=33855416

saurik 4y ago
I am claiming there are two paths: one where this (specific) branch of tech is a dead end, and one where it doesn't matter how much bullshit exists in the training set (and so we shouldn't be too concerned about that). I claim this is the case because a truly intelligent system will still be able to learn despite the bullshit. Do you believe this is wrong? That I should simultaneously be concerned that some iteration of this tech--not some different concept but this current lineage of large models--is intelligent and yet ALSO that it isn't going to work because the training set is full of garbage? The version of this tech that works--and maybe someone is working on it right now--isn't going to care about bullshit in the training set. That simply doesn't seem to be a mere scale-up of this tech to run in more computers (or, of course, using more training data): that seems like it requires a fundamentally different algorithm concept.
- anon7725 4y ago
  You can interact with the system and see that it is working on some level today. It’s not hard to extrapolate where its capabilities will be a few years from now, since these are changes of degree not of kind. We have witnessed the change of kind with this model. Is it intelligent? A great question for science, and one that could be investigated while entire industries are upended by this thing.
  - saurik 4y ago
    Oh yeah, it totally works! I have even had quite a bit of fun with Stable Diffusion. I'd probably also be playing with something like Copilot if it were open source. But like, the person I am responding to is concerned--as are many people--that we are going to flood the world with shitty training data and then no longer be able to build these models... and that's either not the case and no one should concern themselves with that or, alternatively, these models need some fundamental improvement before they don't have the property of only being as good as average inputs.
    - alchemist1e9 4y ago
      Edit: Already happening https://news.ycombinator.com/item?id=33855416 https://news.ycombinator.com/item?id=33855416 So I’ll respond here instead as the conversation progressed. I would say the quality of the input data is likely very important component and I think you are wrong overall in your opinion. I would say the quality of input training data is so important that I’ve personally been thinking I should probably start data hoarding myself, specifically around my skillsets. Additionally when you understand that embedding like word2vec are perhaps a significant part of the improvement, not just the transformers, it occurs that adding symbolic capabilities, like classic symbolic reasoning and for example symbolic computing, like Mathematica, then maybe provide also true computational power, floating point, so it can write, debug, and execute it’s own output… it must be getting closer and closer to AGI. when you play with the system, which I feel most of the commenters on HN which are being dismissive likely have not personally spent enough time exploring it’s current capabilities, then there is no way any well trained and experienced SWE isn’t blown away. This is why I said you are in denial. I happen to thing AGI will be very beneficial for humanity and I also think this is a positive for SWE by humans, including myself. I will very likely be a paying customer when the trial ends in a few weeks.

mk_stjames 4y ago

I kinda agree... and I would really not want to be a very young person right now, as I feel the world will be much harder to navigate and learn from. It takes so much more energy to refute bullshit than to make it, and if this starts creeping into computer science then christ I wouldn't want to be a part of it. I can imagine a sci-fi like story in the near future where CS students are searching out for 'coveted' copies of K&R, and reading human-written Python documentation, all pre-2023-vintage, because that was 'the last of the good stuff'. Hell, I could see far future stories about youth who join religions around the 'old ways' seeking the wisdom that comes with learning from actual books and docs written by actual people, instead of regurgitated teachings from an inbred, malformed, super-AI.

ricardobayes 4y ago
We are experiencing the same as our forefathers who worked on steam engines or wrote calligraphy by hand. Or like the ancient Egyptian accountants using abacus. Jobs change, yes, we might undergo a major change, but we will do just fine.
deleted 4y ago
[deleted]
rogual 4y ago
One day it'll be hard to know what the old ways even were. Which of the thousands of slightly-different PDFs claiming to be the original K&R is the real one? Which of the million history texts? Which Bible? Which maps?