3 ms·

Your comment confuses me. You say the author > conveniently sidestep[s] the possibility of the development of new tools and techniques. I don't believe they d

Your comment confuses me. You say the author > conveniently sidestep[s] the possibility of the development of new tools and techniques. I don't believe they did this at all. Here is their summary of their third point: > To automatically construct a good dataset, we require an actionable understanding of which datapoints are important for learning. This turns out to be incredibly difficult. The field has, thus far, completely failed to make progress on this problem, despite expending significant effort. Cracking it would be a field-changing breakthrough, comparable to transitioning from alchemy to chemistry. That does not read to me as conveniently sidestepping the possibility of new tools. Rather, it is acknowledges that to overcome the problem we NEED a new tool, aka a breakthrough. I also disagree with the framing of the third section as: > And that entire section basically boils down to "I personally believe this is hard and will take a long time." They provide both theoretical and empirical evidence of their claim. I find it was well argued, and I'm inclined to agree. Every scenario I can think of with a runaway superintelligence requires a way to automatically improve datasets as part of the learning process.

mrshadowgoose 3y ago
From one of the final paragraphs: "Also, they aren’t getting more-solved over time: we’ve made little-to-no progress on any problem of this sort in the last decade, certainly not the reliable improvements of the sort we’ve seen from supervised learning. This indicates that a breakthrough is needed — and that it is unlikely to be close." Past lack-of-progress is not an indicator of future lack-of-progress. "unlikely to be close" is unknowable, and is just the author's gut feeling.
- scythmic_waves 3y ago
  > Past lack-of-progress is not an indicator of future lack-of-progress. Past lack-of-progress is not proof of future lack-of-progress. But it's most definitely an indicator. > "unlikely to be close" is unknowable, and is just the author's gut feeling. Again I'm sorry but I have to disagree with you here. The very next paragraph reads: > Something that would change my mind on this is if I saw real progress on any problem that is as hard as understanding generalization, e.g. if we were able to train large networks without adversarial examples. Basically, the author has identified a class of problem. They are claiming that little to no progress has been made on any problem in that class. So it's not just that no progress has been made on this specific problem, but that we are stuck on all problems of this type. Thinking of it in that way, I do not find it unreasonable to say we are "unlikely to be close" to a solution. If you have a CS background, I'll make an analogy to computability classes: It's like the author is saying this is an NP hard problem. We have made no progress on any NP hard problem. I think it's reasonable to say that we are unlikely to be close to solving a particular NP hard problem because we've made no progress on any NP hard problem. You're free to disagree with the author's premises. E.g. "this problem is not like those other problems" or "we actually have made progress on those other problems". But I think that the conclusions are reasonable given the premises.
- Certhas 3y ago
  This is just manifestly wrong. Past lack-of-progress absolutely is an indicator of future lack of progress. Like any indicator it isn't perfect. The counterexamples, where progress suddenly starts after decades of no progress, are celebrated breakthroughs of the type the author argues is needed here. These are by definition extremely rare.

mrshadowgoose 3y ago

From one of the final paragraphs: "Also, they aren’t getting more-solved over time: we’ve made little-to-no progress on any problem of this sort in the last decade, certainly not the reliable improvements of the sort we’ve seen from supervised learning. This indicates that a breakthrough is needed — and that it is unlikely to be close." Past lack-of-progress is not an indicator of future lack-of-progress. "unlikely to be close" is unknowable, and is just the author's gut feeling.

scythmic_waves 3y ago
> Past lack-of-progress is not an indicator of future lack-of-progress. Past lack-of-progress is not proof of future lack-of-progress. But it's most definitely an indicator. > "unlikely to be close" is unknowable, and is just the author's gut feeling. Again I'm sorry but I have to disagree with you here. The very next paragraph reads: > Something that would change my mind on this is if I saw real progress on any problem that is as hard as understanding generalization, e.g. if we were able to train large networks without adversarial examples. Basically, the author has identified a class of problem. They are claiming that little to no progress has been made on any problem in that class. So it's not just that no progress has been made on this specific problem, but that we are stuck on all problems of this type. Thinking of it in that way, I do not find it unreasonable to say we are "unlikely to be close" to a solution. If you have a CS background, I'll make an analogy to computability classes: It's like the author is saying this is an NP hard problem. We have made no progress on any NP hard problem. I think it's reasonable to say that we are unlikely to be close to solving a particular NP hard problem because we've made no progress on any NP hard problem. You're free to disagree with the author's premises. E.g. "this problem is not like those other problems" or "we actually have made progress on those other problems". But I think that the conclusions are reasonable given the premises.
Certhas 3y ago
This is just manifestly wrong. Past lack-of-progress absolutely is an indicator of future lack of progress. Like any indicator it isn't perfect. The counterexamples, where progress suddenly starts after decades of no progress, are celebrated breakthroughs of the type the author argues is needed here. These are by definition extremely rare.