3 ms·

"It doesn't know how it knows what it knows" is a very common criticism. "It's just predicting the next token but doesn't really 'understand' anything" is anoth

by fnovd 4y ago

"It doesn't know how it knows what it knows" is a very common criticism. "It's just predicting the next token but doesn't really 'understand' anything" is another. To me it's like saying, "How can humans actually play baseball if we don't even explain how we throw balls and swing bats at moving objects?" Wording it that way, it just sounds ridiculous. As ridiculous as I think this AI conversation is. I don't know why we care so much about model transparency. Yes, it's worth scientific pursuit to understand what we're building and how it works, we should absolutely try to know as much as we can. But if the bar for "true intelligence" is a machine capable of doing things that we can't do ourselves, is that saying we aren't "truly" intelligent, either? If we're all of a sudden not allowed to leverage systems we don't fully understand then I guess we shouldn't even be at the spot where we are as a species. We've been doing the right things for the wrong reasons for ages, it seems to work OK.

shinycode 4y ago
Because contrary to your example, this intelligence could « understand » the flaws of our systems and lock us down if it wants to. For example by creating a massive ransomware and exploiting 0-days because it’s much better and faster than us to analyze code. What happens then ? If it’d like to do harm for any reason (it might not be harm for an AI but just the « right thing » according to its criterias) what would we do ? Wouldn’t it be a good thing to know how it works inside precisely ? Or « whatever happens, happens » ?
agalunar 4y ago
I suppose it depends on what you mean by "knowing how I know what I know". That could refer to "knowing the physiology of my brain", which is what you seemed to be referring to. But it could also refer to "explaining my feelings or decision making process". I've lived in my head for many years and had many chances to observe myself, so I can perform post hoc self-analysis with reasonable accuracy, and on a good day, I can even be self-aware in the very moment (which is useful in getting myself to do what I want). I think maybe that second thing is what people are worried about AI lacking. > If we're all of a sudden not allowed to leverage systems we don't fully understand then I guess we shouldn't even be at the spot where we are as a species. We've been doing the right things for the wrong reasons for ages, it seems to work OK. I don't say this merely to be contentious, but I don't think I have quite as optimistic an outlook myself ^_^' which isn't to say I think we shouldn't meddle with the world, just that we sometimes get ourselves in over our heads. But I'm hopeful.

by fnovd 4y ago

shinycode 4y ago

Because contrary to your example, this intelligence could « understand » the flaws of our systems and lock us down if it wants to. For example by creating a massive ransomware and exploiting 0-days because it’s much better and faster than us to analyze code. What happens then ? If it’d like to do harm for any reason (it might not be harm for an AI but just the « right thing » according to its criterias) what would we do ? Wouldn’t it be a good thing to know how it works inside precisely ? Or « whatever happens, happens » ?

agalunar 4y ago

I suppose it depends on what you mean by "knowing how I know what I know". That could refer to "knowing the physiology of my brain", which is what you seemed to be referring to. But it could also refer to "explaining my feelings or decision making process". I've lived in my head for many years and had many chances to observe myself, so I can perform post hoc self-analysis with reasonable accuracy, and on a good day, I can even be self-aware in the very moment (which is useful in getting myself to do what I want). I think maybe that second thing is what people are worried about AI lacking. > If we're all of a sudden not allowed to leverage systems we don't fully understand then I guess we shouldn't even be at the spot where we are as a species. We've been doing the right things for the wrong reasons for ages, it seems to work OK. I don't say this merely to be contentious, but I don't think I have quite as optimistic an outlook myself ^_^' which isn't to say I think we shouldn't meddle with the world, just that we sometimes get ourselves in over our heads. But I'm hopeful.