8 ms·
Deep learning is creating computer systems we don't fully understand
- iofj 10y agoWhy would anyone use machine learning if not to create systems they don't understand ? That's the whole point of doing it. Not having to understand complex system to understand -or even predict- them is a very good thing indeed. I don't know anything about how to mathematically analyze pixel values to read text, but I can create such a formula using machine learning. After creating the formula I know nothing more about it. So the same person can create programs to read text, transcribe audio, predict loan repayment, ... And when it comes to the bias in the data - bias in the system thing, well, same thing goes for human judgement. Another way of putting this would be garbage in-garbage out. Humans might be able to tell you of bias in the inputs, rather than just using it, but this ability rapidly deteriorates when the dimensionality of the inputs rises. You can tell if there's something weird about combinations of 3 numbers, but you can't tell if there is something off about 150 number sequences.
- ghgfhgfhn 10y agoAs is often the case, the most insightful comment is tucked away at the bottom of the page.
- unclebucknasty 10y agoPerhaps. But the inability to verify or predict output is a real concern, especially in the nascent stages of applying the tech to an increasing number of critical situations.
- abtinf 10y agoAlong these lines, I think these classes of algorithms might have much broader applications and could possibly solve problems that we don't even have good definitions for yet. I'm particularly curious to see how the field develops new techniques for systems that start off with poor quality results but evolve with new data to produce better results. btw, I vouched for and upvoted the parent. It makes an excellent point and, frankly, I'm baffled by the downvotes/dead status.
- blazespin 10y agoThis sounds far fetch and sci fi, but intelligible machines are particularly of importance in avoiding machines becoming self aware unbeknownst. However, I think it's also possible that we may never be able to create machines that are self aware as people are unless we do allow them to build without our understanding of them. There could be a recursive relationship in that an intelligence is not capable of understanding itself.
- flippyhead 10y agoWe could just wait until we completely understand how the human brain works, then replicate it directly. It would take longer but be (supposedly) safer than going for a novel but faster approach involving machine learning, genetic algorithms and the like.
- willismichael 10y agoWe already have the technology to replicate the human brain, you and I and everybody else were born with it.
- calgoo 10y agoI think tech is helping us understand how some of the brain works, or at least helps us to model it. (my own idea, no "proof" or sources)
- danieldk 10y agoI think self-awareness is still so far off that it is not the most pressing problem. The more serious problem is that once we make life-changing choices using machine learning models, the models should be just and accountable. Obviously, these are vague terms, but in terms of machine learning models they can have concrete meanings: * Just: choices should be made on what we consider (morally) relevant inputs. * Accountable: choices should be traceable to inputs and parameters (or eventually training data). E.g. suppose that we find drones that stun people with a weapon acceptable. Such a drone would be just if it decides to stun a person because they have a weapon. Such a drone would be unjust if it decides to stun a person because they have a particular skin color [1]. Now, suppose that such a drone stuns a person robbing a supermarket with a fake weapon. Regardless of whether we consider this to be just, we probably want to know why the drone misrecognized the gun as real (the model should be accountable). [1] There may be a non-uniform prior distribution p(weapon_use|race). But I hope we all agree that using such prior distributions would be absolutely unfair towards the individual.
- scotty79 10y ago> The humans looked straight to the windows to answer this question, but the machines, for some reason, looked at the beds instead. Maybe the machine wasn't sure if most important thing in the question is "window" or "cover", so it looked at a bed looking for bed cover?
- deleted 10y ago[deleted]
- tensor 10y agoGiven that humans all recognize that there is a bed, it's almost certainly the case that humans also looked at the bed when first scanning the image. We don't require a human show the breakdown of it's visual processing cortex for every decision, why do we want to require it of a machine?
- vardump 10y agoWait. Are there still computer systems in 2016 we can fully understand? Except maybe some simpler embedded systems. Last computer system I can somewhat fully understand is OCS Commodore Amiga. CPU, chipset and software.
- gambler 10y agoNormal software is split into logical modules and layers, which you can understand on their own and then use that to reason about code at higher and higher levels. Many ML models do not offer anything comparable.
- deleted 10y ago[deleted]
- jimbokun 10y agoUnequivocally yes. Obviously no one has a full mental model in their head of everything going on in their phone (for example) at a given moment in time. But when it really matters, and there is a specific question to answer, and a programmer with experience writing the kind of software in question has full access to all of the source code, its possible to get an answer to why a program behaved in a particular way. I'm a programmer, and this is what I do every day. And in those rare cases where knowledge of the specific hardware is required, there are experts who understand the hardware and can answer the relevant questions. So for machine learning approaches, what is the equivalent of the "source code"? What resource is there to examine to determine how the system will respond to a given set of inputs, and why? Evidently, there are not clear answers, at least for some machine learning approaches.
- vardump 10y agoTo start with, most computers are running about 20-30 microcontrollers. No source code, most of the time even no access to the firmware. Phones have a bit fewer, but I think you'll find at least about 10. A simple cheap SD memory card has an embedded core, usually ARM7TDMI. Even most lithium batteries have an integrated microcontroller. I'm also a programmer, I do bare metal embedded, kernel drivers and such. I often don't have anyone else to ask from, except occasionally hardware guys. They're also often stumped.
- quadrature 10y agoDoes the Tesla autopilot system use deep learning ?. I assumed it was all control systems and tracking.
- Fishman343 10y agoI think the author has read that Tesla use "deep learning" to make their auto pilot more accurate and presumed that that means a deep learning algorithm is running in the car. What I would guess what they do is run deep learning algorithms on the data produced by the car's systems back at their HQ to design more efficient and complete systems, which they then update all of their cars to use.
- PhantomGremlin 10y agoMinor nit ... Was anyone else bothered by the phrase "show their working"? That's the kind of thing that grates on me because it's just slightly wrong. When I grew up it was always "show your work", not "working". Is this a regional thing?
- AnimalMuppet 10y agoI think it's British rather than US English, but I'm not sure.
- PeCaN 10y agoDo Brits use gerunds other than ‘working’ as nouns or is it specific to that case? Either way, that's really weird.
- strebler 10y agoI have heard Dr. Andrew Ng point out the "well known" fact (in the computer vision community) that nobody, including Dr. David Lowe himself, really understands why SIFT works either. So at least for those of us in the computer vision industry, this isn't maybe such a surprise.
- btbuildem 10y agoIsn't that the essence of this particular approach though? To "learn" a solution in ways us humans would not conceive of? It's a clear consequence that we cannot easily follow that path of "reasoning" easily.
- guidopallemans 10y agoThis is the main difference between Machine Learning and Data Mining; in Machine Learning it's the Machine that learns, in Data Mining it's you (and the machine).
- xkcd-sucks 10y agoMaybe we'll see the emergence of a profession kind of like clinical psychology for diagnosing strange behavior in trained pattern-recognizing machines :)
- gambler 10y agoAsimov thought there will be robopsychologists. Maybe he was right. Although, his robots had a common foundation laid out by the Three Laws of Robotics, and they didn't consume gigabytes of data to learn.
- daveguy 10y ago> and they didn't consume gigabytes of data to learn. Didn't they though? I'm pretty sure a couple of cameras for vision would mean they consume a couple of gigabytes rather quickly. That doesn't include sound, motor stress feedback and any other included sensors. Asimov's robots would have consumed massively more data to learn. Of course, it wouldn't have required the data be curated and classified by humans.
- sghi 10y agoI thought the sirius cybernetics corporation already had something like that!
- deleted 10y ago[deleted]
- PepeGomez 10y agoThe problem is I think that we don't actually know how to program an AI. Neural networks only sidestep the problem by being easier to program by trial and error, but it's nothing more than emulating a different architecture and brute forcing its programming.
- ghgfhgfhn 10y agoBut you see the process of training is a solution of the problem "how to program AI". For a machine to be intelligent it has to form it's own model of the world. This model is necessarily somewhat opaque because it includes representations which the machine has created for itself. An intelligent machine that is interpretable in a straightforward manner is an impossibility, it is a contradiction in terms. You can teach the machine to answer questions about it's internal state in order to analyse it's function or you can apply your own mind to painstakingly deduce it's operation. But I contend you cannot have a generalised rubric for determining it's operation.
- nurettin 10y agoI think there is an economic threshold for the need to know something versus the effort to reverse engineer and find out exactly how that thing happened. (don't xkcd this, randall!)
- charlieflowers 10y agoI think that's exactly the point. Evolution produced our brains, and no one understands them yet. But evolution was able to produce them because it cares about success-testing, not understanding. I think we limit ourselves quite a bit in the mainstream software industry by insisting that everything be hand-crafted by humans and understandable. (Not to mention how short we fall of making software "understandable").
- TeMPOraL 10y agoBy "limiting" ourselves this way we can go forward much faster than evolution can. We create designs in hours what would take evolution millions of years to achieve. Also, it's a mistake to treat products of evolution as some kind of gold standard. It's literally worst design that could still work. We can do better, but we do it via different means. Another thing is that understanding is a key component of trust. Animals come to mind here. We can't understand their firmware per se; by observing them over thousands of years, both passively and actively (through domestication, training regimens, etc.), we've learned to expect some things from them but not others. So I can be sure that e.g. my cat won't kill me in my sleep. But I can absolutely not trust it to go somewhere when I want it to go there, or to stay in place when I want it to stay, or even to eat food when I give it. We want to write software that will bear much greater, more important responsibilities than any animal in history. So we need to make them more understandable than animals if we're to ever trust them with those responsibilities.
- charlieflowers 10y agoSure, I agree. Evolution did not take us from horse and buggy to planes and autonomous cars in 100-ish years. But when we push up against the limits of the problems we can solve with software, it's usually because it has outgrown our ability to understand it. That's exactly what is happening with this new burst of progress in machine learning. We're "teaching" the computer to do things, and we've let go of the ability to directly understand how the computer "represents its learnings."
- keithnoizu 10y agoI think there's an existing analogy here to service animals. You can train a sheep dog to manage the herd, you can train a seeing eye dog to stop it's owner at the lights. You can improve their training, etc. but you'll never really ever know why they sometimes don't follow the expected script. That isn't necessarily a bad thing. You get more done with them than you would with out them.
- phee 10y agoThe analogy is with anything with a brain, not just animals. Take a poorly trained human and put him in a decision-making position, he will probably make bad decisions. But no amount of training will guarantee you that he won't make any. The learning data set is just too big to be sure you won't have any bias.
- ninjakeyboard 10y ago"LIKE GOOD STUDENTS, COMPUTERS NEED TO BE ABLE TO SHOW THEIR WORKING" Is working used as a noun here or is that a typo? Just never heard that use of the word working. Like ya, you can see I'm working vs you can see my working.
- ajb 10y agoA noun. Normally used to refer to the intermediate steps produced by a student doing mathematics.
- PeCaN 10y agoI've never seen that before. Using a gerund as a noun seems very unconventional and incorrect almost by definition. Simply ‘work’ reads much better—is it really frequently used as ‘working’ in mathematics‽
- creamyhorror 10y agoHere in Singapore, "working" is the standard term used in schools to refer to the written steps in solving a maths problem (as ajb said). I guess this means it's simply BrE (which Singapore English is a cousin of). I know Americans use "work" for the same purpose instead. > Using a gerund as a noun seems very unconventional and incorrect almost by definition. In English, gerunds are nouns (by definition). In "his singing was terrible", singing is a noun. Furthermore, gerunds do often become used to refer to physical objects/results of actions: "writing", "reading" (e.g. readings for a class), "drawing", "booking" (of a ticket), "crossing" (on a road), "wiring", "etching", and others. Using "working" to refer to the written steps of working out a maths problem is just another example in that vein.
- PeCaN 10y agoYou're right, I got something backwards there! I suppose it's not that unnatural on second thought. Your examples are good, though the only ones that are moderately common in American English seem to be “drawing” and “crossing”.
- apathy 10y agoOr, overfitting strikes again. Except that here the overfit is buried in a bunch of layers. Still -- eventually, especially by reversing the models to generate instead of discriminate, it is possible to be more transparent about what's going on. I have to agree that the EU, despite normally being a bunch of martinets, is likely getting it right here. It's too easy to construct and overfitted DNN and have it produce bizarre biases that affect patients, payrolls, or patents. There needs to be an appropriate level of transparency baked in as a precondition for licensable IP, IMHO.