6 ms·
The "new-fangled" AI, as the article calls it, is often useful when the stakes are low, and you can accept mistakes in outcomes. Examples of such applications a
by triska 4y ago
The "new-fangled" AI, as the article calls it, is often useful when the stakes are low, and you can accept mistakes in outcomes. Examples of such applications are: trying to determine which of your friends occur in a photo, which movies a subscriber may be interested in, or which action could lead to victory in a computer game. Getting a rough translation of a newspaper entry, as mentioned in the article, is also a good example.
As soon as you need reliable outcomes, such as certainty whether an erroneous state can arise in a program, whether a proof for a mathematical conjecture exists, or whether a counterexample exists, exhaustive search is often necessary.
The question then soon becomes: How can we best delegate this search to a computer, in such a way that we can focus on a clear description of the relations that hold between the concepts we are reasoning about? Which symbolic languages let us best describe the situation so that we can reliably reason about it? How can we be certain that the computed result is itself correct?
The article states: "The heart of GOFAI is searching – of trees and, more generally, graphs." I think one could with the same conviction state: "The heart of GOFAI is reasoning – about relations and, more generally, programs."
- blueyes 4y agoGOFAI was never more than a rules engine. If-then statements. Agree with you about probabilistic AI being useful in low-stakes situations, at least at first.
- mistrial9 4y agoIf-then questions can lead to non-deterministic outputs with some simple feedback systems not disagreeing completely, but.. both "questions that are reasoned about", and "the code that reasons about questions".. need more careful classification in order to make use of these new data methods.. personally, I see the hype on DeepLearning to solve "find pattern in varying digital content" that is so clearly useful to the FAANG content Feudal Lords, is engaging in an investor shouting match that paves over simple use cases where DeepLearning is really not appropriate.
- thwayunion 4y ago> GOFAI was never more than a rules engine. GOFAI "aged out" of the AI label and became: compilers, databases, search algorithms, planning algorithms, programming languages, theorem proving, a million things that are still commonly used in NLP/CV/robotics, comp arch, etc. Aka, most of what makes computers actually useful. If something's over 30 years old and is still called AI, that's just shorthand for saying it's a failed idea that in the best case hasn't had its moment yet. > If-then statements. 99.999% of the software I use is just if-then statements. (Also, this is like saying that deep learning is Linear Algebra.)
- cscurmudgeon 4y agoSorry, thats really wrong. I wouldn't call theorem provers just "if-then" statements. By that logic, everything, even large models, are if-then statements.
- bheadmaster 4y agoTechnically, they are. There was a post on HN recently about how every model is equivalent to a decision tree.
- dragonwriter 4y agoI mean, if it can be specified by a computational process, that’s trivially true, right?
- bheadmaster 4y agoI'm not so sure about that. Decision trees are finite set of if-else branches, and work on finite input - much like models. I assume many of the classic arbitrary-size-input algorithms cannot be represented by a decision tree. Adding two arbitrary-sized numbers comes to mind - how would one model it as a decision tree? if (a[0] + b[0] == 0) { c[0] = 0 } else if (a[0] + b[0] == 1) { c[0] = 1 } else if ... ... } else if (a[0] + b[0] == 10) { c[0] = 0 if (a[1] + b[1] == 0) { c[1] = 1 } else ... ... } ... You can see that for arbitrary-sized input, this decision tree would have grow infinitely - which is contradictory to the finite nature of decision trees. I can't think of a proof that such a computation cannot be represented by a decision tree, and it's possible that my definitions aren't quite correct. But intuitively, I think this is the way it works. Of course, if I'm wrong, any explanation on why and how would be very welcome.
- thwayunion 4y ago> As soon as you need reliable outcomes, such as certainty whether an erroneous state can arise in a program, whether a proof for a mathematical conjecture exists, or whether a counterexample exists, exhaustive search is often necessary. Checking proofs is easier than finding proofs. > The question then soon becomes: How can we best delegate this search to a computer, in such a way that we can focus on a clear description of the relations that hold between the concepts we are reasoning about? Which symbolic languages let us best describe the situation so that we can reliably reason about it? These questions are largely answered. Or, at least, the methodology for investigating these types of questions is well-developed. I think the more interesting question is co-design. What do languages and logics look like when they are designed for incorporation into new-fangled AI systems (perhaps also with a human), instead of for purely manual use?
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- bade 4y agoWell said
- falcolas 4y agoSo, there's an area of research that's under way called "AI Assurance" which seeks to answer many of these questions. Some things they're attempting: - Creating explainable outcomes by tracing the inner works of ML models. - Looking for biases in models using random inputs & looking for biased outputs. - Using training sets with differently weighted models to find attacks and biases. etc.
- nonrandomstring 4y agoThe tragedy is that GOFAI did all these things as built-ins. Procedural expert systems have been doing introspection, backtracing, declaring confidence intervals etc since the 1960s. Layering "assurance" on top of inherently jittery statistical/stochastic and neural systems seems to misunderstand how these models evolved, where they come from and why there are alternatives.
- ShamelessC 4y agoAnother tragedy, GOFAI hasn’t made a dent on any of these problems for a very long time.
- sdenton4 4y agoWhile horses don't consume fossil fuels, they also don't solve the problem of transoceanic flight. Yet for some reason every discussion of airplane design ends up dominated by a vocal contingent of buggy whip salesmen.
- wizzwizz4 4y agoTurns out, we don't need transoceanic flight; we can put horses on stirrups, and have an elegant system of clockwork to allow them to power a boat. https://www.newyorker.com/tech/annals-of-technology/the-pastry-ai-that-learned-to-fight-cancer https://www.newyorker.com/tech/annals-of-technology/the-past...
- ativzzz 4y ago
- shafoshaf 4y agoThe only barrier for higher stakes applications is going to be the frequency of errors. Flying an airplane or running a factory has a lot less margin for error, but humans don't do those things perfectly either (Chernobyl, Three Mile Island, Union Carbide-Bhopal disaster). It doesn't have to be perfect, just better than humans. And in fact, I'd argue that by having no deterministic outcomes prevents systemic failure, like having a single point of failure for all those drones in the crappy episodes of Star Wars.
- nradov 4y agoBut there's the rub. It's impossible to determine through testing whether a particular AI system will actually have a lower frequency of errors than humans. You can program an AI system to handle certain failure modes and test for those in simulation. But complex systems tend to have hidden failure modes which no one ever anticipated, so by definition it's impossible to test how the AI will handle those. Whereas an experienced human can often determine the correct course of action based on first principles. For example, see US Airways Flight 1549. Airbus had never tested a double engine failure in those exact circumstances so the flight crew disregarded some steps in the written checklist and improvised a new procedure. Would an AI have handled the emergency as well? Doubtful.
- joe_the_user 4y agoThe only barrier for higher stakes applications is going to be the frequency of errors. IE, "The only barrier to the software working perfectly is it's tendency to fail". Which is to say this sort of argument effectively assumes, without proof, that are no structural barriers to improving neural network performance in the real world. The thing is, the slow progress on self-driving cars shows that reducing the "frequency of errors" can turn from a simple exercise in optimizing and pumping in more data to a decades long debug process.
- MonkeyMalarky 4y agoThere's a weird bit of induced demand like widening a freeway, makes errors less often than a human but is more scalable so the absolute number of errors increases. I guess in the case of self driving cars, it could be from hordes of autonomous shipping trucks that outnumber existing truck drivers.
- skissane 4y ago> As soon as you need reliable outcomes, such as certainty whether an erroneous state can arise in a program, whether a proof for a mathematical conjecture exists, or whether a counterexample exists, exhaustive search is often necessary. Proof checking requires 100% reliability. But if you are searching the space of all possible proofs for a valid one, that process does not require 100% reliability. On the contrary, automated theorem provers rely on heuristics to guide their exploration of that space, none of which work 100% of the time. "Exhaustive search" is an infeasible strategy, because the search space is just too large. Finding proofs is the really hard part (NP-hard), and the part which most stands to benefit from "AI" techniques – checking their validity is a lot easier (polynomial time). "New AI" deep-learning techniques can be used to augment automated theorem provers, by giving them guidance on which areas of the search space to target – see for example https://arxiv.org/abs/1701.06972 https://arxiv.org/abs/1701.06972 – that produced a seemingly modest improvement (3 percentage points) – but keep in mind how hard the problem is, a 3 percentage point improvement on a very hard problem can actually be a big deal – plus I don't know if any more recent research has improved on that.
- _carbyau_ 4y agoThe idea of chainlinking "AI to guide/choose AI for the next step" is where I expect more impressive results in future. It will be important to understand the limitations of AI to be sure of proper placement.
- versteegen 4y agoFinding a proof of a mathematical conjecture is much harder than NP-hard, it's undecidable (the Entscheidungsproblem) because the search space is infinite (and worse, doesn't necessarily include a proof even if the statement is true - Gödel's first incompleteness thm). So even if we restrict to conjectures that are actually machine-provable, it's still much harder than NP decision problems which are combinatorial so have an exponentially growing search space.
- ogogmad 4y agoRemark: Deciding if a proof of a proposition exists* up to a certain length n is NP-COMPLETE. Deciding if a proof exists at all - which is the Entscheidungsproblem - is semi-decidable. So the parent wasn't entirely wrong. * - In Peano arithmetic, let's say
- pj_mukh 4y agoI used to believe exactly this, and I think this is mostly correct. There are exceptions though. The one that made me reconsider this proposition was Andrew Ng's early work. "New Fangled" AI to precisely control a helicopter! [1] Can't really accept mistakes or imprecision in the outcome. A lot of it was RL work [2] [1]: https://www.youtube.com/watch?v=M-QUkgk3HyE&ab_channel=Stanford https://www.youtube.com/watch?v=M-QUkgk3HyE&ab_channel=Stanf... [2]: https://papers.nips.cc/paper/2003/hash/b427426b8acd2c2e53827970f2c2f526-Abstract.html https://papers.nips.cc/paper/2003/hash/b427426b8acd2c2e53827...
- pharmakom 4y agoI agree with your broad point however I think the proof example is off. Finding a proof is the hard part. Verifying a proof is the easier part. So the AI could search for likely proofs and we can use other mechanisms to reliably verify them. This doesn’t require high accuracy in the search stage to be reliable overall.