6 ms·

In my experience, domain practitioners who spend our days elbow-deep in tensors are well aware that what we call AI right now has nothing to do with AGI. That

In my experience, domain practitioners who spend our days elbow-deep in tensors are well aware that what we call AI right now has nothing to do with AGI. That knowledge does not seem to generalize well, though. Even technically literate people who aren't in the trenches seem to have frankly insane ideas about AGI. This phenomenon is not helped by armchair philosophy, the frothy chatter of the tech-adjunct world, or the naive idea that "well, transistors are easy, so brains should be too!" that is all too common in people who haven't stuffed electrodes into brains and then spent weeks figuring out what the hell could be going on in there.

adrianN 8y ago
You'd have a pretty hard time figuring out what a CPU does if all you could do is poking a small number of needles into it, cutting it into slices to study under a microscope and measuring which parts get hot while you use it.
- hprotagonist 8y ago
  https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo... true fact.
thwy12321 8y ago
The thing is that progress is so fast in some areas of ML, you dont really know when something could emerge. Who knows, maybe with enough training data and a deep enough network with the right supervision, something extraordinary pops out. Google google bert nlp. Google recently released an NLP algorithm that basically beats all predecessors, but the algorithm itself is extremely simple. Just lots of data and compute, no telling what will happen in ten years when the amount of data has exploded and GPUs are cheap.
- throwawaymath 8y ago
  This mentality is exactly what hprotagonist is talking about. All signs currently point to AGI being far away - at least several breakthroughs in currently unknown unknowns away, in fact. If the most you can say about the likelihood of something happening is that, "there's no telling what can happen in the next ten years!", then the only informed forecast is that it won't happen. We can likewise say we don't know if an extinction level event will happen due to a meteor (that we couldn't detect for some reason) hitting us in the next decade. Every time someone makes a sobering statement grounded in reality about artificial intelligence, another comment pops up which reduces to, "but who knows!" It doesn't mean anything.
- glup 8y ago
  bert provides incremental improvements on tasks that don't really stress-test contextualized world knowledge in linguistic tasks — contrast with Winograd schemas (SOTA: https://arxiv.org/pdf/1806.02847.pdf https://arxiv.org/pdf/1806.02847.pdf). Corpus data and compute don't magically solve everything.
  - p1esk 8y ago
    The paper you linked to shows how a simple DL brute force approach (large model trained on lots of data) "successfully discovers important features of the context that decide the correct answer, indicating a good grasp of commonsense knowledge." Which kinda goes contrary to your last sentence. Go figure.
    - glup 8y ago
      My point is that while the DL + large corpus approach is the best available, it's still way, way lower than human performance (Tables 4 and 5).
      - p1esk 8y ago
        Sure, but we are talking about how fast it's progressing. Given that NNs for NLP field pretty much started with Bengio's paper in 2003, I'd say it's accelerating at an amazing rate.
        glup 8y ago
        But NNs for language processing have been around much longer than that, e.g., Elman, 1990: https://crl.ucsd.edu/~elman/Papers/fsit.pdf https://crl.ucsd.edu/~elman/Papers/fsit.pdf.
- freeone3000 8y ago
  Yeah, Bert's great, but it has the Google Paper problem in that nobody else can reproduce the results because nobody else has the compute to train the dang thing. It also doesn't solve the fundamental issues - it scores a few percent higher on some standardized sets, but it gets wrong answers that are just as explainable as any other system - shrug-emoji. ‍️‍️
chriskanan 8y ago
A couple months ago, I was on an NPR show trying very hard to refocus the discussion on real near-term threats caused by AI: possible job loss as well as enabling authoritarianism, predictive policing, and quashing of dissent. They wanted to talk about AI coming to get us all. (Evil) AGI seems to be a more exciting topic.
- germinalphrase 8y ago
  "...enabling authoritarianism, predictive policing, and quashing of dissent" sounds adequately evil to me, but - you know - I get what you're saying. Can you recommend any good reading about these threats?
- tambourine_man 8y ago
  Do you have a link to that show? Seems like a good one to hear.
  - soohyung 8y ago
    I think this is the show he was referring to: http://www.wxxinews.org/post/connections-did-2001-space-odyssey-accurately-predict-ai-future http://www.wxxinews.org/post/connections-did-2001-space-odys...

adrianN 8y ago

You'd have a pretty hard time figuring out what a CPU does if all you could do is poking a small number of needles into it, cutting it into slices to study under a microscope and measuring which parts get hot while you use it.

hprotagonist 8y ago
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo... true fact.

thwy12321 8y ago

The thing is that progress is so fast in some areas of ML, you dont really know when something could emerge. Who knows, maybe with enough training data and a deep enough network with the right supervision, something extraordinary pops out. Google google bert nlp. Google recently released an NLP algorithm that basically beats all predecessors, but the algorithm itself is extremely simple. Just lots of data and compute, no telling what will happen in ten years when the amount of data has exploded and GPUs are cheap.

throwawaymath 8y ago
This mentality is exactly what hprotagonist is talking about. All signs currently point to AGI being far away - at least several breakthroughs in currently unknown unknowns away, in fact. If the most you can say about the likelihood of something happening is that, "there's no telling what can happen in the next ten years!", then the only informed forecast is that it won't happen. We can likewise say we don't know if an extinction level event will happen due to a meteor (that we couldn't detect for some reason) hitting us in the next decade. Every time someone makes a sobering statement grounded in reality about artificial intelligence, another comment pops up which reduces to, "but who knows!" It doesn't mean anything.
glup 8y ago
bert provides incremental improvements on tasks that don't really stress-test contextualized world knowledge in linguistic tasks — contrast with Winograd schemas (SOTA: https://arxiv.org/pdf/1806.02847.pdf https://arxiv.org/pdf/1806.02847.pdf). Corpus data and compute don't magically solve everything.
- p1esk 8y ago
  The paper you linked to shows how a simple DL brute force approach (large model trained on lots of data) "successfully discovers important features of the context that decide the correct answer, indicating a good grasp of commonsense knowledge." Which kinda goes contrary to your last sentence. Go figure.
  - glup 8y ago
    My point is that while the DL + large corpus approach is the best available, it's still way, way lower than human performance (Tables 4 and 5).
    - p1esk 8y ago
      Sure, but we are talking about how fast it's progressing. Given that NNs for NLP field pretty much started with Bengio's paper in 2003, I'd say it's accelerating at an amazing rate.
      - glup 8y ago
        But NNs for language processing have been around much longer than that, e.g., Elman, 1990: https://crl.ucsd.edu/~elman/Papers/fsit.pdf https://crl.ucsd.edu/~elman/Papers/fsit.pdf.
freeone3000 8y ago
Yeah, Bert's great, but it has the Google Paper problem in that nobody else can reproduce the results because nobody else has the compute to train the dang thing. It also doesn't solve the fundamental issues - it scores a few percent higher on some standardized sets, but it gets wrong answers that are just as explainable as any other system - shrug-emoji. ‍️‍️

chriskanan 8y ago

A couple months ago, I was on an NPR show trying very hard to refocus the discussion on real near-term threats caused by AI: possible job loss as well as enabling authoritarianism, predictive policing, and quashing of dissent. They wanted to talk about AI coming to get us all. (Evil) AGI seems to be a more exciting topic.

germinalphrase 8y ago
"...enabling authoritarianism, predictive policing, and quashing of dissent" sounds adequately evil to me, but - you know - I get what you're saying. Can you recommend any good reading about these threats?
tambourine_man 8y ago
Do you have a link to that show? Seems like a good one to hear.
- soohyung 8y ago
  I think this is the show he was referring to: http://www.wxxinews.org/post/connections-did-2001-space-odyssey-accurately-predict-ai-future http://www.wxxinews.org/post/connections-did-2001-space-odys...