26 ms·

Predicting where AI is going in 2020

122 points by alfozan 7y ago

sillysaurusx 7y ago
Re: PyTorch TPU support, has anyone checked it out beyond "it works"? There are many aspects of TPUs that I'm not convinced are easy to port: Colocating gradient ops, scoping operation to specific TPU cores, choosing to run operations in a mode that can use all available TPU memory (which is up to 300GB in some cases), and so on. These aren't small features. If you don't have them, you don't get TPU speed. The reason TPUs are fast are because of those features. I only glanced at PyTorch TPU support, but it seemed like there wasn't a straightforward way to do most of these. If you happen to know how, it would be immensely helpful! As far as predictions go, AI will probably take the form of "infinite remixing." AI voice will become very important, and will begin proliferating through several facets of daily life. One obvious application is to apply the "abridged" formula to old sitcoms. (An "abridged" show is when you rewrite it using editing and new dialog, e.g. https://www.youtube.com/watch?v=2nYozPLpJRE https://www.youtube.com/watch?v=2nYozPLpJRE. Someone should do Abridged Seinfeld.) AI audio as already made inroads on Twitch, where streamers like Forsen allow donation messages to be read off in the voice of various political figures (and even his own voice). The Pony Preservation Project was recently solved with AI voice (https://twitter.com/gwern/status/1203876674531667969 https://twitter.com/gwern/status/1203876674531667969) meaning it's possible to do realistic voice simulations of all the MLP characters with precise control over intonation and aesthetics. Natural language AI will continue to ramp up, and people will learn how to apply it to increasingly complex situations. For example, AI dungeon is probably just the beginning. I recently tried to do GPT-2 chess (https://twitter.com/theshawwn/status/1212272510470959105 https://twitter.com/theshawwn/status/1212272510470959105) and found that it can in fact play a decent game up to move 12 or so. AI dungeon multiplayer is coming soon, and it seems like applying natural language AI to videogames in general is going to be rather big. Customer support will also take the form of AI, moreso than it is already. It turns out that GPT-2 1.5B was pretty knowledgable about NordVPN. (Warning: NSFW ending, illustrating some of the problems we still need to iron out before we can deploy this at scale.) https://gist.github.com/shawwn/8a3a088c7546c7a2948e369aee876902 https://gist.github.com/shawwn/8a3a088c7546c7a2948e369aee876... AI will infiltrate the gamedev industry slowly but surely. Facial animation will become increasingly GAN-based, because the results are so clearly superior that there's almost no way traditional toolsets will be able to compete. You'll probably be able to create your own persona in videogames sooner than later. With a snippet of your voice and a few selfies, you'll be able to create a fairly realistic representation of yourself as the main hero of e.g. a Final Fantasy 7 type game.
- nl 7y ago
  I did try training a Pytorch BERT-derived model on TPUs (on Colab) and it didn't work out-of-the box. (Where didn't work=was using the CPU) I didn't dig into it to find out why.
  - zak 7y ago
    (I'm one of the Cloud TPU product leads) We've seen multiple BERT-related PyTorch models training successfully on Cloud TPUs, including training at scale on large, distributed Cloud TPU Pod slices. Would you consider filing a GitHub issue at https://github.com/pytorch/xla https://github.com/pytorch/xla or emailing pytorch-tpu@googlegroups.com to provide a bit more context about the specific issue you encountered? Here's the current PyTorch/TPU troubleshooting guide, which provides information on how to collect and interpret metrics that are very helpful for debugging: https://github.com/pytorch/xla/blob/master/TROUBLESHOOTING.md https://github.com/pytorch/xla/blob/master/TROUBLESHOOTING.m... Thanks!
    - riku_iki 7y ago
      > BERT-related PyTorch models training successfully on Cloud TPUs How do you see it? Do you look at your client's code?
      - nl 7y ago
        Google wrote BERT and they provide technical support to the FB Pytorch TPU port so it's not entirely surprising. RoBERTa, (Fb's variant) would be a good candidate to test it with.
        zak 7y ago
        We only see code when customers open-source it or otherwise explicitly share it with us. We are directly in touch with several customers who are using the PyTorch / TPU integration, so we hear feedback from them, and we also run a variety of open-source PyTorch models on Cloud TPUs ourselves as we continue to improve the integration.
corporateslave5 7y ago
Natural language processing is going to absolutely decimate content on the internet, forcing everyone into walled gardens
- etaioinshrdlu 7y ago
  Via seo spam and other types of spam that are indistinguishable from real content?
  - antupis 7y ago
    Probably more likely hitting that annoying point that you cannot be quite sure is article machine-generated lorem ipsum which sounds convincing but does not have any real information behind it. something like http://news.mit.edu/2015/how-three-mit-students-fooled-scientific-journals-0414 http://news.mit.edu/2015/how-three-mit-students-fooled-scien... but with scale.
- PeterStuer 7y ago
  Unless walled gardens bring in mechanisms that bring a real cost to the table for creating content, such as limitations on posts and account creation, they would be in the same boat. In either system users can go to a strict whitelisting approach but that would com at a cost of discoverability and serendipity. This would strengthen the position of anointed 'influencers' and curators, and diminish the value of algorithmic feeds so eating into the revenue model of those that rely on this. Since those forces are counteracting it is hard to make predictions, but I will anyway but take it with a strong dose of uncertainty. My dystopian take is the ubiquitous deployment of 'fake' actors will further undermine the general inter-human levels of trust. Evolutionary less fragile altruistic strategies rely on unfakable or at least costly signals to stand up to the 'free-rider' intrusion. Sadly undermining trust will accelerate the further descend into identity tribalism we are already witnessing today, a segregation into near-immutable trait based groups where cross-clan transgressions are punished with extreme measures. As a caricature think of the television portrayal of tribal gang cultures in maximum security prisons. My utopian take is that the onslaught of fake noise will restore the reliance on offline contacts and connections. Due to physical proximity these offline groups have to share more of the negative externalities caused by their actions which could lead to more altruistic consideration in consumption and production descisioning. Direct verifyable contribution and impact might trump systemic distrust and lead us out of the current innate identity tribal descend.
jeffshek 7y ago
I love PyTorch, but I’m not confident the claim that it is the most popular is close to true. The cited link, which brings up a lot of new research is in PyTorch simply doesn’t account for the amount of TensorFlow in production. Sure, a lot of academics may be embracing PyTorch, but almost all production models have been in TensorFlow. Tesla is a huge notable example that’s using PyTorch at scale. I do suspect that the split of TensorFlow 1 and 2 is perhaps one of the worst times for TF 2, many teams will likely try out PyTorch instead. I think both are amazing frameworks, however TF was designed for Google Scale .... which leads to a lot of difficulties since 99.9 are not at Google scale.
- rckoepke 7y ago
  Depends on how you measure it, of course. However, stackoverflow survey, google trends, and github octoverse all show PyTorch is on a steep upward trajectory that recently reached effective parity with TensorFlow and has not yet started slowing down.
TACIXAT 7y ago
>Human babies don’t get tagged data sets, yet they manage just fine, and it’s important for us to understand how that happens I do not really understand this. Human babies get a constant stream of labeled information from their parents. Contextualized speech is being fed to them for years. Toddlers repeat everything you say. Is this referring to something else that babies can do?
- ben_w 7y ago
  The things you’re calling labels are themselves learned data, as we don’t start off with an innate knowledge of language. Whether that distinction is important or not, I don’t know enough brain science to guess.
- onefuncman 7y ago
  I agree with you -- we have little idea how much labeled information is encoded in the genome either. I rather like to think of the genome as the most well trained model of life we have access to.
- richardw 7y ago
  I show my 19 month old daughter like three cartoon drawings of owls and she recognises a live one at the bird park instantly, unprompted. We have a way to go.
  - taneq 7y ago
    I believe cartoons are our equivalent of adversarial images. They typically look nothing like (photos of) their namesake and yet we recognise them usually without prompting.
    - EliRivers 7y ago
      It is my understanding (although I sure don't have any evidence on me) that cartoons and such (at least, the ones where we haven't simply learned that this cartoon means this animal) work by being a picture of what we remember about an animal. Akin to a caricature; the cartoon contains the most salient features. It doesn't work by looking like the actual animal; it works by reacting with how we remember the animal.
jansbor 7y ago
Is it just me that are wondering why they did not use AI to predict where AI is going in 2020?
- deleted 7y ago
  [deleted]
- TheOtherHobbes 7y ago
  They did, but they didn't understand the results.
mark_l_watson 7y ago
The linked page threw up a suspicious looking overlay. I left the site, too bad since I wrote a blog with my predictions last night and wanted to compare my AI predictions.
- CodeGlitch 7y ago
  If you use Firefox, just enable "Reader View" for the site (just hit F9). Removes all the crud around the page and just shows the text and required images.
- reportgunner 7y ago
  https://outline.com/yf9ACE https://outline.com/yf9ACE
graycat 7y ago
I believe we will (1) find some basic data structures and algorithms to do real AI. (2) At first it will be able to do I/O only via text or simple voice. (3) Due to (1) it will learn very quickly from humans or other sources. (4) Soon it will be genuinely smart, enough, say, to discover and prove new theorems in math, to understand physics and propose new research directions, to understand drama and write good screen plays, to understand various styles and cases of music and compose for those, etc. Broadly from (1) with the data structures it will be able to represent and store data and, then, from the algorithms, manipulate that data generating more data to be stored, etc. In particular it will be able to do well with thought experiments and generation and evaluation of scenarios. Good image understanding will come later but only a little later; the ideas in (1) may have to be revised to do well on image understanding.
- tsimionescu 7y ago
  You believe that we will achieve (1) in 2020? Or do you believe that we will achieve this at some point in general?
  - graycat 7y ago
    Sorry, from reading another post about predictions for the next decade, I was thinking by 2030, not just 2020! Besides, for AI, just 2020 seems a bit too short!
- The_rationalist 7y ago
  * I believe we will (1) find some basic data structures and algorithms to do real AI.* There's no new kind of data structure to discover, humanity has made a disjonction of all possibles. The choice of data structure for semantic parsing is trivial, it's an hypergraph. The debate isn't the datastructure but how to fill it correctly while keeping the same Expressivity as in the original input (natural language). There's no reason to think we will make progress on this task beyond wishful thinking. Only a handful of humans are working on semantic parsing, which is the real AI task.
  - 7y ago
dijksterhuis 7y ago
The page forwards itself to a spam google survey for me? Page history fills with the spam survey and cant navigate back to the article. iOS safari with reader view enabled.
- kzzzznot 7y ago
  I experienced the same behaviour
bitL 7y ago
- individual GPUs will hit a plateau at around 25TFlops in FP32 due to Moore's law and thermal dissipation however it will be easier than ever to interconnect multiple GPUs into large virtual ones due to interconnect tech improvements and modularization of GPU processing units - only large companies will be able to train and use SOTA models with training costs in $10M-$100M per training run and those models will hit law of diminishing returns quickly - 50% of all white collar jobs will be automated away, including a significant chunk of CRUD software work. Increased productivity won't be shared back with society, instead two distinct wealth strata of society will be formed worldwide due to scale effects, like in Latin America (<1% owners, >99% fighting for their lives). - AI will make marketing, ads and behavioral programming much more intrusive and practically unavoidable
- mkl 7y ago
  I think you're talking about a much longer period than just 2020. The job prediction seems unlikely to happen within the next decade, even.
- petra 7y ago
  >> including a significant chunk of CRUD software work. Can you please expand on how that will look like ?
- wnkrshm 7y ago
  So the Butlerian Jihad [0] for the 2040s? [0] https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian_Jihad https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian...
DagAgren 7y ago
Away. (Or less snarkily, https://twitter.com/iquilezles/status/1212377355349417986 https://twitter.com/iquilezles/status/1212377355349417986)
- 2sk21 7y ago
  "Derp learning" :-)
szc 7y ago
Honesty, repeatability, numerical analysis. Canonicalization. Honesty: how many times was the exact same data processed? Was the result cherry picked and the best one published? For the sake of integrity how is it possible to scientifically improve on this result? (example, your AI outputs some life altering decision?) Repeatability: In science, if a result can be independently verified, it gives validity to the "conclusion" or result. Most AI results cannot be independently verified. Not indepentently verifiable really ought to give the "science" the same status as an 1800's "Dr. Bloobar's miracule AI cure". Numerical Analysis: performing billions / trillions of computations on bit-restricted numerical values will introduce a lot of forward propagated errors (noise). What does that do? Commentary: Video cards don't care if a few bits of your 15 million pixel display are off by a few LSB bits, they do that 60 or 120 frames a second and you don't notice. It is an integral part of their design. The issue is, how does this impact AI models? This affects repeatability -> honesty. If error of a quantized size is a necessarily required property to achieve "AI learning that converges", there is still an opportunity for canonicalization -- a way to map "different" converged models to explain why they are effectively the "same". This does not seem to be a "thing", why not? In my opinion, in 2020, the AI emperor still has no clothes.
- fluffything 7y ago
  For most of the engineering applications I work on, AI is useless. When we talk about controlling machines, our control algorithms have mathematically proven strict error bounds, such that if we provide an input with a particular maximal error (e.g. from a sensor that has some error tolerance), we can calculate what's the maximum error possible in the response that our model would produce, and then use that to evaluate whether this is even an input that should be handled by the current algorithm or not. These control algorithms all take some inputs, and use them to "predict" what will happen, and using that prediction, compute some response to correct it. These predictions need to happen much faster than real time, since you often need to perform an optimization step to compute an "optimal" response. These predictions are usually computed using a reduced-order model, e.g., if you had to solve a PDE over 10^9 unknowns to compute the actual prediction, you can instead reduce that to a system with 10 unknowns, by doing some pre-computation a priori. Most tools to do these kinds of reductions developed in the last 60 years come with tight error bounds, that tell you, depending on your inputs, the training data, etc. what's the largest error than the prediction can have, so you can just plug these in into your control pipeline. People have been plugin in neural-networks to control robots, cars, and pretty much anything you can imagine into these pipelines for 10 years, yet nobody knows what the upper bound on the errors that these neural-networks give for a particular input, training set, etc. Until that changes, machine learning just makes your whole pipeline unreliable, and e.g. a car manufacturer must tell you that in "autonomous driving" mode you are liable for everything your car does, and not them, so you have to keep your hands on the driving wheels and pay attention at all times, which... kind of defeats the point of autonomous driving. --- Prediction: we won't have any tight error bounds for real-world neural networks in the 2020-2030 time frame. These are all non-linear by design (that's why they are good), error bounds for simple non-linear interpolants are pretty much non-existent, people have tried for 20-30 years, and real-world NNs are anything but simple.
juskrey 7y ago
Fat tails
- bordercases 7y ago
  This is the content I come to Hacker News for.
y1tan 7y ago
I predict the broader ML/DL community will keep pumping out iterative papers that push the ball just a little bit forward while maintaining job security : Gatekeeping, no one thinking outside of the box, benchmark putting, just enough for the appearance of progress, and nothing broadly innovative or disruptive. The applications of ML/DL will continue to be gimmicky consumer products that have questionable valuable, questionable profit potential, add even more to disinformation/misinformation, produce more informational noise, only serve to rebuff a big corp's cloud offerings, and waste people's time. I predict tons more 'bought' articles that hype up AI technology for the typical 'household' names. I predict the same ol' echo chamber of thought and reinforcement of 'gatekept' ideology. I expect a number of more prominent articles critiquing the shortfalls of the technology. I expect a number of young minds steeped in DL/ML coming to the realization that it's not what they expected... That its a big profit/revenue story for Universities and established corporate platforms. I expect a number of them to realize ML/DL is truly not "AI" or anything close to it. That they aren't doing cutting edge research and that they are not allowed to think outside of the echo chamber of 'approved' approaches. I predict more useless chatbots that utter unpredictable word salads. I expect more gimmicky entertainment focused uses of it. I expect more assistants being adopted for data collection. I expect more people who aren't busy or doing anything important, using assistance assistants and text-to-speech to speed up their tasks so they can waste more of their time on social media/youtube/entertainment. Samsung Neon is coming out in some days.. making use of that 'Viv' acquisition. I expect more feverish attempts at attacking low hanging fruit jobs with overly complex solutions. I predict failures in a number of startups targeting this. I predict no pronounced progress in self-driving cars nor any particular grand use for them. I predict several hollow attempts to overlay symbolic systems over ML/DL or integration attempts of it with ML/DL from prominent AI figures. I predict pronounced failures in this effort cementing a partial end to the hype of ML/DL. I predict we will get a pronounced development outside of run-of-the-mill corporate/academic gatekept/walled garden ML/DL that will forge a new and higher path for AI. Hinton's words from prior years will have been heeded and the results of a new approach to AI presented. A change of guard, a break from the necessity of a PhD, a break from the echo-chamber of names, and a broader and more deeply thought out vision. Disruption not of low-hanging-fruit but disruption directed at the heart of the AI/Technology industry... So that we may finally progress from this stalled out disinformation/misinformation/hype/gatekeeping/cloud/all-your data-belongs-to-us cycle. It's 2020 after-all, time for a new age.
zackmorris 7y ago
Some axioms that I'm not seeing talked about much: * Artificial general intelligence (AGI) is the last problem in computer science, so it should be at least somewhat alarming that it's being funded by internet companies, wall street and the military instead of, say, universities/nonprofits/nonmilitary branches of the government. * Machine learning is conceptually simple enough that most software developers could work on it (my feeling is that the final formula for consciousness will fit on a napkin), but they never will, because of endlessly having to reinvent the wheel to make rent - eventually missing the boat and getting automated out of a job. * AI and robot labor will create unemployment and underemployment chaos if we don't implement universal basic income (UBI) or at the very least, reform the tax system so that automation provides for the public good instead of the lion's share of the profit going to a handful of wealthy financiers. * Children aren't usually exposed to financial responsibility until around the age of 15 or so, so training machine learning for financial use is likely to result in at least some degree of sociopathy, wealth inequality and further entrenchment of the status quo (what we would consider misaligned ethics). * Humans may not react well when it's discovered that self-awareness is emotion, and that as computers approach sentience they begin to act more like humans trapped in boxes, and that all of this is happening before the world can even provide justice and equality for the "other" (women, minorities, immigrants, oppressed creeds, intersexed people, the impoverished, etc etc etc). My prediction for 2020: nothing. But for 2025: an optimal game-winning strategy is taught in universities. By 2030: the optimal game-winning strategy is combined with experience from quantum computing to create an optimal search space strategy using exponentially fewer resources than anything today (forming the first limited AGI). By 2035: AGI is found to require some number of execution cycles to evolve, perhaps costing $1 trillion. By 2040: cost to evolve AGI drops to $10 billion and most governments and wealthy financiers own what we would consider a sentient agent. By 2045: AGI is everywhere and humanity is addicted to having any question answered by the AGI oracle so progress in human-machine merging, immortality and all other problems are predicted to be solved within 5 years. By 2050: all human problems have either been enumerated or solved and attention turns to nonhuman motives that can't be predicted (the singularity).
drongoking 7y ago
I'm generally pessimistic about predictions of the future. In this case I can't help but smile. They're trying to predict how a field (AI), which deals with complex adaptation, will intelligently adapt its adaptive techniques in the coming year, within an environment (we humans) that are themselves changing behavior while adapting to AI. That's approximately three meta levels. Good luck, guys!

122 points by alfozan 7y ago

Re: PyTorch TPU support, has anyone checked it out beyond "it works"? There are many aspects of TPUs that I'm not convinced are easy to port: Colocating gradient ops, scoping operation to specific TPU cores, choosing to run operations in a mode that can use all available TPU memory (which is up to 300GB in some cases), and so on. These aren't small features. If you don't have them, you don't get TPU speed. The reason TPUs are fast are because of those features. I only glanced at PyTorch TPU support, but it seemed like there wasn't a straightforward way to do most of these. If you happen to know how, it would be immensely helpful! As far as predictions go, AI will probably take the form of "infinite remixing." AI voice will become very important, and will begin proliferating through several facets of daily life. One obvious application is to apply the "abridged" formula to old sitcoms. (An "abridged" show is when you rewrite it using editing and new dialog, e.g. https://www.youtube.com/watch?v=2nYozPLpJRE https://www.youtube.com/watch?v=2nYozPLpJRE. Someone should do Abridged Seinfeld.) AI audio as already made inroads on Twitch, where streamers like Forsen allow donation messages to be read off in the voice of various political figures (and even his own voice). The Pony Preservation Project was recently solved with AI voice (https://twitter.com/gwern/status/1203876674531667969 https://twitter.com/gwern/status/1203876674531667969) meaning it's possible to do realistic voice simulations of all the MLP characters with precise control over intonation and aesthetics. Natural language AI will continue to ramp up, and people will learn how to apply it to increasingly complex situations. For example, AI dungeon is probably just the beginning. I recently tried to do GPT-2 chess (https://twitter.com/theshawwn/status/1212272510470959105 https://twitter.com/theshawwn/status/1212272510470959105) and found that it can in fact play a decent game up to move 12 or so. AI dungeon multiplayer is coming soon, and it seems like applying natural language AI to videogames in general is going to be rather big. Customer support will also take the form of AI, moreso than it is already. It turns out that GPT-2 1.5B was pretty knowledgable about NordVPN. (Warning: NSFW ending, illustrating some of the problems we still need to iron out before we can deploy this at scale.) https://gist.github.com/shawwn/8a3a088c7546c7a2948e369aee876902 https://gist.github.com/shawwn/8a3a088c7546c7a2948e369aee876... AI will infiltrate the gamedev industry slowly but surely. Facial animation will become increasingly GAN-based, because the results are so clearly superior that there's almost no way traditional toolsets will be able to compete. You'll probably be able to create your own persona in videogames sooner than later. With a snippet of your voice and a few selfies, you'll be able to create a fairly realistic representation of yourself as the main hero of e.g. a Final Fantasy 7 type game.

nl 7y ago
I did try training a Pytorch BERT-derived model on TPUs (on Colab) and it didn't work out-of-the box. (Where didn't work=was using the CPU) I didn't dig into it to find out why.
- zak 7y ago
  (I'm one of the Cloud TPU product leads) We've seen multiple BERT-related PyTorch models training successfully on Cloud TPUs, including training at scale on large, distributed Cloud TPU Pod slices. Would you consider filing a GitHub issue at https://github.com/pytorch/xla https://github.com/pytorch/xla or emailing pytorch-tpu@googlegroups.com to provide a bit more context about the specific issue you encountered? Here's the current PyTorch/TPU troubleshooting guide, which provides information on how to collect and interpret metrics that are very helpful for debugging: https://github.com/pytorch/xla/blob/master/TROUBLESHOOTING.md https://github.com/pytorch/xla/blob/master/TROUBLESHOOTING.m... Thanks!
  - riku_iki 7y ago
    > BERT-related PyTorch models training successfully on Cloud TPUs How do you see it? Do you look at your client's code?
    - nl 7y ago
      Google wrote BERT and they provide technical support to the FB Pytorch TPU port so it's not entirely surprising. RoBERTa, (Fb's variant) would be a good candidate to test it with.
      - zak 7y ago
        We only see code when customers open-source it or otherwise explicitly share it with us. We are directly in touch with several customers who are using the PyTorch / TPU integration, so we hear feedback from them, and we also run a variety of open-source PyTorch models on Cloud TPUs ourselves as we continue to improve the integration.

corporateslave5 7y ago

Natural language processing is going to absolutely decimate content on the internet, forcing everyone into walled gardens

etaioinshrdlu 7y ago
Via seo spam and other types of spam that are indistinguishable from real content?
- antupis 7y ago
  Probably more likely hitting that annoying point that you cannot be quite sure is article machine-generated lorem ipsum which sounds convincing but does not have any real information behind it. something like http://news.mit.edu/2015/how-three-mit-students-fooled-scientific-journals-0414 http://news.mit.edu/2015/how-three-mit-students-fooled-scien... but with scale.
PeterStuer 7y ago
Unless walled gardens bring in mechanisms that bring a real cost to the table for creating content, such as limitations on posts and account creation, they would be in the same boat. In either system users can go to a strict whitelisting approach but that would com at a cost of discoverability and serendipity. This would strengthen the position of anointed 'influencers' and curators, and diminish the value of algorithmic feeds so eating into the revenue model of those that rely on this. Since those forces are counteracting it is hard to make predictions, but I will anyway but take it with a strong dose of uncertainty. My dystopian take is the ubiquitous deployment of 'fake' actors will further undermine the general inter-human levels of trust. Evolutionary less fragile altruistic strategies rely on unfakable or at least costly signals to stand up to the 'free-rider' intrusion. Sadly undermining trust will accelerate the further descend into identity tribalism we are already witnessing today, a segregation into near-immutable trait based groups where cross-clan transgressions are punished with extreme measures. As a caricature think of the television portrayal of tribal gang cultures in maximum security prisons. My utopian take is that the onslaught of fake noise will restore the reliance on offline contacts and connections. Due to physical proximity these offline groups have to share more of the negative externalities caused by their actions which could lead to more altruistic consideration in consumption and production descisioning. Direct verifyable contribution and impact might trump systemic distrust and lead us out of the current innate identity tribal descend.

jeffshek 7y ago

I love PyTorch, but I’m not confident the claim that it is the most popular is close to true. The cited link, which brings up a lot of new research is in PyTorch simply doesn’t account for the amount of TensorFlow in production. Sure, a lot of academics may be embracing PyTorch, but almost all production models have been in TensorFlow. Tesla is a huge notable example that’s using PyTorch at scale. I do suspect that the split of TensorFlow 1 and 2 is perhaps one of the worst times for TF 2, many teams will likely try out PyTorch instead. I think both are amazing frameworks, however TF was designed for Google Scale .... which leads to a lot of difficulties since 99.9 are not at Google scale.

rckoepke 7y ago
Depends on how you measure it, of course. However, stackoverflow survey, google trends, and github octoverse all show PyTorch is on a steep upward trajectory that recently reached effective parity with TensorFlow and has not yet started slowing down.

TACIXAT 7y ago

>Human babies don’t get tagged data sets, yet they manage just fine, and it’s important for us to understand how that happens I do not really understand this. Human babies get a constant stream of labeled information from their parents. Contextualized speech is being fed to them for years. Toddlers repeat everything you say. Is this referring to something else that babies can do?

ben_w 7y ago
The things you’re calling labels are themselves learned data, as we don’t start off with an innate knowledge of language. Whether that distinction is important or not, I don’t know enough brain science to guess.
onefuncman 7y ago
I agree with you -- we have little idea how much labeled information is encoded in the genome either. I rather like to think of the genome as the most well trained model of life we have access to.
richardw 7y ago
I show my 19 month old daughter like three cartoon drawings of owls and she recognises a live one at the bird park instantly, unprompted. We have a way to go.
- taneq 7y ago
  I believe cartoons are our equivalent of adversarial images. They typically look nothing like (photos of) their namesake and yet we recognise them usually without prompting.
  - EliRivers 7y ago
    It is my understanding (although I sure don't have any evidence on me) that cartoons and such (at least, the ones where we haven't simply learned that this cartoon means this animal) work by being a picture of what we remember about an animal. Akin to a caricature; the cartoon contains the most salient features. It doesn't work by looking like the actual animal; it works by reacting with how we remember the animal.

jansbor 7y ago

Is it just me that are wondering why they did not use AI to predict where AI is going in 2020?

deleted 7y ago
[deleted]
TheOtherHobbes 7y ago
They did, but they didn't understand the results.

mark_l_watson 7y ago

The linked page threw up a suspicious looking overlay. I left the site, too bad since I wrote a blog with my predictions last night and wanted to compare my AI predictions.

CodeGlitch 7y ago
If you use Firefox, just enable "Reader View" for the site (just hit F9). Removes all the crud around the page and just shows the text and required images.
reportgunner 7y ago
https://outline.com/yf9ACE https://outline.com/yf9ACE

graycat 7y ago

I believe we will (1) find some basic data structures and algorithms to do real AI. (2) At first it will be able to do I/O only via text or simple voice. (3) Due to (1) it will learn very quickly from humans or other sources. (4) Soon it will be genuinely smart, enough, say, to discover and prove new theorems in math, to understand physics and propose new research directions, to understand drama and write good screen plays, to understand various styles and cases of music and compose for those, etc. Broadly from (1) with the data structures it will be able to represent and store data and, then, from the algorithms, manipulate that data generating more data to be stored, etc. In particular it will be able to do well with thought experiments and generation and evaluation of scenarios. Good image understanding will come later but only a little later; the ideas in (1) may have to be revised to do well on image understanding.

tsimionescu 7y ago
You believe that we will achieve (1) in 2020? Or do you believe that we will achieve this at some point in general?
- graycat 7y ago
  Sorry, from reading another post about predictions for the next decade, I was thinking by 2030, not just 2020! Besides, for AI, just 2020 seems a bit too short!
The_rationalist 7y ago
* I believe we will (1) find some basic data structures and algorithms to do real AI.* There's no new kind of data structure to discover, humanity has made a disjonction of all possibles. The choice of data structure for semantic parsing is trivial, it's an hypergraph. The debate isn't the datastructure but how to fill it correctly while keeping the same Expressivity as in the original input (natural language). There's no reason to think we will make progress on this task beyond wishful thinking. Only a handful of humans are working on semantic parsing, which is the real AI task.
- 7y ago

dijksterhuis 7y ago

The page forwards itself to a spam google survey for me? Page history fills with the spam survey and cant navigate back to the article. iOS safari with reader view enabled.

kzzzznot 7y ago
I experienced the same behaviour

bitL 7y ago

- individual GPUs will hit a plateau at around 25TFlops in FP32 due to Moore's law and thermal dissipation however it will be easier than ever to interconnect multiple GPUs into large virtual ones due to interconnect tech improvements and modularization of GPU processing units - only large companies will be able to train and use SOTA models with training costs in $10M-$100M per training run and those models will hit law of diminishing returns quickly - 50% of all white collar jobs will be automated away, including a significant chunk of CRUD software work. Increased productivity won't be shared back with society, instead two distinct wealth strata of society will be formed worldwide due to scale effects, like in Latin America (<1% owners, >99% fighting for their lives). - AI will make marketing, ads and behavioral programming much more intrusive and practically unavoidable

mkl 7y ago
I think you're talking about a much longer period than just 2020. The job prediction seems unlikely to happen within the next decade, even.
petra 7y ago
>> including a significant chunk of CRUD software work. Can you please expand on how that will look like ?
wnkrshm 7y ago
So the Butlerian Jihad [0] for the 2040s? [0] https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian_Jihad https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian...

DagAgren 7y ago

Away. (Or less snarkily, https://twitter.com/iquilezles/status/1212377355349417986 https://twitter.com/iquilezles/status/1212377355349417986)

2sk21 7y ago
"Derp learning" :-)

szc 7y ago

Honesty, repeatability, numerical analysis. Canonicalization. Honesty: how many times was the exact same data processed? Was the result cherry picked and the best one published? For the sake of integrity how is it possible to scientifically improve on this result? (example, your AI outputs some life altering decision?) Repeatability: In science, if a result can be independently verified, it gives validity to the "conclusion" or result. Most AI results cannot be independently verified. Not indepentently verifiable really ought to give the "science" the same status as an 1800's "Dr. Bloobar's miracule AI cure". Numerical Analysis: performing billions / trillions of computations on bit-restricted numerical values will introduce a lot of forward propagated errors (noise). What does that do? Commentary: Video cards don't care if a few bits of your 15 million pixel display are off by a few LSB bits, they do that 60 or 120 frames a second and you don't notice. It is an integral part of their design. The issue is, how does this impact AI models? This affects repeatability -> honesty. If error of a quantized size is a necessarily required property to achieve "AI learning that converges", there is still an opportunity for canonicalization -- a way to map "different" converged models to explain why they are effectively the "same". This does not seem to be a "thing", why not? In my opinion, in 2020, the AI emperor still has no clothes.

fluffything 7y ago
For most of the engineering applications I work on, AI is useless. When we talk about controlling machines, our control algorithms have mathematically proven strict error bounds, such that if we provide an input with a particular maximal error (e.g. from a sensor that has some error tolerance), we can calculate what's the maximum error possible in the response that our model would produce, and then use that to evaluate whether this is even an input that should be handled by the current algorithm or not. These control algorithms all take some inputs, and use them to "predict" what will happen, and using that prediction, compute some response to correct it. These predictions need to happen much faster than real time, since you often need to perform an optimization step to compute an "optimal" response. These predictions are usually computed using a reduced-order model, e.g., if you had to solve a PDE over 10^9 unknowns to compute the actual prediction, you can instead reduce that to a system with 10 unknowns, by doing some pre-computation a priori. Most tools to do these kinds of reductions developed in the last 60 years come with tight error bounds, that tell you, depending on your inputs, the training data, etc. what's the largest error than the prediction can have, so you can just plug these in into your control pipeline. People have been plugin in neural-networks to control robots, cars, and pretty much anything you can imagine into these pipelines for 10 years, yet nobody knows what the upper bound on the errors that these neural-networks give for a particular input, training set, etc. Until that changes, machine learning just makes your whole pipeline unreliable, and e.g. a car manufacturer must tell you that in "autonomous driving" mode you are liable for everything your car does, and not them, so you have to keep your hands on the driving wheels and pay attention at all times, which... kind of defeats the point of autonomous driving. --- Prediction: we won't have any tight error bounds for real-world neural networks in the 2020-2030 time frame. These are all non-linear by design (that's why they are good), error bounds for simple non-linear interpolants are pretty much non-existent, people have tried for 20-30 years, and real-world NNs are anything but simple.

juskrey 7y ago

Fat tails

bordercases 7y ago
This is the content I come to Hacker News for.

y1tan 7y ago

I predict the broader ML/DL community will keep pumping out iterative papers that push the ball just a little bit forward while maintaining job security : Gatekeeping, no one thinking outside of the box, benchmark putting, just enough for the appearance of progress, and nothing broadly innovative or disruptive. The applications of ML/DL will continue to be gimmicky consumer products that have questionable valuable, questionable profit potential, add even more to disinformation/misinformation, produce more informational noise, only serve to rebuff a big corp's cloud offerings, and waste people's time. I predict tons more 'bought' articles that hype up AI technology for the typical 'household' names. I predict the same ol' echo chamber of thought and reinforcement of 'gatekept' ideology. I expect a number of more prominent articles critiquing the shortfalls of the technology. I expect a number of young minds steeped in DL/ML coming to the realization that it's not what they expected... That its a big profit/revenue story for Universities and established corporate platforms. I expect a number of them to realize ML/DL is truly not "AI" or anything close to it. That they aren't doing cutting edge research and that they are not allowed to think outside of the echo chamber of 'approved' approaches. I predict more useless chatbots that utter unpredictable word salads. I expect more gimmicky entertainment focused uses of it. I expect more assistants being adopted for data collection. I expect more people who aren't busy or doing anything important, using assistance assistants and text-to-speech to speed up their tasks so they can waste more of their time on social media/youtube/entertainment. Samsung Neon is coming out in some days.. making use of that 'Viv' acquisition. I expect more feverish attempts at attacking low hanging fruit jobs with overly complex solutions. I predict failures in a number of startups targeting this. I predict no pronounced progress in self-driving cars nor any particular grand use for them. I predict several hollow attempts to overlay symbolic systems over ML/DL or integration attempts of it with ML/DL from prominent AI figures. I predict pronounced failures in this effort cementing a partial end to the hype of ML/DL. I predict we will get a pronounced development outside of run-of-the-mill corporate/academic gatekept/walled garden ML/DL that will forge a new and higher path for AI. Hinton's words from prior years will have been heeded and the results of a new approach to AI presented. A change of guard, a break from the necessity of a PhD, a break from the echo-chamber of names, and a broader and more deeply thought out vision. Disruption not of low-hanging-fruit but disruption directed at the heart of the AI/Technology industry... So that we may finally progress from this stalled out disinformation/misinformation/hype/gatekeeping/cloud/all-your data-belongs-to-us cycle. It's 2020 after-all, time for a new age.

zackmorris 7y ago

Some axioms that I'm not seeing talked about much: * Artificial general intelligence (AGI) is the last problem in computer science, so it should be at least somewhat alarming that it's being funded by internet companies, wall street and the military instead of, say, universities/nonprofits/nonmilitary branches of the government. * Machine learning is conceptually simple enough that most software developers could work on it (my feeling is that the final formula for consciousness will fit on a napkin), but they never will, because of endlessly having to reinvent the wheel to make rent - eventually missing the boat and getting automated out of a job. * AI and robot labor will create unemployment and underemployment chaos if we don't implement universal basic income (UBI) or at the very least, reform the tax system so that automation provides for the public good instead of the lion's share of the profit going to a handful of wealthy financiers. * Children aren't usually exposed to financial responsibility until around the age of 15 or so, so training machine learning for financial use is likely to result in at least some degree of sociopathy, wealth inequality and further entrenchment of the status quo (what we would consider misaligned ethics). * Humans may not react well when it's discovered that self-awareness is emotion, and that as computers approach sentience they begin to act more like humans trapped in boxes, and that all of this is happening before the world can even provide justice and equality for the "other" (women, minorities, immigrants, oppressed creeds, intersexed people, the impoverished, etc etc etc). My prediction for 2020: nothing. But for 2025: an optimal game-winning strategy is taught in universities. By 2030: the optimal game-winning strategy is combined with experience from quantum computing to create an optimal search space strategy using exponentially fewer resources than anything today (forming the first limited AGI). By 2035: AGI is found to require some number of execution cycles to evolve, perhaps costing $1 trillion. By 2040: cost to evolve AGI drops to $10 billion and most governments and wealthy financiers own what we would consider a sentient agent. By 2045: AGI is everywhere and humanity is addicted to having any question answered by the AGI oracle so progress in human-machine merging, immortality and all other problems are predicted to be solved within 5 years. By 2050: all human problems have either been enumerated or solved and attention turns to nonhuman motives that can't be predicted (the singularity).

drongoking 7y ago

I'm generally pessimistic about predictions of the future. In this case I can't help but smile. They're trying to predict how a field (AI), which deals with complex adaptation, will intelligently adapt its adaptive techniques in the coming year, within an environment (we humans) that are themselves changing behavior while adapting to AI. That's approximately three meta levels. Good luck, guys!