16 ms·

Seems like the person who wrote the blog works in "classical" deep learning. So do I, so here's the fairest take I can come up with: "AI" has for recent memory

Seems like the person who wrote the blog works in "classical" deep learning. So do I, so here's the fairest take I can come up with: "AI" has for recent memory been a marketing term anyway. Deep learning and variations have had a good run at being what people mean when they refer to AI, probably overweighting towards big convolution based computer vision models. Now, "AI" in people's minds means generative models. That's it, it doesn't mean generative models are replacing CNNs, just like CNNs don't replace SVMs or regression or whatever. It's just that pop culture has fallen in love with something else.

JohnFen 4y ago
Spot on. I work with deep learning systems in industrial control, and generative models are simply ill-suited for this sort of work. Wrong tool for the job. But neither the traditional nor generative models are "AI" in the sense that normal people think when they hear "AI".
- fakedang 4y ago
  I'm curious about your work, because I worked on something similar during my grad school. What kind of applications in industry do you use deep learning systems for? Process control?
  - JohnFen 4y ago
    Yes, process control. It's used in coordination with vision systems to analyze work pieces, determine the best way of processing them, and direct other machinery how to do that processing.
    - fakedang 4y ago
      That's cool. If you don't mind me asking, would you have any shallow level stuff that I could read on about this? Even a website or a blog post would be great. In my grad school, we were working on something similar - using computer vision to analyze reactor flows to then change process variables. The results would be fed back into the system for RL. Too bad the project sorta froze after I graduated.
      - JohnFen 4y ago
        Your grad school project sounds very similar, yes, although we work with discrete objects rather than fluids. Fluid dynamics is much, much more complicated. We actually use more than one neural network for this. The software is designed so the NN component is a plugin. The reason we do this is because some types of neural nets work better for some tasks than others. Most (but not all) of our nets are convolutional. Since you've already done some work with this sort of thing, I'm unsure about what level of overview would be of value to you, but this looks reasonable for a technically competent person who is new to the topic: https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53 https://towardsdatascience.com/a-comprehensive-guide-to-conv... As with all neural nets, the "secret sauce" isn't the code, it's the training.
- nico 4y ago
  To me what’s exciting about Chat/GPT type of tech, is that they can be the “coordinators” of other models. Imagine asking an AI assistant to perform a certain industrial control task. The assistant, instead of executing the task “itself”, could figure out which model/system should perform the task and have it do it. Then even monitor the task and check it’s completion.
  - JohnFen 4y ago
    I'm not sure how the likes of ChatGPT could accomplish that even in theory, but I won't say it's not possible at some point in the future. Gpt itself, perhaps, someday.
    - IanCal 4y ago
      You can just tell the models to and tell them what tools they have available and how to call out to them. Langchain supports this iirc.
    - nico 4y ago
      Already ChatSpot is doing it. Their system is essentially a ChatGPT-enhanced Hubspot management system using chatux. ChatSpot can understand your commands and then perform actions in the system for you, for example add a lead, change their contact info, write a blog post, publish it, add an image… Edit: but if you connected it with physical actions, it could control your house, maybe check your smart refrigerator, order food on Instacart, send you recipe, schedule the time to cook in your calendar, request an Uber to pick you up from work, invite someone over, play music… There’s a discussion about this on another homepage thread here: https://news.ycombinator.com/item?id=35172362 https://news.ycombinator.com/item?id=35172362
      - JohnFen 4y ago
        Ah, ok. I thought you were talking about something a bit more profound than that.
      - nostromo123 4y ago
- deleted 4y ago
  [deleted]
- njarboe 4y ago
  The term "AI" was corrupted as described. People now use the term "artificial general intelligence" (AGI) to refer to what used to be called AI.
  - JohnFen 4y ago
    I was talking about what the average person thinks when they hear "AI". The average person has never even heard the term "AGI".
fnordpiglet 4y ago
AI has been marketing term since the day it was coined. It means literally nothing, which means it can mean anything.
- burbankio 4y ago
  As the old joke goes, "AI" is anything that doesn't work yet. Once an "AI" system becomes reliable, we quickly take it for granted and it no longer seems impressive or interesting. It's just a database. Or an image classifier. Or a chatbot.
  - fnordpiglet 4y ago
    Magic is just science we don’t understand yet.
    - Jensson 4y ago
      Science is just magic we do understand is a cooler take.
      - fnordpiglet 4y ago
        Yes that’s the one I use for my daughter ;-)
  - Tanjreeve 4y ago
    I'd argue a database is possibly as far from an AI as you get. Indexing and data structures and storage systems that go into them are very deterministic data structures you can model out and roughly know the behaviour of before writing a single line of code. Image classifiers and Chatbots you don't know what you're getting out till you train it and deploy it.
- deleted 4y ago
  [deleted]
gautamdivgi 4y ago
Isn’t most of the mathematics of AI old, as in really old? Regression, both linear and logistic are from the mid 1800s to early 1900s. Neural networks, at least the basics are from around 1950. What has really changed is the engineering, the data volume and the number of fields we can apply the mathematics to. The math itself (or what is the basis of AI) is really old.
- sterlind 4y ago
  backpropagation didn't get solved until the '80s, weirdly. before then people were using genetic algorithms to train neural networks. and it was only in the last decade that the vanishing gradients problem was tamed. my impression is that ML researchers were stumbling along in the mathematical dark, until they hit a combination (deep neural nets trained via stochastic gradient descent with ReLU activation) that worked like magic and ended the AI winter.
  - version_five 4y ago
    Right, and the practice of neural networks has significantly overshot the mathematical theory. Most of the aspects we know work and result in good models have poorly understood theoretical underpinnings. The whole overparamiterized thing for example, or generalization generally. There's a lot that "just works" but we don't know why, thus the stumbling around and landing on stuff that works
  - cma 4y ago
    > and it was only in the last decade that the vanishing gradients problem was tamed. One of the big pieces was Schmidhuber's lab's highway nets, done ~30 years ago, but just didn't land until a more limited version was rediscovered.
sterlind 4y ago
do people actually use SVMs anymore? like, regression, sure - because it's a tool to measure how well a hypothesis (polynomial function) matches the data (points.) and CNNs are still foundational in computer vision. but the first and last time I heard of SVMs was in college, by professors who were weirdly dismissive of these newfangled deep neural networks, and enamored by the "kernel trick." but aren't SVMs basically souped up regression models? are they used in anything ML-esque, i.e. besides validating a hypothesis about the behavior of a system?
- jacksnipe 4y ago
  We loved them in medical testing. Very explainable models.
- teruakohatu 4y ago
  > do people actually use SVMs anymore? Yes they are. They allow for non-linear decision boundaries and more dimensions than rows of data, which for many other ML methods is a problem. Linear regression, logistic regression, SVM and CART decision trees are all still very popular in the real world where data is hard to come by.
- superdisk 4y ago
  > but the first and last time I heard of SVMs was in college, by professors who were weirdly dismissive of these newfangled deep neural networks, and enamored by the "kernel trick." LOL. Exact same experience in my college courses. Glad to know it's universal.
rkhacker 4y ago
The Generative AI is the AI for the masses. While people were getting overhyped with all the possibilities and promises of AI and deep learning etc. it is for the first time that they can also tinker and get surprised by its results. People feel creative interacting with it.

JohnFen 4y ago

Spot on. I work with deep learning systems in industrial control, and generative models are simply ill-suited for this sort of work. Wrong tool for the job. But neither the traditional nor generative models are "AI" in the sense that normal people think when they hear "AI".

fakedang 4y ago
I'm curious about your work, because I worked on something similar during my grad school. What kind of applications in industry do you use deep learning systems for? Process control?
- JohnFen 4y ago
  Yes, process control. It's used in coordination with vision systems to analyze work pieces, determine the best way of processing them, and direct other machinery how to do that processing.
  - fakedang 4y ago
    That's cool. If you don't mind me asking, would you have any shallow level stuff that I could read on about this? Even a website or a blog post would be great. In my grad school, we were working on something similar - using computer vision to analyze reactor flows to then change process variables. The results would be fed back into the system for RL. Too bad the project sorta froze after I graduated.
    - JohnFen 4y ago
      Your grad school project sounds very similar, yes, although we work with discrete objects rather than fluids. Fluid dynamics is much, much more complicated. We actually use more than one neural network for this. The software is designed so the NN component is a plugin. The reason we do this is because some types of neural nets work better for some tasks than others. Most (but not all) of our nets are convolutional. Since you've already done some work with this sort of thing, I'm unsure about what level of overview would be of value to you, but this looks reasonable for a technically competent person who is new to the topic: https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53 https://towardsdatascience.com/a-comprehensive-guide-to-conv... As with all neural nets, the "secret sauce" isn't the code, it's the training.
nico 4y ago
To me what’s exciting about Chat/GPT type of tech, is that they can be the “coordinators” of other models. Imagine asking an AI assistant to perform a certain industrial control task. The assistant, instead of executing the task “itself”, could figure out which model/system should perform the task and have it do it. Then even monitor the task and check it’s completion.
- JohnFen 4y ago
  I'm not sure how the likes of ChatGPT could accomplish that even in theory, but I won't say it's not possible at some point in the future. Gpt itself, perhaps, someday.
  - IanCal 4y ago
    You can just tell the models to and tell them what tools they have available and how to call out to them. Langchain supports this iirc.
  - nico 4y ago
    Already ChatSpot is doing it. Their system is essentially a ChatGPT-enhanced Hubspot management system using chatux. ChatSpot can understand your commands and then perform actions in the system for you, for example add a lead, change their contact info, write a blog post, publish it, add an image… Edit: but if you connected it with physical actions, it could control your house, maybe check your smart refrigerator, order food on Instacart, send you recipe, schedule the time to cook in your calendar, request an Uber to pick you up from work, invite someone over, play music… There’s a discussion about this on another homepage thread here: https://news.ycombinator.com/item?id=35172362 https://news.ycombinator.com/item?id=35172362
    - JohnFen 4y ago
      Ah, ok. I thought you were talking about something a bit more profound than that.
    - nostromo123 4y ago
deleted 4y ago
[deleted]
njarboe 4y ago
The term "AI" was corrupted as described. People now use the term "artificial general intelligence" (AGI) to refer to what used to be called AI.
- JohnFen 4y ago
  I was talking about what the average person thinks when they hear "AI". The average person has never even heard the term "AGI".

fnordpiglet 4y ago

AI has been marketing term since the day it was coined. It means literally nothing, which means it can mean anything.

burbankio 4y ago
As the old joke goes, "AI" is anything that doesn't work yet. Once an "AI" system becomes reliable, we quickly take it for granted and it no longer seems impressive or interesting. It's just a database. Or an image classifier. Or a chatbot.
- fnordpiglet 4y ago
  Magic is just science we don’t understand yet.
  - Jensson 4y ago
    Science is just magic we do understand is a cooler take.
    - fnordpiglet 4y ago
      Yes that’s the one I use for my daughter ;-)
- Tanjreeve 4y ago
  I'd argue a database is possibly as far from an AI as you get. Indexing and data structures and storage systems that go into them are very deterministic data structures you can model out and roughly know the behaviour of before writing a single line of code. Image classifiers and Chatbots you don't know what you're getting out till you train it and deploy it.
deleted 4y ago
[deleted]

gautamdivgi 4y ago

Isn’t most of the mathematics of AI old, as in really old? Regression, both linear and logistic are from the mid 1800s to early 1900s. Neural networks, at least the basics are from around 1950. What has really changed is the engineering, the data volume and the number of fields we can apply the mathematics to. The math itself (or what is the basis of AI) is really old.

sterlind 4y ago
backpropagation didn't get solved until the '80s, weirdly. before then people were using genetic algorithms to train neural networks. and it was only in the last decade that the vanishing gradients problem was tamed. my impression is that ML researchers were stumbling along in the mathematical dark, until they hit a combination (deep neural nets trained via stochastic gradient descent with ReLU activation) that worked like magic and ended the AI winter.
- version_five 4y ago
  Right, and the practice of neural networks has significantly overshot the mathematical theory. Most of the aspects we know work and result in good models have poorly understood theoretical underpinnings. The whole overparamiterized thing for example, or generalization generally. There's a lot that "just works" but we don't know why, thus the stumbling around and landing on stuff that works
- cma 4y ago
  > and it was only in the last decade that the vanishing gradients problem was tamed. One of the big pieces was Schmidhuber's lab's highway nets, done ~30 years ago, but just didn't land until a more limited version was rediscovered.

sterlind 4y ago

do people actually use SVMs anymore? like, regression, sure - because it's a tool to measure how well a hypothesis (polynomial function) matches the data (points.) and CNNs are still foundational in computer vision. but the first and last time I heard of SVMs was in college, by professors who were weirdly dismissive of these newfangled deep neural networks, and enamored by the "kernel trick." but aren't SVMs basically souped up regression models? are they used in anything ML-esque, i.e. besides validating a hypothesis about the behavior of a system?

jacksnipe 4y ago
We loved them in medical testing. Very explainable models.
teruakohatu 4y ago
> do people actually use SVMs anymore? Yes they are. They allow for non-linear decision boundaries and more dimensions than rows of data, which for many other ML methods is a problem. Linear regression, logistic regression, SVM and CART decision trees are all still very popular in the real world where data is hard to come by.
superdisk 4y ago
> but the first and last time I heard of SVMs was in college, by professors who were weirdly dismissive of these newfangled deep neural networks, and enamored by the "kernel trick." LOL. Exact same experience in my college courses. Glad to know it's universal.

rkhacker 4y ago

The Generative AI is the AI for the masses. While people were getting overhyped with all the possibilities and promises of AI and deep learning etc. it is for the first time that they can also tinker and get surprised by its results. People feel creative interacting with it.