3 ms·

This is about deep learning, of which LLMs are a subset. If you are interested in machine learning, then you should learn deep learning. It is incredibly useful

This is about deep learning, of which LLMs are a subset. If you are interested in machine learning, then you should learn deep learning. It is incredibly useful for a lot of reasons. Unlike other areas of ML, the nature of deep learning is such that its parts are interoperable. You could use a transformer with a CNN if you wish. Also, deep learning enables you to do machine learning on any type of data, text, images, video, audio. Finally, it can naturally scale computationally. As someone pretty involved in the field, I lament that LLMs are turning people away from ML and deep learning, and following the misconceptions that there’s no reason to do it anymore. Large algorithms are expensive to run, have slow throughput and are still generally poorer performing than purpose built models. They’re not even that easy to use for a lot of tasks, in comparison to encoder networks. I’m biased, but I think it’s one of the most fun things to learn in computing. And if you have a good idea, you can still build state of the art things with a regular gpu at your house. You just have to find a niche that isn’t getting the attention that LLMs are ;)

dafaqueue 3y ago
I started off being really excited to learn, but as time went on I actually lost interest in the field. The whole thing is essentially curve fitting. The ML field is essentially an art more than a science and it's all about tricks and intuitions on different ways of getting that best fit curve. From this angle the whole field got way less interesting. The field has nothing deeper or more insightful to offer beyond this concept of curve fitting.
- dbmikus 3y ago
  I've found this fun way to think of it: the goal is to invent a faster form of evolution for pattern recognition, learning, and autonomous task completion. I think one needs to consider it more like biology and a science than pure logic and math. We can discover things that work, and then after that we can study them to learn why they work, just like we don't fully understand the brain yet. I think there are some really cool problems, such as: 1. Is synthetic data viable for training? 2. How do you make deep learning agents that can do task planning and introspection in complex environments? 3. How do we efficiently build memory and data lookup into AI agents? And is this better/worse than making longer context windows?
- Slix 3y ago
  Are deep learning and neural networks just curve fitting? I thought those were significantly different.
  - galangalalgol 3y ago
    You could argue all the building blocks are forms of curve fits, but that isn't a terribly useful statement even if true. If you can fit a curve to the desired behavior of any function, or composition of functions (which is a function) then you can solve any problem you can express the desired behavior of. Including the expressing of desired behavior for some other class if problems. Saying it is just curve fitting is like saying something is just math. The entirety of reality is just math.
  - deepsquirrelnet 3y ago
    By that logic, anything that is predictive is curve fitting, including entire academic fields like physics and climatology. You could say that all automation is curve fitting. I don’t think there’s much to be gained by being that reductive. From a technical standpoint, it’s not correct analogy either, because it assumes you have a curve to fit. What curve is language? What’s curve is images? No answer, because there isn’t one. Deep learning is about modeling complex behaviors, not curve fitting. Images and language for instance are based in social and cultural patterns and not intrinsic curves to be fit. At best, it’s an imprecise statement. But I’d disagree entirely.
- wiz21c 3y ago
  Although it fundamentally is curve fitting, I'd venture to say that at some point, having to handle millions of parameters makes the curve fitting problem unrecognizable... A change in quantity is a change is nature if you will. IOW: to me, fitting a generalized linear model is very different than fitting a convolutional network.

dafaqueue 3y ago

I started off being really excited to learn, but as time went on I actually lost interest in the field. The whole thing is essentially curve fitting. The ML field is essentially an art more than a science and it's all about tricks and intuitions on different ways of getting that best fit curve. From this angle the whole field got way less interesting. The field has nothing deeper or more insightful to offer beyond this concept of curve fitting.

dbmikus 3y ago
I've found this fun way to think of it: the goal is to invent a faster form of evolution for pattern recognition, learning, and autonomous task completion. I think one needs to consider it more like biology and a science than pure logic and math. We can discover things that work, and then after that we can study them to learn why they work, just like we don't fully understand the brain yet. I think there are some really cool problems, such as: 1. Is synthetic data viable for training? 2. How do you make deep learning agents that can do task planning and introspection in complex environments? 3. How do we efficiently build memory and data lookup into AI agents? And is this better/worse than making longer context windows?
Slix 3y ago
Are deep learning and neural networks just curve fitting? I thought those were significantly different.
- galangalalgol 3y ago
  You could argue all the building blocks are forms of curve fits, but that isn't a terribly useful statement even if true. If you can fit a curve to the desired behavior of any function, or composition of functions (which is a function) then you can solve any problem you can express the desired behavior of. Including the expressing of desired behavior for some other class if problems. Saying it is just curve fitting is like saying something is just math. The entirety of reality is just math.
- deepsquirrelnet 3y ago
  By that logic, anything that is predictive is curve fitting, including entire academic fields like physics and climatology. You could say that all automation is curve fitting. I don’t think there’s much to be gained by being that reductive. From a technical standpoint, it’s not correct analogy either, because it assumes you have a curve to fit. What curve is language? What’s curve is images? No answer, because there isn’t one. Deep learning is about modeling complex behaviors, not curve fitting. Images and language for instance are based in social and cultural patterns and not intrinsic curves to be fit. At best, it’s an imprecise statement. But I’d disagree entirely.
wiz21c 3y ago
Although it fundamentally is curve fitting, I'd venture to say that at some point, having to handle millions of parameters makes the curve fitting problem unrecognizable... A change in quantity is a change is nature if you will. IOW: to me, fitting a generalized linear model is very different than fitting a convolutional network.