4 ms·

You are plain exaggerating. You can't do all of them in a few weeks. Algorithms: Lin Reg -> Log Reg -> NN -> CNN + RNN -> GANs + Transformers -> ViT ->

by rg111 4y ago

You are plain exaggerating. You can't do all of them in a few weeks. Algorithms: Lin Reg -> Log Reg -> NN -> CNN + RNN -> GANs + Transformers -> ViT -> Multimodal AI + LLMs + Diffusion + Auto Encoders SVM, PCA, kNN, k-means clustering, etc. LightGBM, XGboost, Catboost, etc. Optimization and optimizers. Application-wise: Classification, Semantic Segmentation, Pose Estimation, Text Generation, Summarization, NER, Image Generation, Captioning, Sequence Generation (like music/speech), text to speech, speech to text, recommender systems, sentiment amalysis, tabular data, etc. Frameworks: pandas, sklearn, PyTorch, Jax -> training inference, data loading Platforms: AWS + GCP + Azure And a lot of GPU shenanigans + framework/platform specific quirks All these will take you ~2 years or 1.5 years at least, given that: - you already know Python/any programming language properly - you already know college level math (many people say you don't need it, but haven't met a single soul in ML research/modelling without college level math) - you know Stats 101 matching a good uni curriculum and ability to learn beyond - you know git, docker, cli, etc. Every influencer and their mother promising to teach you Data Science in 30 days are plain lying. Edit: I see that I left out Deep RL. Let's keep it that way for now. Edit2: Added tree based methods. These are very important. XGBoost outperforms NNs every time on tabular data. I also once used an RF head appended to a DNN, for final prediction. Added optimizers.

jimbokun 4y ago
> SVM, PCA, kNN, k-means clustering Are these still relevant in the age of Deep Neural Networks?
- rg111 4y ago
  Yes. Different problems require different solutions. Sometimes, an NN would be overkill. And stakeholders in many situations would like insights why the prediction is what it is. NNs are miles behind LogReg in terms of interpretablity.
- popinman322 4y ago
  PCA is a foundational dimension reduction technique, and kNN can be used in conjunction with embeddings. k-means is still great when you have prior/domain knowledge about the number of groups.
  - nephanth 4y ago
    K-means is pretty poor when the clusters are not linearly separated, but it is the basis of a lot of more modern clustering techniques (kernel K-means if you have prior knoweledge, spectral clustering...)
- PeterisP 4y ago
  Yes, there are all kinds of tasks where the appropriate solution is to use a DNN for much of the learning (either directly learning the correlations or as transfer learning from some large-data self-supervised task) and then, once you have the results of that DNN inference, work with these methods - apply PCA for interpreting the resulting vector, or to separate out specific dimensions to expose them for adjustment in some generative task; or perhaps the best way for the final decision is a kNN on top of the DNN output, etc.
- jeffreyrogers 4y ago
  It's not in your list but decision trees still outperform DNN on many tabular problems and can be trained faster.
- nephanth 4y ago
  Also boosting. But yes these algs are the basis of a lot of more modern algorithms. A deep NN won't do unsupervised clustering for ex, and NNs perform more poorly than simpler models on small datasets

by rg111 4y ago

jimbokun 4y ago

> SVM, PCA, kNN, k-means clustering Are these still relevant in the age of Deep Neural Networks?

rg111 4y ago
Yes. Different problems require different solutions. Sometimes, an NN would be overkill. And stakeholders in many situations would like insights why the prediction is what it is. NNs are miles behind LogReg in terms of interpretablity.
popinman322 4y ago
PCA is a foundational dimension reduction technique, and kNN can be used in conjunction with embeddings. k-means is still great when you have prior/domain knowledge about the number of groups.
- nephanth 4y ago
  K-means is pretty poor when the clusters are not linearly separated, but it is the basis of a lot of more modern clustering techniques (kernel K-means if you have prior knoweledge, spectral clustering...)
PeterisP 4y ago
Yes, there are all kinds of tasks where the appropriate solution is to use a DNN for much of the learning (either directly learning the correlations or as transfer learning from some large-data self-supervised task) and then, once you have the results of that DNN inference, work with these methods - apply PCA for interpreting the resulting vector, or to separate out specific dimensions to expose them for adjustment in some generative task; or perhaps the best way for the final decision is a kNN on top of the DNN output, etc.
jeffreyrogers 4y ago
It's not in your list but decision trees still outperform DNN on many tabular problems and can be trained faster.
nephanth 4y ago
Also boosting. But yes these algs are the basis of a lot of more modern algorithms. A deep NN won't do unsupervised clustering for ex, and NNs perform more poorly than simpler models on small datasets