5 ms·

Reading list to join AI field from Hugging Face cofounder

114 points by triyambakam 2y ago

deleted 2y ago
[deleted]
apwell23 2y ago
he probably felt the need to prescribe dense theoretical books given his position. prbably look like a fraud if he recommended ' grokking machine learning on manning.com'
- LatticeAnimal 2y ago
  Are you implying that he is adjusting his reading list to fake competence? Thomas Wolf has the second highest number of commits on HuggingFace/transformers. He is clearly competent & deeply technical https://github.com/huggingface/transformers/ https://github.com/huggingface/transformers/
- elicksaur 2y ago
  This comment seems unnecessarily personal. The difference between the reading list and Grokking Machine Learning appears to be the layer of application. ‘Grokking’s description indicates it is for a very applied role: “teaches you how to apply ML to your projects using only standard Python code and high school-level math”. The linked reading list appears to be targeted at one level up the stack so to speak. Instead of learning how to string a few Python libraries together (which is useful, not knocking it), the goal would possibly be writing the Python library itself for a ML architecture. Given the author’s position, it makes sense why they found the content of the latter useful in getting to that position.
sampo 2y ago
> Probabilistic Graphical Models Specialization from coursera (https://www.coursera.org/specializations/probabilistic-graphical-models https://www.coursera.org/specializations/probabilistic-graph...) Are Probabilistic Graphical Models still being used? They don't get much visibility these days.
seattle_spring 2y ago
Dumb question, but... Is AI and ML the same thing? I ask because I am seeing a lot of my ML peers claim to be AI experts with years of experience, even though the term AI wasn't really used at all until OpenAI got big.
- Ukv 2y ago
  AI as a field has been around since the 50s and gone through multiple paradigms, but modern AI (since its resurgence around 2010) is effectively all machine learning.
  - cscurmudgeon 2y ago
    Not really though. Modern "publicly known and commercial" AI is effectively all machine learning For a sample of non-ML AI that ML still has trouble solving, see papers published in: https://ijcai-23.org/main-track-accepted-papers/ https://ijcai-23.org/main-track-accepted-papers/
- Centigonal 2y ago
  This is the venn diagram I keep seeing when this question is asked: https://www.researchgate.net/publication/363930739/figure/fig2/AS:11431281224920472@1708517958416/Venn-Diagram-relationship-between-artificial-intelligence-machine-learning-and-deep.jpg https://www.researchgate.net/publication/363930739/figure/fi... GenAI ⊆ Deep Learning ⊆ Machine Learning ⊆ AI, where ⊆ means "Is a subset of."
  - Scarblac 2y ago
    Also, originally I think Machine Learning was more related to the field of Statistics than other AI methods that were more traditional CS.
- wdh505 2y ago
  Im not a programmer, but my understanding is that back in the '80s investors got in a frenzy when the idea of AI was big. They threw money at it and... it was decades away. Then they tried the "e" phase that became the dot com boom... and it was overhyped. This pattern has occured several times with programs. Technology has been progressing and the definition of AI is loose at best when marketing is nearby. I have a question: what kinds of ways do people describe "unemployed" on linkedin? It is probably glossed over and hyped based on whatever marketing is difficult to verify and is at least adjacent to reality. I think this is similar to ML researchers classifying themselves as AI researchers. HOWEVER, there may be a lot more overlap than simply "adjacent", so someone please correct me if the term "machine learning" or "AI" are regulated terms for public advertisement.
unraveller 2y ago
Aren't the LLMs on hf good enough to learn from?
philipov 2y ago
I've always wondered if Hugging Face is meant to evoke the feeling of being attacked by a Facehugger.
- Der_Einzige 2y ago
  It's meant to evoke the feeling of being in therapy. It started as an attempt to be an AI therapy company, and they pivoted to "github for AI" around 2019.
- Mkengine 2y ago
  Apparently, the name goes back to the unicode emoji for a hug (): https://qz.com/hugging-face-microsoft-artificial-intelligence-1850490270 https://qz.com/hugging-face-microsoft-artificial-intelligenc...
ks2048 2y ago
I appreciate simplicity, but this could use take advice from https://perfectmotherfuckingwebsite.com/ https://perfectmotherfuckingwebsite.com/
gth158a 2y ago
I have read all of them and I have watched almost all of the training videos mentioned and that, is the list I would give anybody who I would want to discourage from joining the field. The textbooks I mean COME ON. They are comprehensive and neatly academic but not pragmatic at all. There are playlists in youtube that would cut through all the unnecessary content and will get you building RAG systems in a couple of hours. Kudos to the Langchain team for all the content online.
- ipnon 2y ago
  The fastest way to learn is always to go after a difficult problem you find fascinating. Learning by the books is the second fastest, and is suitable for those without the experience to have a fascinating problem to motivate them. This applies to many but not all, for example a university student might have a vague interest in AI but not know where to start. If you’re already in the field though, it’s best to just keep hacking forwards, even if you can’t prove every theorem from the major textbooks.
- Copenjin 2y ago
  Feel free to post those playlists for those discouraged by the list of academic resources :)
  - ckrusk 2y ago
    I second this!
teleforce 2y ago
Thanks for the list. Interesting to see that the book by the late David MacKay made it to the list, it is available to download in several formats from the book's website. Information Theory, Inference, and Learning Algorithms: https://www.inference.org.uk/mackay/itila/ https://www.inference.org.uk/mackay/itila/ The lectures based on the book by David MacKay himself: https://youtu.be/BCiZc0n6COY https://youtu.be/BCiZc0n6COY Probably it's not a surprise that the father of information theory Claude Shannon is the earliest one to propose stochastic LLM based on Markov chain.
jszymborski 2y ago
Bengio & Goodfellows book, while a great reference, is certainly not what I'd recommend for beginners. It's great for experienced folks IMHO, but otherwise it's quite opaque. I've used it to teach in the past and beginners really struggled with it. Still an amazing reference though!

114 points by triyambakam 2y ago

deleted 2y ago

[deleted]

he probably felt the need to prescribe dense theoretical books given his position. prbably look like a fraud if he recommended ' grokking machine learning on manning.com'

LatticeAnimal 2y ago
Are you implying that he is adjusting his reading list to fake competence? Thomas Wolf has the second highest number of commits on HuggingFace/transformers. He is clearly competent & deeply technical https://github.com/huggingface/transformers/ https://github.com/huggingface/transformers/
elicksaur 2y ago
This comment seems unnecessarily personal. The difference between the reading list and Grokking Machine Learning appears to be the layer of application. ‘Grokking’s description indicates it is for a very applied role: “teaches you how to apply ML to your projects using only standard Python code and high school-level math”. The linked reading list appears to be targeted at one level up the stack so to speak. Instead of learning how to string a few Python libraries together (which is useful, not knocking it), the goal would possibly be writing the Python library itself for a ML architecture. Given the author’s position, it makes sense why they found the content of the latter useful in getting to that position.

sampo 2y ago

> Probabilistic Graphical Models Specialization from coursera (https://www.coursera.org/specializations/probabilistic-graphical-models https://www.coursera.org/specializations/probabilistic-graph...) Are Probabilistic Graphical Models still being used? They don't get much visibility these days.

seattle_spring 2y ago

Dumb question, but... Is AI and ML the same thing? I ask because I am seeing a lot of my ML peers claim to be AI experts with years of experience, even though the term AI wasn't really used at all until OpenAI got big.

Ukv 2y ago
AI as a field has been around since the 50s and gone through multiple paradigms, but modern AI (since its resurgence around 2010) is effectively all machine learning.
- cscurmudgeon 2y ago
  Not really though. Modern "publicly known and commercial" AI is effectively all machine learning For a sample of non-ML AI that ML still has trouble solving, see papers published in: https://ijcai-23.org/main-track-accepted-papers/ https://ijcai-23.org/main-track-accepted-papers/
Centigonal 2y ago
This is the venn diagram I keep seeing when this question is asked: https://www.researchgate.net/publication/363930739/figure/fig2/AS:11431281224920472@1708517958416/Venn-Diagram-relationship-between-artificial-intelligence-machine-learning-and-deep.jpg https://www.researchgate.net/publication/363930739/figure/fi... GenAI ⊆ Deep Learning ⊆ Machine Learning ⊆ AI, where ⊆ means "Is a subset of."
- Scarblac 2y ago
  Also, originally I think Machine Learning was more related to the field of Statistics than other AI methods that were more traditional CS.
wdh505 2y ago
Im not a programmer, but my understanding is that back in the '80s investors got in a frenzy when the idea of AI was big. They threw money at it and... it was decades away. Then they tried the "e" phase that became the dot com boom... and it was overhyped. This pattern has occured several times with programs. Technology has been progressing and the definition of AI is loose at best when marketing is nearby. I have a question: what kinds of ways do people describe "unemployed" on linkedin? It is probably glossed over and hyped based on whatever marketing is difficult to verify and is at least adjacent to reality. I think this is similar to ML researchers classifying themselves as AI researchers. HOWEVER, there may be a lot more overlap than simply "adjacent", so someone please correct me if the term "machine learning" or "AI" are regulated terms for public advertisement.

unraveller 2y ago

Aren't the LLMs on hf good enough to learn from?

philipov 2y ago

I've always wondered if Hugging Face is meant to evoke the feeling of being attacked by a Facehugger.

Der_Einzige 2y ago
It's meant to evoke the feeling of being in therapy. It started as an attempt to be an AI therapy company, and they pivoted to "github for AI" around 2019.
Mkengine 2y ago
Apparently, the name goes back to the unicode emoji for a hug (): https://qz.com/hugging-face-microsoft-artificial-intelligence-1850490270 https://qz.com/hugging-face-microsoft-artificial-intelligenc...

ks2048 2y ago

I appreciate simplicity, but this could use take advice from https://perfectmotherfuckingwebsite.com/ https://perfectmotherfuckingwebsite.com/

gth158a 2y ago

I have read all of them and I have watched almost all of the training videos mentioned and that, is the list I would give anybody who I would want to discourage from joining the field. The textbooks I mean COME ON. They are comprehensive and neatly academic but not pragmatic at all. There are playlists in youtube that would cut through all the unnecessary content and will get you building RAG systems in a couple of hours. Kudos to the Langchain team for all the content online.

ipnon 2y ago
The fastest way to learn is always to go after a difficult problem you find fascinating. Learning by the books is the second fastest, and is suitable for those without the experience to have a fascinating problem to motivate them. This applies to many but not all, for example a university student might have a vague interest in AI but not know where to start. If you’re already in the field though, it’s best to just keep hacking forwards, even if you can’t prove every theorem from the major textbooks.
Copenjin 2y ago
Feel free to post those playlists for those discouraged by the list of academic resources :)
- ckrusk 2y ago
  I second this!

teleforce 2y ago

Thanks for the list. Interesting to see that the book by the late David MacKay made it to the list, it is available to download in several formats from the book's website. Information Theory, Inference, and Learning Algorithms: https://www.inference.org.uk/mackay/itila/ https://www.inference.org.uk/mackay/itila/ The lectures based on the book by David MacKay himself: https://youtu.be/BCiZc0n6COY https://youtu.be/BCiZc0n6COY Probably it's not a surprise that the father of information theory Claude Shannon is the earliest one to propose stochastic LLM based on Markov chain.

jszymborski 2y ago

Bengio & Goodfellows book, while a great reference, is certainly not what I'd recommend for beginners. It's great for experienced folks IMHO, but otherwise it's quite opaque. I've used it to teach in the past and beginners really struggled with it. Still an amazing reference though!