5 ms·

Noob question here - what’s the best tutorials to get started in mixing LLM models and building on top of one another, assuming very good programming background

Noob question here - what’s the best tutorials to get started in mixing LLM models and building on top of one another, assuming very good programming background but little AI background? I asked chatGPT this question, and it was helpful but not comprehensive, but I figure intelligent humans on this forum will give the best answers.

RockyMcNuts 3y ago
not sure what 'mixing LLM models' entails but these are maybe some good starting points - karpathy - https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY - https://towardsdatascience.com/beautifully-illustrated-nlp-models-from-rnn-to-transformer-80d69faf2109 https://towardsdatascience.com/beautifully-illustrated-nlp-m... - https://dzone.com/articles/a-deep-dive-into-the-transformer-architecture-the https://dzone.com/articles/a-deep-dive-into-the-transformer-... - https://peterbloem.nl/blog/transformers https://peterbloem.nl/blog/transformers - http://nlp.seas.harvard.edu/2018/04/03/attention.html http://nlp.seas.harvard.edu/2018/04/03/attention.html - https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/ https://lilianweng.github.io/posts/2023-01-27-the-transforme... - https://blog.quickchat.ai/post/tokens-entropy-question/ https://blog.quickchat.ai/post/tokens-entropy-question/ - https://dugas.ch/artificial_curiosity/GPT_architecture.html https://dugas.ch/artificial_curiosity/GPT_architecture.html - https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... - https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf https://d4mucfpksywv.cloudfront.net/better-language-models/l... - https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf - https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf - https://arxiv.org/pdf/2303.17564.pdf https://arxiv.org/pdf/2303.17564.pdf
extasia 3y ago
My answer would be quite specific to what exactly you're trying to achieve. Id be wary of just hacking away without understanding at least the fundamentals of ML + NLP or you'll find yourself lost pretty quick. I'm a former SWE turned NLP researcher, so i was recently in your position:)
- gmreads 3y ago
  Curious why and how you did the transition? Rapid progress in this space I don't think SWE is a viable career for next 20 years.
  - extasia 3y ago
    I felt that my software work in the gambling industry wasn't aligned with my ethical beliefs (also family member with gamba addiction..) I really enjoyed the research I did in my undergraduate degree (Physics) and so when my partner suggested I apply to the doctoral training program (it's called a CDT in England) it was sort of a no brainer to shoot my shot - the project sounded interesting and the course was designed for those coming from industry. tl;dr opportunity came up and I jumped ship from SWE to ML research.

RockyMcNuts 3y ago

not sure what 'mixing LLM models' entails but these are maybe some good starting points - karpathy - https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY - https://towardsdatascience.com/beautifully-illustrated-nlp-models-from-rnn-to-transformer-80d69faf2109 https://towardsdatascience.com/beautifully-illustrated-nlp-m... - https://dzone.com/articles/a-deep-dive-into-the-transformer-architecture-the https://dzone.com/articles/a-deep-dive-into-the-transformer-... - https://peterbloem.nl/blog/transformers https://peterbloem.nl/blog/transformers - http://nlp.seas.harvard.edu/2018/04/03/attention.html http://nlp.seas.harvard.edu/2018/04/03/attention.html - https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/ https://lilianweng.github.io/posts/2023-01-27-the-transforme... - https://blog.quickchat.ai/post/tokens-entropy-question/ https://blog.quickchat.ai/post/tokens-entropy-question/ - https://dugas.ch/artificial_curiosity/GPT_architecture.html https://dugas.ch/artificial_curiosity/GPT_architecture.html - https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... - https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf https://d4mucfpksywv.cloudfront.net/better-language-models/l... - https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf - https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf - https://arxiv.org/pdf/2303.17564.pdf https://arxiv.org/pdf/2303.17564.pdf

extasia 3y ago

My answer would be quite specific to what exactly you're trying to achieve. Id be wary of just hacking away without understanding at least the fundamentals of ML + NLP or you'll find yourself lost pretty quick. I'm a former SWE turned NLP researcher, so i was recently in your position:)

gmreads 3y ago
Curious why and how you did the transition? Rapid progress in this space I don't think SWE is a viable career for next 20 years.
- extasia 3y ago
  I felt that my software work in the gambling industry wasn't aligned with my ethical beliefs (also family member with gamba addiction..) I really enjoyed the research I did in my undergraduate degree (Physics) and so when my partner suggested I apply to the doctoral training program (it's called a CDT in England) it was sort of a no brainer to shoot my shot - the project sounded interesting and the course was designed for those coming from industry. tl;dr opportunity came up and I jumped ship from SWE to ML research.