3 ms·

MIT Course: Generative AI for Constructive Communication

164 points by tim_sw 3y ago

akmittal 3y ago
Is it available to be taken remotely at own pace?
- hislaziness 3y ago
  From the course page - Classes are in person on Wednesdays from 2:00PM - 4:00PM Eastern Standard Time in E14-526
- pratap103 3y ago
  Doesn't look like it but the slides and recommended reading are posted and should be able to give you some color.
mercurialsolo 3y ago
The linked courseware (required readings) is open and pretty good to get started.
revskill 3y ago
I want a course that can simulate LLM using Small language model instead. It could run fast on a CPU machine, for the sake of learning.
- benob 3y ago
  [1] is a good start, although if you want to train from scratch on CPU, you'll have to downscale as transformers need quite a bit of data before they learn to use position embeddings. For example try a single-layer RNN on Shakespeare texts [2] or a list of movie titles from IMDB [3]. You'll have to fill in the blanks because things have evolved quite a bit since those were used for language models, but you can find some tutorials [4] and examples [5]. [1] https://jaykmody.com/blog/gpt-from-scratch/ https://jaykmody.com/blog/gpt-from-scratch/ [2] https://github.com/demmojo/text-rnn/blob/master/datasets/shakespeare.txt https://github.com/demmojo/text-rnn/blob/master/datasets/sha... [3] https://datasets.imdbws.com/ https://datasets.imdbws.com/ [4] https://pytorch.org/tutorials/beginner/chatbot_tutorial.html https://pytorch.org/tutorials/beginner/chatbot_tutorial.html [5] https://github.com/pytorch/examples/tree/main/word_language_model https://github.com/pytorch/examples/tree/main/word_language_...
brokenbutton 3y ago
Is it possible to view the lectures online?
- gnicholas 3y ago
  It doesn't appear to have an online component, nor is there any mention of OCW, so I'm guessing not. Perhaps they'll record it and post later, though. I was curious to see why this was posted if it's not something that is open to remote participation or viewing, but I guess it was posted for the syllabus/readings?

164 points by tim_sw 3y ago

akmittal 3y ago

Is it available to be taken remotely at own pace?

hislaziness 3y ago
From the course page - Classes are in person on Wednesdays from 2:00PM - 4:00PM Eastern Standard Time in E14-526
pratap103 3y ago
Doesn't look like it but the slides and recommended reading are posted and should be able to give you some color.

mercurialsolo 3y ago

The linked courseware (required readings) is open and pretty good to get started.

revskill 3y ago

I want a course that can simulate LLM using Small language model instead. It could run fast on a CPU machine, for the sake of learning.

benob 3y ago
[1] is a good start, although if you want to train from scratch on CPU, you'll have to downscale as transformers need quite a bit of data before they learn to use position embeddings. For example try a single-layer RNN on Shakespeare texts [2] or a list of movie titles from IMDB [3]. You'll have to fill in the blanks because things have evolved quite a bit since those were used for language models, but you can find some tutorials [4] and examples [5]. [1] https://jaykmody.com/blog/gpt-from-scratch/ https://jaykmody.com/blog/gpt-from-scratch/ [2] https://github.com/demmojo/text-rnn/blob/master/datasets/shakespeare.txt https://github.com/demmojo/text-rnn/blob/master/datasets/sha... [3] https://datasets.imdbws.com/ https://datasets.imdbws.com/ [4] https://pytorch.org/tutorials/beginner/chatbot_tutorial.html https://pytorch.org/tutorials/beginner/chatbot_tutorial.html [5] https://github.com/pytorch/examples/tree/main/word_language_model https://github.com/pytorch/examples/tree/main/word_language_...

brokenbutton 3y ago

Is it possible to view the lectures online?

gnicholas 3y ago
It doesn't appear to have an online component, nor is there any mention of OCW, so I'm guessing not. Perhaps they'll record it and post later, though. I was curious to see why this was posted if it's not something that is open to remote participation or viewing, but I guess it was posted for the syllabus/readings?