joemckenna

3 ms·

1. 1.▲
   by joemckenna 3y ago

   Leading up to ChatGPT, virtually no model innovation was needed. Just some minor tweaks to activation functions and the order in which operations were completed in the feedfoward layers. So they were essentially trying to trademark the inno
2. 2.▲
   by joemckenna 3y ago

   Whereas vanilla scaled dot-product attention forces outputs to fall in the convex hull of V, your proposal allows them to drift into the convex hull of V+{0}. It's pretty unlikely that the origin is already contained in the convex hull
3. 3.▲
   by joemckenna 4y ago

   I was only matching queries against parts of the YouTube auto-generated captions. In theory, this should work for queries that are guests' names because Lex states the guest name in the beginning of the episode. But, in practice it doe
4. 4.▲
   by joemckenna 4y ago

   Sure! https://github.com/joepatmckenna/podsearch . Thanks!
5. 5.▲
   by joemckenna 4y ago

   For some videos, embedding on third-party sites has been disabled. Previously filtered, but revised now. Thanks!
6. 6.▲
   by joemckenna 4y ago

   As a side project, I created a search engine for the Lex Fridman podcast. It uses https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... to encode the YouTube auto-generated captions. Feedback is greatly appre
7. 7.▲
   Lex Fridman Podcast search engine(lexpodsearch.com)

   3 points by joemckenna 4y ago|8 comments