4 ms·
Side track: There is so much going on this space. I wish there was a chronological flow of a machine learning scenario/story with all the terms being introduced
by deskamess 2y ago
Side track: There is so much going on this space. I wish there was a chronological flow of a machine learning scenario/story with all the terms being introduced as we meet them (data, pre-training, training, inference, mixture of experts, RAG). Like someone walking me through a factory explaining what happens at each stage (like Mr Rogers used to do). Most of the time I do not know where the terms fit in the big picture. When I first came across pre-training I thought it was something done to the data before training happened but it was actually another training.
- blennon 2y agoI highly recommend Harmonious.ai written by the folks at AI2 Incubator who are brilliant and reputable.
- berkes 2y ago> Most of the time I do not know where the terms fit in the big picture. Nor do the majority of "AI" experts and consultants that I see on LinkedIn, Twitter or in podcasts. The S/N ratio is very low in this field. Just pick some documentation from "industry leaders" like Langchain and see that not only is it already and always outdated, it sometimes simply contradicts itself. In the "blockchain hype" this was similar, so I guess it's a trait of the hype train.
- highwaylights 2y agoTotally agree with the above, although I’m not sure that documentation on tools like Langchain is a reflection of the hype in the way social media is. I think in that case it’s just a reflection of the pace things are moving at.
- pixl97 2y ago>Nor do the majority of "AI" experts... I mean yes, this is what a rapidly expanding field looks like that's probing the boundaries of its problem space. Kind of like following physics in the early to mid 1900s. Different classes of problems have barely been tested against each other, much less fully explored themselves. In some ways it reminds me of the earlier days of the internet when progress was still very rapid.
- highwaylights 2y agoI feel your pain and excitement in equal measure. It can be hard to know where to start with some of these concepts, especially so given that a lot of recent developments (e.g. RAG) are developing so rapidly that there’s unlikely to be a reference book you could refer to anytime soon that would be current. That said, I do find that documentation is getting better depending on where you look. The documentation for higher level tools like LlamaIndex is a good starting point for understanding the concepts (not so much in terms of explaining the concepts, but showing where they fit into the overall picture, then you can deep-dive elsewhere on the different parts). YouTube has always been a mixed bag of very little solid information in a sea of non-experts trying to attract clicks for the latest trends, so it’s not a great starting point IMHO.
- phkahler 2y ago>> YouTube has always been a mixed bag As an outsider but avid reader of this stuff linked from HN, I would recommend the channel 3blue1brown. He's got several NN and AI related videos, and the couple I've seen were pretty good. https://www.youtube.com/watch?v=aircAruvnKk https://www.youtube.com/watch?v=aircAruvnKk
- renonce 2y agoYeah but the other side of the coin is that they only explain the very basic concepts that are already settled for several years, not any of these "latest trends"
- saddabbas 2y agoI recommend checking out Machine Learning Q and AI by Sebastian Raschka
- deskamess 2y agoBased on the TOC it seems pretty deep. At this point I am looking for something high level.
- jstummbillig 2y agoPeople waste too much time building out stuff that is really bad in AI right now. Of course, everything is, but instead of taking on the task of patching that up, the better approach would be to pretend there will be something that is a lot better than GPT-4 in the near future (because there will be) and design a differentiated product under that premise.
- sunir 2y agoThat’s an interesting idea. What do you mean? I understand the prompts as a service is short term… but what is a long term product you see?
- redblacktree 2y agoI may be misunderstanding your meaning, but I'm not convinced that "prompts as a service" is short term. I think we'll see a number of apps pop up that will be essentially that, i.e. powered by a generative AI, but with a great UX. Not everyone is good at prompting, and although it is a skill many will develop, packaging up great prompts in niche problem areas still looks like an area of opportunity to me. I'm not talking necessarily about chat experiences, but apps that can, as an example, maintain task lists for you after consuming your incoming communications.
- objektif 2y agoHow many times did you have issues communicating with your spouse because of prompting issues? And how did you resolve it? Why would you need an extra layer here?
- malnourish 2y agoWhy do PR firms and copy editors exist?
- redblacktree 2y agoI don't understand your comment. I was talking about apps built on LLMs where the prompts aren't given by the users, but the LLM is still an important part of the functionality.
- reece_omahoney 2y agoCheck out Lilian Weng's blog https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/ https://lilianweng.github.io/posts/2023-01-27-the-transforme...
- deskamess 2y agoAs far as definitions go that was actually pretty good.
- snorkel 2y agoStrongly recommend watching Andrej Karpathy’s “Lets build GPT-2” videos on YouTube which dives into an actual PyTorch implementation, then download the code and study it carefully. Then study “Spreadsheets is all you need” to see what the internal data structures look like.