5 ms·
As someone who has no technical knowledge of Llama or any of the LLM work, from conceptual understanding to technical implementation, is there any benefit to si
by mattfrommars 2y ago
As someone who has no technical knowledge of Llama or any of the LLM work, from conceptual understanding to technical implementation, is there any benefit to sit down and go through this from start to finish? Or is effort better spent somewhere else?
Like a roadmap, do A, do B And finally go through this in the end.
- krainboltgreene 2y agoOnly do it if you want the illusion of LLM's to be shattered. Suddenly every day you'll see two to three highly upvoted links on HN and be unable to keep your eyes from rolling.
- exe34 2y agothat's like saying if you study real neurons your illusion of the human mind will be shattered.
- krainboltgreene 2y agoI have a friend who is a neuroscientist and he disagrees immensely.
- exe34 2y agoDoes s/he believe in free will too?
- krainboltgreene 2y agoI'm not going to have a conversation with a programmer about free will on HN. It's great that OpenAI made you think about this stuff for the first time, but I read my science fiction as a teenager.
- MuffinFlavored 2y agomy opinion: it quickly gets into "the math behind LLMs" that make no sense to me words i understand but don't really get: weights, feed forward, layers, tensors, embeddings, normalization, transformers, attention, positioning, vector There's "programming" in the plumbing sense where you move data around through files/sockets and then there's this... somebody without a math background/education... very unlikely you'll understand it. it's just skimming python and not understand the math/library calls it makes
- lithos 2y agoIt's Signals and Systems in part to get the notation and explanation. MIT has the course online for free. (Though probably a little more general than what you need, since the class is also used to prep electrical engineers for robotics and radio communication).
- gradascent 2y agoIf you want to gain familiarity with the kind of terminology you mentioned here, but don't have a background in graduate-level mathematics (or even undergrad really), I highly recommend Andrew Ng's "Deep Learning Specialization" course on Coursera. It was made a few years ago but all of the fundamental concepts are still relevant today.
- antonjs 2y agoFei Fei Li and Andrej Karpathy's Stanford CS231N course is also a great intro to the basic of the math from an engineering forward perspective. I'm pretty sure all the materials are online. You build up from the basic components to an image focused CNN.
- deleted 2y ago[deleted]
- danielmarkbruce 2y agoNot as a starting point. Google and find the examples where someone does it in a spreadsheet. It's much more approachable that way. You are going to find it's not that complicated.
- gohwell 2y agoSounds interesting. Do you have a link?
- gricha2380 2y agohttps://news.ycombinator.com/item?id=39700256 https://news.ycombinator.com/item?id=39700256
- deleted 2y ago[deleted]
- joenot443 2y agohttps://bbycroft.net/llm https://bbycroft.net/llm This was posted on HN a while ago and led to some great discussion. Myself and others agreed that this type of stateful visualization was _way_ more effective at conceptualizing how an LLM works than reading code or stepping through a debugger.
- arjonagelhout 2y agoHere is the concerning post and discussion: [0] https://news.ycombinator.com/item?id=38505211 https://news.ycombinator.com/item?id=38505211