7 ms·

I were 17, I'd learn how to build LLMs from scratch

• khalic 1mo ago
  It’s really disheartening to see how many people don’t know shit about LLMs, by reading the comments… ironic given what OP is trying to say
  • dosisking 1mo ago
    Why is it disheartening? LLMs are a dead-end technology with respect to AI.
    • khalic 1mo ago
      Have you been living under a rock? We might not get "super-intelligences" (if such a thing even exists) from LLMs but they've proven incredibly useful in basically anything relating to text, including code.
      • Grombobulous 1mo ago
        The point is that they’re going to be surpassed by a much better technology that doesn’t have the inherent and unavoidable downsides of LLMs. Like, right now especially in the US you’d probably sound crazy saying that one day nobody’s going to buy a gasoline car, and that EVs are going to completely replace them. But that’s basically an inevitability based on the direction we know technology is going to go, it’s only a matter of when.
        • khalic 1mo ago
          World models will definitely incorporate LLMs or at least text decoders in their final architecture, LLMs are here to stay.
          • Grombobulous 1mo ago
            Maybe this is a dumb analogy but is that kind of like saying “well EVs will still use lubricant! Cars that use oil are here to stay!”? The point being that the way we heavily rely on LLMs specifically right now is likely to cease to be the case, and that in the future they’ll be almost like an irrelevant component.
            • khalic 1mo ago
              There's far more shared concepts between them than you think. You should check the code of both world models (JEPA class for example) and compare to GPT. Many of the tricks stay the same, representation is still embeddings, there is a loss function, etc. The exact architecture will change, but unless there's a new discovery in that area, we've cracked the text component already. We're hitting the limits of LLMs because of the intrinsic limits of text as a medium. But the way we work with text is pretty much settled, fundamentally.
              • Grombobulous 1mo ago
                This is very interesting to me, and I have new things to read into, thank you.
                • khalic 1mo ago
                  anytime, have fun!
• CqtGLRGcukpy 1mo ago
  Twitter mirror: https://nitter.net/paulg/status/2091544343589060625 https://nitter.net/paulg/status/2091544343589060625
• Cheyana 1mo ago
  He bases this decision on all of the experience he has amassed, as a 61 year old man in the tech industry. An actual 17 year old, with 17 years of experience, would not think like this, nor should they.
  • protocolture 1mo ago
    What about a 19 year old? >Whoa. I’m 19 and I trained a 100M language model from scratch. Did a v2 now with a new SFT experiment to see if I can get better results on same size.
  • netcan 1mo ago
    Most. But, that isn't the point. Either way, this isn't really advice for 17 year olds. Pg is thinking out loud about the pathways for founders.
  • vintermann 1mo ago
    Yes. And they almost certainly have a better understanding of their own situation that him. This is not a dig at Paul Graham, the closer anyone is in age, the better they understand what they have to deal with. I'm roughly in the middle between Paul G and the 17 year old, and even though I'm really quite fascinated with zoomer culture and probably come more in touch with it than most (due to relatives in the age range etc.) I realize I have very little idea what it's like to grow up in the world they grow up in.
  • Atouraya 1mo ago
    Naturally 17 year olds don't think long-term like this which is why PG's advice is so useful. It gives them a pathway to follow that they likely wouldn't have reasoned otherwise. I told my much younger brother when he was 12 what programming was and it'd be a great career. He looked into it and within months was writing CLI games. Eventually releasing his own unity 3d game on steam as a teen. Eventually he got into CS and did really well because none of it was scary and new. He parlayed that into role at Meta out of university. My point being, 17 year olds have time to learn new skills and guidance can go a long way.
• Freedom2 1mo ago
  Another great quote by PG. I've been really enjoying his essays recently - truly a great and curious mind.
• sscaryterry 1mo ago
  I'd learn a trade in all seriousness. (Edit: And learn how honest business works)
  • sph 1mo ago
    Depends if you’re 17 with rich parents or not.
    • repeekad 1mo ago
      This only changes whether you are naive enough to believe “honest” business means anything in today’s age. If anything, I worry being honest is holding back smart people who try to compete in a rigged game.
      • embedding-shape 1mo ago
        > you are naive enough to believe “honest” business means anything in today’s age Might be that these people are from outside the US as well, where things like "honest business" is very much possible today, probably most businesses I interact with AFK on a daily business are "honest businesses".
      • sph 1mo ago
        There's two ways to read my comment. I believe you read it along the lines of "you need to be rich to start a business". What I meant was that if you're rich you don't need any trade school or to run a business, you can just sit on your ass living off interest from capital. Only "poor" people need to work for a living.
      • sscaryterry 1mo ago
        Honesty is a big problem today. I'd rather live with myself, knowing I didn't screw anyone...
  • nxobject 1mo ago
    With the hindsight of experience, the remnants of my 18-year old energy go “woah, that’s cool!” at plenty of engineering feats… and my decades-older second brain goes “well d’oh, I could’ve just learned a trade to work on that!” I think the last one was seeing a skilled electronics repairman do surgery on a CT machine controller.
• deleted 1mo ago
  [deleted]
• utopiah 1mo ago
  ... and it would be totally pointless. I mean first that is already what plenty of 17yo are actually doing, because that is what they do at school or in parascholar activities. There are already countless of such tutorials where you can do that in an afternoon. The pointless part though is precisely why Amazon and others are hunting for rare books, all the low hanging fruits have been picked already so just training a bigger model will simply mean burning more energy and money. Sure training a small one for the basic principle is a great pedagogical thing, training another one, medium, then maybe a large one, is also good in term of learning the process and architecture, but one should not expect it to be useful out of that context. Pure players are precisely doing everything they can to corner the market by making their own scale unreachable by others. Smaller players with access to lesser infrastructure are thus betting on different market, e.g. embedded systems. 17yos should definitely build their (L)LMs from scratch and whatever bigger model they can train for free, or for cheap, but they should not expect that to bring them any riches.
  • eptcyka 1mo ago
    Why would 17 year old do something that only brings them money? I do not think Mr Graham here is advocating for the path that makes most money as a result of learning how to train a model. I assume that tinkering and learning about LLMs is what enterprising 17 year olds will do to discover ways they can get a competitive edge or further the SotA with their insights further down the line.
    • utopiah 1mo ago
      he does mention that it would later on be about building a startup, which is about making money
    • latexr 1mo ago
      No wonder the state of everything when 17 year olds are getting pressured to be “enterprising” and “get a competitive edge”. How about learning to be empathetic, respecting your fellow humans, caring for the place you live in, enhancing the lives of others? We shouldn’t be teaching 17 year olds to be greedy, selfish, self-aggrandising blowhards like Zuckerberg, Musk, and Graham. They are not good role models for the future of humanity.
• zombot 1mo ago
  I would not waste my time with yesterday's fad. The next unicorn generator will be something else.
• oersted 1mo ago
  There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM company, and still then it's a struggle. Those few that do train, they spend most of their budget on compute and have relatively small teams. Getting experience in this field requires having access to very expensive hardware to begin with. And the skills will be quite hard to convert into any real value for someone, leading to a decent income, unless you have a ton of funding from patient investors, or you have decent contacts in Bay Area networks to get hired at the right place. With all due respect, paulg is in somewhat of a bubble, this is not congruent with the global situation.
  • kevmo314 1mo ago
    That's like saying the only way to do real engineering is with Google-scale Borg deployments. You can do quite a lot on very little hardware, r/StableDiffusion is a prime example.
    • oersted 1mo ago
      You can do plenty of "real engineering" under normal conditions. But specifically when it comes to LLM engineering, no there's really not much you can do, they are called "large" for a reason. You can play around at small scale, but those lessons you learn will not be very relevant to the real problems in the market. Sure you can gradually climb the ladder by demonstrating your skills bit by bit and getting access to more resources. It has very good prospects if you do manage to push through. But it's a hard and risky path, and you will not be able to get any interesting results for the longest time. For a young middle-class student, it just doesn't make much sense. You can do much more impressive and impactful things with your time without getting into that black hole. I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex. But they can't do much with it without capital or access. Good engineering has never been a bottleneck in this field, it's been all about having access to capital and taking smart but dangerous risks burning it on compute, without much idea of how long you need to keep burning for. There's still no end in sight, some are still managing to convince investors and keep burning, and we are seeing progress, but the business case is still unclear. If you want to get in that game, go ahead, but it's not something I would advice the average young engineer.
• kubb 1mo ago
  It’s crazy how much survivorship bias gets repackaged as generic advice. Wait no it’s not, that was always happening. What’s crazy is that people still believe in it.
• vasco 1mo ago
  Can't this guy enjoy being rich in silence? His takes get worse with every passing year.
  • embedding-shape 1mo ago
    He did get rich by being pretty much the opposite of silent, so I'm guessing you can't just turn off that part, kind of comes with the package ;)
• BoredomIsFun 1mo ago
  I do not think it is a proper thing to do for 17 y.o., unless they are exceptionally mathematically gifted, as proper understanding of how LLMs are trained requires a good grasp of calculus, understanding modern OS and SDE tools for proper implementation of pipeline etc. I'd rather simply write another mnist implementation and check if I really like all that AI stuff at first place. Even then, before going into mature-on-the-way-to-dying tech (LLMs) I'd rather focus on fundamentals - good ols linear models, regressions, stat etc.
  • embedding-shape 1mo ago
    > I do not think it is a proper thing to do for 17 y.o If I'd get a buck every time someone said something like this to me when I was in the 13-18 range, I wouldn't have a ton of money, but it's so very annoying when people tell you this. Regardless if they're "gifted" or not, regardless if you believe in myths like that or not, let children explore what they want to explore, even if you don't understand what it is or why they want to explore that, just let people explore, regardless of age. It was such a terrible experience being a young kid growing up, with so many adults spending hours trying to convince me to stop sitting in front of the computer so much doing whatever; "why are you even trying to learn that stuff, you have to go to school to understand anything of this" and so much other similar trash. Sorry, not your fault and I'm borderline trauma-dumping now, but really sad to see this sort of gatekeeping on HN of all places, age is irrelevant to learning ANYTHING, in my humble opinion at least. Kids, find anything interesting? Jump into it, ignore what adults tell you, and do whatever you feel like, you'll find your place eventually.
    • VoidWhisperer 1mo ago
      Agree with this - started programming through learning scripting in ROBLOX when I was like 12 (this was back like 17 or 18 years ago) and it developed into a life-long passion for software engineering. I am thankful I had people around me (my parents), who were aware enough to realize I wasn't just playing video games and gave me the time I needed on the computer to learn and experiment with programming... This also meant that by the time I was actually offered to take a programming class in school (junior year of HS), I had already been able to self-teach myself well beyond what that class was covering, thanks to just working on random projects that scratched an itch I had at the time, looking up anything I didn't know or understand, and internalizing those concepts over time. In short though, I definitely agree, young kids and teens (and also, frankly, adults too!) should be encouraged to explore things that they have a passion for, without being told 'you need to go to school for this' or 'you cant understand this at your age'
• nvch 1mo ago
  When I was not 17 at the times of GPT2, I decided to not bother with learning how to build LLMs because it’s too expensive for an individual. This escalated quickly.
• onion2k 1mo ago
  I learned HTML when I was 17 in about 1995 and it's certainly taken me on a pretty fun career path. Less technical than LLMs for sure, but 'figure out where the industry is going and move what you're learning to there' is solid advice.
• chris_va 1mo ago
  I am kind of amazed how negative the comments are here, especially on HN. Learning to hack something together in high school using the latest technology (vacuum tubes, radios, microprocessors, web/javascript) has been a common theme in the tech world for generations. With LLMs and online tutorials, this isn't even a difficult suggestion. Do people think learning new tech is somehow wasted effort?
  • embedding-shape 1mo ago
    > I am kind of amazed how negative the comments are here, especially on HN. I can't recall or point out exactly when, but there is a stark before/after moment where the opinions of anything pg went from "Interesting and maybe true in some ways" to what we see today, lots of knee-jerk reactions and hardly any comments about the actual content. Hazarding a guess, I think the moment Altman became the CEO and later during COVID, the sentiment seemed to have been shifting towards what we see today. But this is all based on hazy memory, rather than looking at the data. I'm sure there is a blog post waiting to be written about analyzing the sentiment of comments to PGs articles on HN, and you'll see a shift somewhere.
    • deleted 1mo ago
      [deleted]
    • trentor 1mo ago
      Because at some point in life everyone gets tired of fairytales. He started mending the anecdotes to his content which always rubs people the wrong way.
      • embedding-shape 1mo ago
        So, this comment of yours obviously isn't in the "knee-jerk reaction" category of comments, I suppose? What exactly from the linked tweet(s) are fairytales here? There is hardly any text at all, so strikes me as a comment about previous pg content, but then this would be one of those comments I talk about? Very confusing.
• mateenah 1mo ago
  This might be of use https://github.com/raiyanyahya/how-to-train-your-gpt https://github.com/raiyanyahya/how-to-train-your-gpt
• koe123 1mo ago
  While knowledge is always great, I would encourage people not to seek advice from successful people like this (survivorship bias). Moreover I am not sure it is even good advice? Would you advise a 17 y.o. to learn how transistors work or how to code (i.e. is LLM training the right level in the stack)? LLM training, a discipline where relevant work is already out of reach for 99.999% of budgets really as essential as this post implies?
  • embedding-shape 1mo ago
    > I would encourage people not to seek advice from successful people like this (survivorship bias). Personally I don't see the problem, as long as you're aware there is survivorship bias involved here. What's the alternative really, seek advice from unsuccessful people? That seems worse :) Personally I do both, read about what worked for people, also read about what didn't work for people, then ignore both and do whatever the fuck I want.
    • InsideOutSanta 1mo ago
      > seek advice from unsuccessful people Intuitively, I would guess that they have a better grasp of what made them fail than successful people have of what made them succeed.
      • embedding-shape 1mo ago
        Intuitively, that'd make sense if that unsuccessful person eventually found success, otherwise who knows if they actually picked up what made them unsuccessful in the first place?
        • InsideOutSanta 1mo ago
          I think it's reasonably common for people to understand what makes them unsuccessful but be unable or unwilling to resolve the problem. For me, I know that one major reason for putting an upper limit on my success is my inability to form effective professional relationships. I know I should go to events and talk to people and use these relationships to my advantage, but that's just not something I've ever been good at, and I find it so incredibly unpleasant that I also just don't want to do it.
• weinzierl 1mo ago
  What are the best resources to learn how to build LLMs from scratch for 17 year olds? I have my opinion on this but I'd like to hear the HN opinion, I will just say one thing: If you are starting with little knowledge, like a 17 year old would, letting an LLM explain it to you is a terrible idea.
  • jillesvangurp 1mo ago
    Andrej Karpathy has a great Youtube series on how to build LLMs from scratch. Perfect for somebody who just learned a lot of high school math. Would start there and get busy with some handson python coding.
• felixrieseberg 1mo ago
  I'll use this post as a shameless opportunity to tell more people about a little side project, I made: http://languagemodelbuilder.com http://languagemodelbuilder.com teaches you (in a few hours to days) how to build an LLM from scratch. It's entirely free, without accounts, and without data collection.
  • le-mark 1mo ago
    This is great, thank you!
  • teitoklien 1mo ago
    This is the coolest link I saw on HN the last 2 months. You rock man ! Thanks a ton for building this.
  • vixen99 1mo ago
    I wish this was available for young folk in Romania: As it happens: macOS: 2.70% of desktop operating-system usage in Romania; OS X: 2.55%; Combined Apple desktop share: approximately 5.25%; Windows: 90.65%; Linux: 4.02% or so AI tells me.
    • BogdanilaAI 1mo ago
      There are GitHub repos with the code from Sebastian Raschka's books, which build an LLM from scratch in PyTorch (some examples run fine even on CPU-only setups) using a GPT-2-like architecture, as well as his new book on reasoning models that came out recently. [Off-topic] Your name sounds familiar...
  • spolitry 1mo ago
    Nice! Please post screenshots and videos for people not ready or able to install yet.
  • bentt 1mo ago
    Beautiful work! What prompted you to build this? Pun not intended.
  • 1mo ago
• tayo42 1mo ago
  I don't think individuals have the resources to build an interesting llm. The l stands for large. You need a dataset too. Llms are only interesting because theyre large And it's basically a weekend project to put transformers together in a ML library and train it. The follow up comment,train it to play a game also doesn't make sense? Llms Sony really play games and there are better ml approaches to do that?
• dismalaf 1mo ago
  The amount of people who missed the point here is absurd. He's advocating for learning about how LLMs work. For the sake of learning. Because no one's going to invent the next thing without at least some understanding of the current thing.
• livinglist 1mo ago
  When I was 17 I was building Windows Phone apps, bad decision on my part.
  • muragekibicho 1mo ago
    Incredible counterexample, but oddly relatable. I'd probably have achieved techbro 'post-economic' status earlier if I focused on Android dev instead of the shiny (and new at that time) Xamarin for Windows phones.
    • livinglist 1mo ago
      God I almost invested in Xamarin after Windows Phone got aborted, I did spend a lil time on UWP, but thank god Flutter came out not so long after that. After all these years I learned to stay away from Microsoft tech stack.
  • deleted 1mo ago
    [deleted]
  • mdp2021 1mo ago
    I also did, and had written code for/on other mobile devices before, and did write a large number of even very ambitious software for other mobile devices later. I do not see any past constructive experience as a waste of time.
    • livinglist 1mo ago
      Did you write anything for Symbian or PalmOS before WP?
      • mdp2021 1mo ago
        Yes, absolutely. (Actually, "this friend of mine" goes back to programmable calculators ;) - my very first versatile palm computing devices.)
• mickeyp 1mo ago
  Writing, supervising and training LLMs are now the purview of... even larger LLMs. Optimising CUDA kernels; hand-writing SIMD assembly to speed up data loading; tinkering with your particular brand of DRAM to see if there's anything to gain from optimising for its memory topology and NUMA --- these are now the job of AI. There is very little reason for humans to get all too engrossed in this type of work now, today, with the hope of being good enough at it to command a high salary in 3-5 years. AI can already do it incredibly well, and they can do it persistently and doggedly 24 hours a day.
  • fancyfredbot 1mo ago
    If this is the job of AI then why does AMD struggle to get the best performance on their hardware? Not enough LLMs?
• sreekanth850 1mo ago
  2 years before everyone was doing custom training. What happened to all those today when frontier models itself become more powerful than custom trained ones?
• badgersnake 1mo ago
  If I were 17, but have the money I have now he means.
  • mdp2021 1mo ago
    The times are different. When I was 17, we had to buy records; a 17yo today can listen to the whole of the available produced music, plus interviews and all other uncommon and related material, for free from the comfort of "here and now". Possibilities exist now that did not exist before. Those who do not exploit this are fools.
• simplegeek 1mo ago
  As an aside, for someone interested and who's an absolute beginner, can someone please recommend good resources on how to build LLMs from scratch? Thank you in advance.
  • dakolli 1mo ago
    Check the front page of this god forsaken website a few times a day and you'll get about 10 different posts a day about it.
  • __rito__ 1mo ago
    1. Build an LLM from Scratch by Sebastian Raschka (https://sebastianraschka.com/llms-from-scratch/ https://sebastianraschka.com/llms-from-scratch/) 2. LLM from 0 to Hero, and nanoGPT by Andrej Karpathy
    • lucasoshiro 1mo ago
      I second 1. I'm a newbie in neural networks and I think it's an excellent book! One of my barriers in ML is the resources, I find them overcomplicated or too simplistic without a mid term. It's not the case of this book, everything is well-explained. Neural Networks aren't fun for me, but this book makes it very interesting.
      • __rito__ 1mo ago
        If you are serious about ML, you can email me to get recommendations on resources. (Fully free, of course.)
  • polnoner 1mo ago
    Stanford CS336 is up on youtube from Spring 2026.
    • sonabinu 1mo ago
      I think this is the best structured class out there that teaches how to scale LLMs . Hope the 17 year old knows linear algebra. Building an intuition for the shape of the matrices is important. A lot of understanding the 'building from scratch' means understanding choices like why RoPE instead of the original frequency based positional encoding. Start with Karpathy and then go to CS 336
• haunter 1mo ago
  If I were 17 again I'd prepare to go for volunteering overseas after high school for 1-2 years (plenty of free options in the EU where you might only need to cover the plane ticket). See the world, you learn a new language, help others and then think about what you want to do.
• loveparade 1mo ago
  Horrible advice. This may have been good advice 10 years ago, but not today. There are no positions for people who "kind of understand how toy LLMs work" because so many engineers do these days. Most of the real LLM optimization work is at the edge of research and highly proprietary and not something you could ever do without infra that costs millions. But of course, 10 years ago this wasn't obvious.
  • dakolli 1mo ago
    Yeah he's a moron with a lot of money, that's about it. I'm sure he's said the same thing about various other bags he had bets on throughout the years.
  • aero-glide2 1mo ago
    you don't reach cutting edge immediately. you start with the basics
    • jatins 1mo ago
      I think the answer then is to get a Phd? I mean, one should learn it to satisfy their curiosity and to build things but I don't think it's going to help you career in any significant way.
  • wy35 1mo ago
    What would be better advice for a 17 year old?
    • rglover 1mo ago
      Stay away from the internet and learn how to build a local business instead.
    • Ampersander 1mo ago
      Stay in school, get good grades.
    • PretzelPirate 1mo ago
• dakolli 1mo ago
  What the fuck does this guy know about? I'm sure if we went back through similar statements he's said over the years he's said the same thing about various technologies that are no longer relevant. The guy is a talentless hack who larps as a blogger and his only "redeeming" quality is having lots of money. Owner of Golf Club Company says I should dedicate my life to golf lmfao.
• austin-cheney 1mo ago
  It seems this is poor advice in that it’s suggesting young people should focus on the current problem as opposed to future problems. Focus on the current problem can result in making some money but it will result in making the incumbents more money, which is not disruptive. Isn’t the goal of radical software startups to maximize disruption? If the two current bottlenecks, for this LLM madness that could very well be a bubble, are processing capacity and accuracy (a second processing problem) then what comes next? Isn’t that where young people should be looking or are we just giving up on innovation?
• fancyfredbot 1mo ago
  I'm (more than) twice that age, but I've spent time learning this exactly this from videos by Andrej Karparthy and from books by Sebastian Raschka. I didn't do it because it was useful to me in a practical sense. It's because LLMs are fascinating and I want to know how they work. From that perspective it's been a great experience. I have afirm grasp of the basics. This makes it much easier to understand frontier concepts like compressed latent attention. I can follow the field and understand it. Not sure I would have got as much out of it at seventeen. I have a lot of background and experience which made it much easier to learn. I wasn't struggling with the linear algebra or with python. I already knew pytorch and neural networks. That helped a lot and I covered these tutorials fast and could skip over large sections. A few evenings and the odd weekend day over a couple of months was enough for me. For seventeen year olds the tutorials are good enough to make it possible to learn this but it would have taken a lot longer to understand. On the other hand I would have learned a lot more. I think I would have learned a lot of valuable stuff. However I also think 17 year old me was studying for his A levels and probably this was right choice in terms of maximising future opportunities. I'm not sure I think learning about LLMs instead is sensible. Indeed it might be bad advice. But I can absolutely agree with the sentiment.I think 17 year old me would have wanted to do this too.
  • sarmike31 1mo ago
    Which resources from these two would you recommend? Or just blanket-recommend all their videos/books?
    • len_testa 1mo ago
      I've read Raschka's "Build Your Own LLM from Scratch" book and really enjoyed it. I haven't tried the code yet, but the code from his previous Python ML book worked great.
    • fancyfredbot 1mo ago
      I kind of want to blanket recommend but that's not very helpful I would suggest starting with Andrej Karpathy's YouTube video: https://youtu.be/kCc8FmEb1nY?is=oiDsrBYJg_MUUmoD https://youtu.be/kCc8FmEb1nY?is=oiDsrBYJg_MUUmoD This video is excellent. I'm a huge fan. Also the video is zero commitment and instantly available which makes it a good way to check you are interested. The book by Sebastian Raschka is slightly less accessible but very reasonably priced and the experience of working through a book is a lot nicer than skipping back and forth in a video (for me). Sebastian's blog posts on recent architectures are absolutely great too.
• didntknowyou 1mo ago
  yeah i loved tech so started learning circuity and soldering. but it was a waste of time i made my living learning how to program web applications.
• yapyap 1mo ago
  This is always such a nonsense question-answer thing, asking a person who already succeeded what they would do if they were young. Even worse when they ask themselves.
  • mdp2021 1mo ago
    Support your idea with arguments.
    • mdp2021 1mo ago
      Edit: not "support", I meant: give it a justification. For example, to give the "succeed" part some sense, instead of suggesting assumptions.
• keybored 1mo ago
  YC Combinator guy says that with a time machine he would learn to build the currently trillions-valued or whatever technology. Okay.
  • eleventhborn 1mo ago
    No, that isn't what he said. He said, if he is 17 _now_, with no family, no commitments, no pressure to start a career, he'd invest his time to learn to build LLMs from scratch instead of trying to start a company.
    • keybored 1mo ago
      Okay sure it’s not a “time machine” in that sense. He wouldn’t go back forty years or whatever. And he’s still saying the same thing that I was alluding to.
• a2800276 1mo ago
  If I were 17, I wouldn't be using a social media plattform run by racist neo-fascists...
• wiseowise 1mo ago
  [flagged]
  • epolanski 1mo ago
    Is going to university really that good of advice nowadays? Everybody goes to college nowadays and the average white collar has lots of debt and relatively minor financial benefits over a skilled trade worker. edit: woah, so many people insulted by that. In my bubble and friends, me and another friend are the only people that make very good money compared to non-graduates. Plenty of others opened their shops, went into trades, one learned to tattoo fake eyelashes, one became a (successful) farmer and most make significantly more than the average law/chemist/mathematician/physics/architecture/languages graduates. Sure, the lowest salaries are to be found among the non-graduates too, but I don't see any evidence that graduates make that much more, and that graduating is worth it. Some answers talking about how "formative college is", but my 25 years old friend with her own shop knows more about real life, business and economy than ivy league MBAs.
    • wy35 1mo ago
      It depends on the university and your goals, I suppose. Many people are surprised to learn how affordable elite colleges are if you genuinely need financial aid. I had no idea -- was pleasantly surprised when my alma mater took over 80% off of my tuition.
      • tripledry 1mo ago
        Indeed, would the advice be good if university was free? Where I live it's basically free and I still sometimes regret not going into the trades. But I suspect this feeling might mostly be a "grass is greener" thing.
    • borski 1mo ago
      If you’re talking purely financial benefit: yes, it is still good advice, as most white collar jobs still require a degree. If you’re talking about a place to mature, around others who are at a similar phase of life, also yes. It is where most people meet their cofounders, for example, even if they don’t found anything until much later.
• laurentiurad 1mo ago
  This might be a helpful resource: https://laurentiugabriel.github.io/token-town/ https://laurentiugabriel.github.io/token-town/
• hugodan 1mo ago
  hackers and painters and kids and ROI and startups and capitalism and the destruction of nature and old guys with money talking like they know better in fascist social networks
  • mdp2021 1mo ago
    > fascist social networks Sometimes if you want to be heard you must go to the public square, whatever the flags there. The rest of the post is unintelligible: add some verbs at least.
• num42 1mo ago
  If you’re 17, you might learn hands on knowledge, like tacit knowledge in areas such as lathes, precision engineering, metrology, and other very niche fields. You can also try climbing and explore arts like music, painting, and drawing. And, of course, spend some time in nature.
• keiferski 1mo ago
  I think pg answered the question as “what I’d do as a project” and not “what I’d do as a career.” So the critical comments are kind of missing the point, IMO. I don’t see why learning how LLMs work is a bad project for a 17 year old. Optimizing your entire career and the next decade+ of your life on LLMs? Yeah, probably not ideal. It’s almost always a bad idea to make long term decisions based on current trendy things. And since everyone is using this topic to give their ideal advice to 17 year olds, my advice as a mid-30s guy: seriously consider becoming highly skilled at a specific thing, and don’t be scared off by the idea that it’ll take 5-10-15 years to get there. When you’re 17-25, the timescale of a decade seems infinite. But it’s really not, and a decade spent “exploring and keeping your options open” sometimes just ends up with you being pretty decent but not amazing at a lot of random things. Sometimes I wish I had just become a carpenter, chef, electrician, etc. – a specific skill set that leads to mastery over time, rather than the endless exciting-new-thing hamster wheel of working in tech.
  • ElProlactin 1mo ago
    > And since everyone is using this topic to give their ideal advice to 17 year olds, my advice as a mid-30s guy: seriously consider becoming highly skilled at a specific thing, and don’t be scared off by the idea that it’ll take 5-10-15 years to get there. And this is a problem with modern society today: expecting 17 year-olds to know what they want to do professionally for the rest of their lives, and to focus heavily on professional development aligned to that. By that age, I think it's not uncommon for individuals to have interests and perhaps even dreams, but a well-defined career focus that serves as the foundation of an actionable skills development plan? Nah. That just isn't common and I'd argue not desirable. 17 year-olds should be exploring their interests, enjoying early adulthood, learning valuable lessons in the social realm, etc. Not training themselves to become compliant little worker bees.
• qsera 1mo ago
  LLMs are incredibly boring to me as a technology..Not in terms of what it can do, but how it works.
  • bananaflag 1mo ago
    Same for me. Neural networks in general. I first studied them in 2011, and I was like what? Just a bunch of partial derivatives? I keep looking at AI to check if now it's something else but it keeps being gradient descent. Okay, it's great that you can perform miracles using gradient descent but that doesn't make it captivating in any way.
• swayson 1mo ago
  Curious, what would you do if you are a 40 year old?
• bewareofscams 1mo ago
  Venture capitalist suggests everyone to become his future employee, just as he has been (successfully) doing for his whole life.
• leothetechguy 1mo ago
  As if you couldn't learn what LLMs are at any Age. The core technoology is pretty basic, developing a rudimentary understanding for why the individual parts work as well as they do is tricky.
• maximgeorge 1mo ago
  [dead]
• sureglymop 1mo ago
  I would (and am) going into MLOps. Not just the general infrastructure/systems administration but how to do inference optimization, caching, quantization, memory pinning, vfio passthrough of gpus etc.
• hashmap 1mo ago
  i am a small fan of pg, nevertheless i find this to be an exceptionally good take and it is strange to me to see so much piling on to this one in particular here. learning about llms is not useful so that you can make llms later, you want to learn about it so that you can work on next generation architectures. llms before long i imagine will be left in the dust by ebm / physics oriented models especially that can have an embodied understanding of the world. but a lot of things you learn about them are transferable by doing something like this
• GardenLetter27 1mo ago
  Easier said than done - where do you get the B300s from? Better to start working with harnesses, evals, statistical analysis, etc. - where you don't need the huge hardware for pre-training etc.
• luciana1u 1mo ago
  [flagged]
• kypro 1mo ago
  I get the sense things have changed a bit since I graduated and there are lot more jobs in AI outside of academia these days, but it's still a very different field from other SWE pursuits, and it's not really accessible to hacker-minded people. Learning AI isn't like learning HTML in the 90s then expecting to get a job at a tech company building websites. You can't just "learn how to build LLMs" and expect a frontier lab to hire you so I'd argue this is rather bad advise. Additionally, unlike web development in the 90s you cant really do anything interesting yourself... All of the interesting/useful stuff will require huge amounts of compute and data so there isn't even much point in learning to start your own thing either. As someone whose built many of NNs from scratch (hand written code, long before the days of LLMs), it's more or less useless knowledge if I wanted to work in a frontier lab or do anything interesting in the field. I also think anyone thinking about going into a field which is basically a crossover of CompSci and Maths is absolutely insane right now. Even if you think there is a place for CompSci and Maths post LLMs, there's almost no chance anything you learn today will be relevant to the skills required in say 5-10 years.
• frabcus 1mo ago
  Oddly, I just remembered I did the nearest possible thing to this when I was 17... back in 1991. On an Amiga, I took various public domain text documents from cover disks and counted the probability of the next word given the previous word. Then spat out random sequences of words from it and printed them out. It was called "Splurge". Basically a very very simple single layer statistical language model. Some of the sentences were randomly not bad sentences, which seemed amazing at the time! That kind of thing (and Core Wars and Tierra etc) did lead me to getting a job at an artificial life startup at the end of the decade. But that was in turn about 10/15 years too early (no GPUs). There's some lesson from this about timing, but honestly I've gained the most as a person when I did something that was fun, ethical and gained an audience. A tricky combination.
• Chance-Device 1mo ago
  I think a problem a lot of people are grappling with here is that due to LLMs and AI generally, it’s basically impossible to predict what the future will look like or what jobs will still be around. I’d probably say something like: do something you enjoy and seems like it might be useful, but accept that the pace of change may mean that whatever you study ends up being irrelevant. Whatever solution there ends up being to this, it’s not going to be one that an individual 17 year old can implement. We’re past the point where individual good and bad choices matter that much to economic outcomes.
  • maciejzj 1mo ago
    I would advise any somewhat ambitious 17 year old to avoid tech and get into healthcare if they can stomach human interactions and bodily fluids. Sure, it is not all sunshine and rainbows, but there will still be plenty of work helping people who are ill or elderly. Even in the worst-case economic scenario, medicine will be a more socially rewarding and stable life path.
    • jgraham 1mo ago
      This seems like a much more interesting question to me. Telling other people's children what to do is easy and basically doesn't have any downside to being wrong. With your own children things are a bit different. So: what are people here with school age children telling their own kids about the future? If their kids ask, what kind of careers would they encourage them to pursue, assuming they have the skills and interest? When I was last in the Bay Area, maybe about a decade ago the bookshops were full of titles like "Python for Preschoolers" (I exaggerate, but only slightly). Clearly at the time a lot of people working in tech thought that cultivating an interest in programming was going to be the path to being a successful (by some metric) adult. Is that still the case?
      • greenowl 1mo ago
        We've wronged an entire generation by pushing "learn to code".
• deepllm 1mo ago
  As someone who's at a similar age and was interested in learning how to do this, there just aren't enough resources to do so. Most LLM research is in the form of academic papers, and there isn't any 'popular' way to learn these things, and besides that all said research assumes you have a B200 cluster ready to go. If you have weaker hardware (say an 8GB nVidia GPU, which is what I have) you're going to be limited to fine tuning small models or torturing yourself working the GPU for days per iteration trying to run things like https://github.com/karpathy/nanochat https://github.com/karpathy/nanochat, which is hardly an educational experience. Renting cloud GPUs is expensive, and at this age the most I could muster up for experimentation is probably $100 or so, which only gets me 25 or so hours on a B200 which just isn't enough. So why would I bother myself with this if I'm already at a disadvantage because of not having access to the right hardware and when surely there are better ways to spend my time? I concluded the only way to learn and be competitive is by finding work at an AI lab somehow (not happening at 17), or studying ML at the right university. Putting this in contrast with programming, I learned coding when I was 8, and it was incredibly stimulating to learn because you can quickly iterate and there were thousands of books and YouTube tutorials that dumb everything down and teach you fundamentals. All you needed was a $300 computer, and you can learn nearly anything you want, without being gatekept from this or that because you don't have enough vRAM / an sm_100 GPU.
  • rvba 27d ago
    [dead]
• _fizz_buzz_ 1mo ago
  Why are people so negative about this? It feels like a fun project and at 17 the stakes are not really high. Something one could easily do on summer break in a couple of weeks.
  • deleted 1mo ago
    [deleted]
  • smcg 1mo ago
    what hardware can a broke teenager get access to in a couple weeks? is this really better than learning how to program?
• deadbabe 1mo ago
  If I were 17, I’d learn how to invest and build financial literacy, and plot potential growth of my networth throughout my life, before even thinking about a career. Then smoke a bowl.
• charlieyu1 1mo ago
  I read a bit about how LLM works, but as a hobbyist it is pretty frustrating that I won’t be building anything useful without throwing a lot of money at it
• kotaKat 1mo ago
  Somewhere in rural America is a 17 year old that doesn't even have working plumbing in their house still. I'm sure they'll get right on powering up their computer from the hamster wheel, Paul. My heart goes to all the kids out there that didn't get the fair shake let alone fair access to tech that gets these condescending "learn to code/learn to LLM" bootstrappy talks from rich pricks that don't know what life really can be like for a lot of American kids out there.
• shell0x 1mo ago
  What would you do if you’re 30 years old now? As a platform engineer being based mainly out of Australia/Hong Kong, opportunities seem to be getting less unless targeting high frequency trading or banking. It seems like building a startup with the help of some AI tools might be the best bet.
  • criddell 1mo ago
    > It seems like building a startup with the help of some AI tools might be the best bet. I'd only recommend taking that path if you already have your first paying customers standing by or are extraordinarily good at marketing.
    • shell0x 24d ago
      Thanks, that’s a fair comment. I definitely have no marketing experience :)
• mattlutze 1mo ago
  A lot of people here are responding to the message but not to the meaning. It would be a good idea for young people to deeply know how these programs work. Not so that they can spend their career building them, but so that they can approach the next class of problems we'll all start trying to solve, with intuition all the way down to the weights and underlying mathematics. And also, to develop a healthy intuition of when "Just LLM it" will not be the right choice. "Build an OS" wasn't a common university project because we were all expected to go out and work on Windows, but because understanding the bare-metal firmware for a computer helps you deeply understand how to intuit building for a whole class of problems.
  • Joel_Mckay 1mo ago
    Depends, the assumption things are predictable always negatively affects both Market Bears and Bulls alike. Indeed, if credulous folks look to the world expecting people to bestow success upon them... than the disillusionment with reality will hit their savings harder. The Shrek movie market correction correlations are undeniably funny, and a new film is due July 2027. OpenAI may be going public in the next few months while still losing $2.25 for every $1 of customer revenue, and with 6 other firms sharing over $4Tn in debt disclosed to investors in a footnote. There is only one direction things can go at the Peak of inflated expectations. Popcorn ready. =3 https://en.wikipedia.org/wiki/Gartner_hype_cycle https://en.wikipedia.org/wiki/Gartner_hype_cycle
  • skydhash 1mo ago
    > "Build an OS" wasn't a common university project because we were all expected to go […] helps you deeply understand how to intuit building for a whole class of problems. Sure, but it was for a specific degree with a syllabus that taught you the foundational knowledge. It was not expected from the law students to learn how to build one.
  • cpt_sobel 1mo ago
    > understanding the bare-metal firmware for a computer IMO this is still relevant, everything surrounding the LLMs needs such a vast infrastructure that I don't know if I would find it more useful to learn the maths behind ML than CS
• dofm 1mo ago
  I think this is weird advice. Learn how to make language models from scratch, yes. But learn how to use them, in the context of other machine learning tools, on very small hardware. When the bubble bursts (and I still tend towards thinking it could burst rather than be deflated in a manageable way), the focus will be on uses of AI that are not like the hyperscalars' products. People will still be interested in useful AI being added to small things — assistive technologies, home security, garden monitoring, their phones and smartwatches, robotics. Instead of reductive, reactive make-an-anthropic-competitor advice like this, what about advising 17 year olds to focus on broad, integrated, helpful AI — or on going back through eighty years of history to look at AI projects that failed and reassess them?
• TrackerFF 1mo ago
  If you're 17, and seriously curious about how modern day AI works, you might as well just sit down and look at a couple of courses on linear algebra + calculus, machine learning, deep learning, and more LLM specific deep learning. Those courses will teach you how to go from writing your first perceptron to a MVP language model. But also so much more.
• antfarm 1mo ago
  Why do intelligent people still use X? Thanks for the xcancel.com link!
• sylware 1mo ago
  Neural net maths are hardly above scientific high-school level.
• DannyBee 1mo ago
  I get this is basically advice for young founders and entrepreneurs, but i would ignore that request and encourage 17 year olds to spend time trying to find a happy medium between work and life. Being a super rich and an unhappy workaholic, or a super-impressive engineer who wakes up one day at 45 and realizes they regret wasting half their life (I ran into way too many of these) is a much worse fate than "not being rich from your startup" and working a relatively regular job while feeling fulfilled and happy by more than just work. Especially in the US, which is uniquely bad at this and encourages people to work themselves to death, mental health and work life balance are much more valuable things for 17 year olds to focus on than finding good startup ideas. In case you think i'm being a bit dramatic, let's look at the state of 17 year old mental health in the heart of Silicon Valley: "The City of Palo Alto and the Palo Alto Unified School District approved a funded contract to place 24/7 human security guards and monitors at all four local Caltrain grade crossings, including the Churchill Avenue crossing directly adjacent to Palo Alto High School." (in case it's not obvious, it's because of suicides by high school students) The 17 year olds do not need advice on better startups, and this situation will never get better if we focus our advice on how to be better at work instead of how to be better at life. This will require redirecting the conversations.
  • rglover 1mo ago
    Thank you. They need human contact, not more "sit in a room alone and get stressed as fuck for little ROI" tech bullshit. Unless the kid has a genuine, self-motivated interest in learning these things (a great, positive thing that should be nurtured), they should file pg's advice under "ok boomer."
• moniosi 1mo ago
  certainly better than wasting time with harness and agent workflows that will become irrelevant at the next evolution, same thing happened with 'prompt engineering'
• bsenftner 1mo ago
  Terrible advice. If I were 17, genetics and bio tech at the next frontier, with opportunities to be more than another corporate drone. AI is a lot of bureaucracy and nepotistic who ya know already.
• DarmokTanagra 1mo ago
  telling a 17 year old to get into tech right now is horrible advice, literally telling them to get at the back of a line with a better part of a million more experienced people in it.
  • dosisking 1mo ago
    Shaka, when the walls fell...
  • Grombobulous 1mo ago
    Telling them not to get into tech is also terribly reactionary advice. The truth of the matter is that we don’t yet know whether tech roles will be eliminated or if they’re just going to follow previous innovation breakthroughs where “one person producing way more work” makes software even more of a desirable industry to be involved in. There really isn’t a very strong correlation between tech industry hiring strength and AI as of yet. Various studies that are out there haven’t even witnessed AI workflows contributing more than modest gains in software engineering efficiency. I.e., being able to write code 20-40% faster isn’t a seismic shift in the industry where everyone is getting laid off tomorrow and we’re all replaced by software. Even with the questions surrounding the current job market, it’s still an incredibly good ROI career compared to so many other jobs out there. For example, in my local area you can get a job as a registered nurse working nights in the emergency room and only make ~$115k. I make almost double that telling an LLM what to do from my house in my pajamas during the day with less time spent in university. Even if tech roles lose half their salary to automation pressure it’s still a really good gig.
    • DarmokTanagra 1mo ago
      “only make $115k” This right here is why nobody is shedding tears for the massive employment crisis in tech. You make double that shilling ai slopware while they work nights saving lives. I would tell a 17 year old that the world will always need nurses, same can’t be said for guys sitting in their pajamas burning tokens. What is happening now in tech has been a long time coming, and it can’t happen fast enough.
• agentultra 1mo ago
  Most 17 year olds I know don’t want anything to do with AI and see the entire industry as an existential threat. Nothing wrong with learning the theory and understanding the papers. Getting to that point you’ll have to get your fundamentals down. Might be an interesting exercise. But as a future? I guess we’ll see. I suspect the next financial apocalypse will determine if there is one. Another AI Winter that may outlast all others so far.
  • draugadrotten 1mo ago
    > Most 17 year olds I know don’t want anything to do with AI The few exceptional individuals will innovate what the millions will benefit from. At 17, the mother of Isaac Newton removed Isaac from school and tried to make him a farmer. We all know that wasn't his destiny.
    • agentultra 1mo ago
      There’s no such thing as destiny. You could also walk out your front door and get hit be a meteor tomorrow.
• yomismoaqui 1mo ago
  I'd just build an agent, it's easier than you think and you'd learn a lot about the "magic" of LLMs.
• ozereray1 1mo ago
  [flagged]
• chasd00 1mo ago
  It’s probably wise to learn how to build one to understand what you’re dealing with. However, if I were 17 I would lean how to apply an LLM to a problem instead of strictly building one.
  • doginasuit 1mo ago
    Why not both? The best thing about LLMs is they make virtually any domain approachable.
• HarHarVeryFunny 1mo ago
  LLMs are the new compilers. I don't think you can really call yourself a developer unless you at least have an idea how to build a more complex software project like a compiler, and maybe have built a toy one either at uni or for fun. It's not clear how long this LLM age of AI will last (to be replaced by something better), but nowadays any developer should at least understand the basics of ANNs, and more than just the "hello world" of a cat vs dog CNN. An LLM/Transformer is maybe the equivalent of a compiler in that regard - something that we all use and is complex enough to present a bit of a challenge. You should at least understand the basics of how an LLM is built, and maybe building a toy LLM will/should become the new Comp. Sci. degree toy compiler replacement.
• angoragoats 1mo ago
  Let’s make the xcancel link the actual link, please!
• Uptrenda 1mo ago
  I genuinely don't think telling young people to do anything tech-related is good career advice. We don't even know if entry level roles will ever come back. The situation couldn't be worse for these roles. PG thinks that some random teenager will build a startup and get rich from it, or some shit. Like get real, man. Completely out of touch, tech bro who hasn't worked a real tech job for the last like 20 years... Remind me how many startups succeed again, Paul? What about the market dynamics for LLMs and how one might "secure compute"?
  • MaxHoppersGhost 1mo ago
    These guys (VCs) entire livelihoods rely on tens of thousands of young people throwing away their early years of lives trying to get super rich. Only .01% even see moderate success but PG and co don't care and continue to pump the dream and urge folks to waste their lives to try and make them rich.
• xtiansimon 1mo ago
  I wonder at the “worlds” Mr Graham envisions, and what is their cardinality. Is this the only 17 yro reimagining, or is there an army of 17yro, of which this LLM curious persona is but one?
• jappgar 1mo ago
  LLMs will take away all jobs except yours, LLM maker. Keep at it, you're safe!
• greenowl 1mo ago
  Yeah, no way. I'd move to the middle of nowhere and work multiple jobs on a farm and in construction. Learn how to grow food, and build things. Meet the farmer's daughter, and marry her. Then, buy my own land, grow my own food, and build my own things.
  • y-curious 1mo ago
    Slight problem with step “buy my own land”. Your working 2 blue collar jobs and paying rent are incompatible with this plan.
    • skylurk 1mo ago
      > Meet the farmer's daughter, and marry her.
  • brainless 1mo ago
    I already do this. I live in a small village where there isn't even a wired Internet connection (wireless only). I work full-time with LLMs, on own product ideas and client projects (all LLM led). I started investing in farms, have 50 pigs and 100+ chickens now. We are planning to grow to 100 pigs and 2000 chickens in a year. We will start growing Shiitake mushrooms in a few months too.
• 1aqhg16 1mo ago
  Didn't his swiss watch essay say he'd essentially leave the industry because there will only be bloat from now on?
• brainless 1mo ago
  Why is it important to train LLMs or even fine-tune them? LLMs have proven their point, costs are crashing, and there are more of them than most companies need. The real value is to unlock meaningful insights and directions from existing data that is there inside companies. I live far outside any tech city, so maybe I do not understand. But working with LLMs full-time, building for clients and tons of own experiments, I see no value in building on LLMs.
• dirkc 1mo ago
  Would building an LLM from scratch imply writing code by hand?
• HellDunkel 1mo ago
  Seek advice from nice people that you actually know instead of rich people on the internet.
• geremiiah 1mo ago
  I'm usually a fan of pg, but this post is ignorant of modern AI technologies. Building an LLM from scratch is both a trivial and a useless exercise. There's probably in the range of 5000 github repos doing exactly that. What makes LLMs work is scale, and what makes engineering and training LLMs hard is also scale. And scale is not something you can achieve in your garage. If the goal is to understand LLMs deeply, one would be better served by either joining one of the big AI companies or doing a PhD. And to be honest, I think this journey should have been started 5 years ago, because right now there's too much competition.
• jimnotgym 1mo ago
  If I was 17 again I would sack off work and study and focus on chasing the opposite sex, without the angst I had at the time. I don't know a single person who regrets having had too much sex when they were young. I would not build an llm, too hard, too expensive to run. Learn how easy life is if your morals allow you to grift money of vcs into your own funds and retire.
• fnoef 1mo ago
  If i were 17, I'd try to distinguish who to take advice from, and would definitely learn that VCs have interest to spread a specific agenda in their message. Also, I would get drunk and have as much fun as could, as the misery of working under the treat of being replaced by AI, would simply kill any desire to live past 25.
• jimmyjazz14 1mo ago
  Really? because I feel like LLMs are already pretty much a commodity, not to say their won't be advances in LLMs but I don't see the models themselves being all that ripe for disruption the way that the web was and such. I'm guessing chip design and manufacturing processes will be more important than models in the future.
• qarl2 1mo ago
  I remember when I caught PG on reddit arguing with some guy who'd said something mean about him. He didn't reveal who he was. But you could tell from his history - his first post was from before reddit opened to the public. Good times. I wonder if you can dig that out of the historical reddit database. I'd like to see that again. I love how everything is recorded now.
  • qarl2 1mo ago
    > I love how everything is recorded now. I mean, except when it's censored. That part is a shame. HEY! Is there a tool that tracks the censored comments? I bet there is. And if there's not... that will be a delicious new project for my agents.
• Tomlmmrs 1mo ago
  This essentially comes down to choosing the search for substance (here in the form of technical depth) over short term gratification and quick wins in life. I think this kind of mindset should be taught way more in school so that people really appreciate learning a subject deeply.
• oldsklgdfth 1mo ago
  I appreciate the sentiment and I'm pretty curious how I could train an LLM, even are really basic one from 5yrs ago without nuts hardware. That said, I a good starting point for a 17yo is reading about perceptrons[0], then the basics of neural networks[1] (ex. 3-layer perceptron) then writing a program to train a 3-layer perceptron and classifying the MNIST dataset[2] - a dataset of characters. This can anywhere between a day and a week and you will demystify the basics of neural networks and work your way forward with more advanced contemporary concepts. Fun fact: any multi-layer perceptron neural net can be reduced to a 3-layer perceptron network. [0] https://en.wikipedia.org/wiki/Perceptron https://en.wikipedia.org/wiki/Perceptron [1] http://geeksforgeeks.org/deep-learning/neural-networks-a-beginners-guide/ http://geeksforgeeks.org/deep-learning/neural-networks-a-beg... [2] https://www.kaggle.com/datasets/hojjatk/mnist-dataset/data https://www.kaggle.com/datasets/hojjatk/mnist-dataset/data
• pranshuchittora 1mo ago
  Can any1 share any resource to learn that skill. I want something that has been tried by you. I can too search on the internet...
• iluvcommunism 1mo ago
  [dead]
• softwaredoug 1mo ago
  My kids CS teacher asked me what they should do after AP CS. Previously they had a class where they'd build apps for other teachers. Like tracking when clubs are, etc. But now that's become easy for teachers to vibe code themselves. I suggested they shouldn't prereq this class on CS. Heck invite anyone in interested in "building things" and they can get practice at building apps for other people / themselves. Maybe that becomes a gateway TO CS - people who want to learn how things work under the hood. I suggested post CS class for the CS people should probably be building an LLM or something :)
• Lerc 1mo ago
  I'm not sure what 17 year old me would have done with YouTube tutorials for everything under the sun available. I'm much older and less wise now, but I still afforded myself the opportunity to follow karpathy's tutorials to build a LLM from scratch. Got to play with a few ideas. Seen similar ideas turn up in frontier model work, which is quite gratifying. There are so many ideas to try. Currently playing with autoencoders that takes A and B and produce latents A', B', and C'. Reconstruction of A is from A' and C', B is from B' and C' The idea is if C' can be made to improve both outputs, it must store as much information as it can about what is common to both inputs.
• rmoriz 1mo ago
  The only realistic approach is to train on a limited data set which is probably less usable than the comibnation of a custom RAG + one of the many available LLMs. Also many people/kids don't have access to proper "productive" systems anymore, since the whole computing and electronics industry shifted to make "consumer"-devices like smartphones or laptops made for netflix, gaming and spotify. Breaking the barrier to build a custom system, install linux (or developer tools for Windows, MacOS) is already a complex AND costly task. It was just way simpler in the late 90s and 00s to get something working.
• jadar 1mo ago
  I get it the sentiment behind the post…but it has some “let them eat cake” vibes though.
• maxgiraldo 1mo ago
  I'm making an assumption here, but I think paulg is implying that "learning LLMs" today is like the equivalent of "learning computers" in the earlier days. We could even divide civilization into two eras: Before Transformers (BT) and After Transformers (AT).
• helloplanets 1mo ago
  What about doing abliteration, weight pruning, representation engineering, etc, directly to open LLMs instead? Building an LLM from scratch has a hard split between a tutorial project you can complete in a weekend (that's useless for actual usage) and then a solid 1km high brick wall if you want to create anything actually useful from scratch. Modified open models have a very active community around them, without the need to look much further than Hugging Face.
• perrygeo 1mo ago
  I've come to accept that some people, when faced with impressive technology, simply want to use it. They genuinely have no interest in understanding how it works. Lately, its even become fashionable to shame people for trying to understand ("you still read code? gross, you know AI can do that for you"..) I will never understand this mentality, to let yourself be so dependent on something you don't understand at all is to live like a child. But it is very common. I doubt too many 17 year olds will bother even trying to understand what an LLM is, let alone build one from scratch.
• bentt 1mo ago
  All the negative comments here are really missing the point. At 17, you should be building your foundation. Kids that can make a custom CPU, or retrofit an old car with an electric motor, or screw around with nuclear energy... these are kids that are doing it just to see if they can. It's not about jobs, it's about curiosity and stretching limits and seeing what you can do, who you can be.
• flanked-evergl 1mo ago
  This is about as intelligent as say "If I were 17, I'd learn digital electroinics". You would waste your time. Sure, in theory it's useful, in reality it's not that useful.
• AIorNot 1mo ago
  I think collectively we should all stop listening to Mr Graham.. He capitalizes on greed and hype but with a soft, sober and thoughtful voice so as to lull you with rationalism and now 20 years of his “disruption” has mostly ruined modern society and a whole generation of techies have been led astray into trying to “change the world” is the world of today (minus the magic technology really any better than 20 years ago?) - Good for him and his Tech Bros, bad for the rest of society
• greenraven 1mo ago
  An LLM isn’t hard to make - the training data is hard to get and prepare. The big companies stole the data. The average person can’t do that
  • eddieroger 1mo ago
    Not arguing how big companies got data, but there is plenty of public domain knowledge and content available to someone who wants to get it.
• Kuyawa 1mo ago
  You can build the program, but to train it is another beast, billions of docs, images, videos, which a mere mortal doesn't have access to Second in line, build your own agent, that's more in our ballpark, then customize it to your needs, both virtual and physical
• CurbStomper 1mo ago
  [dead]
• sur_lal 1mo ago
  This is the ultimate builder’s mindset. Tinkering with the hardest technical problems—even for frivolous things like games—always yields the highest return on curiosity. Time to go back to the fundamentals.
• vibe_that_works 1mo ago
  I really don't understand Paul's reasoning here. Does he predict more scarcity on the model-level? That layer seems to be almost a commodity now + training is damn expensive. If you are really 17, my advice is to identify use-case for AI (ideally relevant for businesses) that work most of the time and find ways to make them work pretty much every time. AI reliability is the scarcity right now.
• elorant 1mo ago
  I built from scratch a sparse text embedding model trained on a 13T token corpus. Not the same as an LLM because there’s no transformer in the mix, but still I learned a shit load of things in order to solve all sorts of problems that emerge when you try to access big datasets and daily update tables with billions of rows. But if it wasn’t for a specific use case that I tried to solve I don’t think that whatever knowledge I gained could be utilized in the market. Sparse models are a very small niche and most people I’ve come across with similar knowledge are in academic circles, not business related ones. So even if LLMs are all the rage these days I doubt the demand for people who know how to build them is that high. Someone who knows how to setup an open weight model and expose an API might be more valuable to a company these days.
• mondainx 1mo ago
  On first glance, this is good advice and in general I'd give the same for this age group. With age, you'll see more of these scenarios come up and if you have experience and foresight, you can provide direction that may prove fruitful to young people. My son and his friend asked "what should we look into and learn?", about 15 years ago, I told them Python and Java; Python because of versatility and cryptocoins; Java for the long term stability in the job market. I despise Python personally, but I could see its potential and still do; especially for AI. At the end of the day, kids have more time than money and its great experience for them to get their hands dirty and find out what they might be interested in; its a long life.
• deleted 1mo ago
  [deleted]
• 11101010010001 1mo ago
  I thought pg was trying to live forever. Has he learned LLMs from scratch?
• deleted 1mo ago
  [deleted]
• nla 1mo ago
  A 17yo can train a small GPT this weekend. nanoGPT is a few hundred lines. Understanding why it works is the part that takes a decade.
• matheusmoreira 1mo ago
  If only we could train LLMs on commodity hardware, right?
• semiquaver 1mo ago
  https://deeplearningwithpython.io/ https://deeplearningwithpython.io/ is a decent (free) read for anyone looking to follow this advice.
• deleted 1mo ago
  [deleted]
• tonymet 1mo ago
  Paul Graham wants you to work for him, not be him.
• sailorganymede 1mo ago
  The core point here is that AI is a massive thing (at the moment) so it's probably a good idea to understand it deeply. Not sure why people are so worked up about it.
  • quikoa 1mo ago
    > (at the moment) That's why. That future is uncertain. So why gamble your future on something that's popular at the moment for something that could change completely a year later?
• peter_d_sherman 1mo ago
  Paul Graham: >"Someone asked what I'd do if I were 17. I'd learn how to build LLMs from scratch..." That's funny, Paul Graham, because if I were 17 again, I'd learn how to program in LISP. https://www.paulgraham.com/rootsoflisp.html https://www.paulgraham.com/rootsoflisp.html https://www.paulgraham.com/iflisp.html https://www.paulgraham.com/iflisp.html https://www.paulgraham.com/hundred.html https://www.paulgraham.com/hundred.html (And/or other LISP derived languages... Clojure, Scheme, Racket, TinyScheme, etc.) I guess "the grass is always greener..." as that old expression, that old "chestnut", goes... :-)
• psadri 1mo ago
  Paul G is not writing this for a general audience of your run of the mill “engineer” hoping to be employed by someone. He is writing it for future founders. What knowledge / skills you need to develop today to be well positioned to have a startup worthy insight when you are 24.
• nixonaddiction 1mo ago
  i was trying to build llms from scratch at 17. failed miserably because i did not know linear algebra. ended up in a different but adjacent field. when chatgtp got big suddenly there were so many people doing llms and i didnt want to compete like that. so much of what was happening was hype, and that really turned me off. am back to building llms from scratch, but like, its a journey teaching myself all the theory on top of my job. i have decent fundamentals, but i need a better grasp of all the advancements in the field in the past 5 years before i would feel comfortable designing anything. baby steps, essentially. am working on better understanding all the layers of an ai while implementing a rag on my local model as an experiment. 17 year olds should learn whatever theyre interested in but need fundamentals in order to do anything advanced.
• hmokiguess 1mo ago
  17 is such a fantastic age to be free and experience the world, you won't get that much of an advantage as these FOMO groomers are selling you into if you start now versus later. If you are 17, go be yourself, whatever that is, in whatever way you want that to be, but do it so authentically and fully. Be unapologetic about what you love and what motivates you, and pursue that with passion and commitment.
• megadragon9 1mo ago
  I think it's more about the scar tissue (a.k.a. intuition) when building LLMs from scratch than whether it's transferable to job search. Maybe the person will decide they don't like LLMs and not develop that into a career, or maybe they become a researcher in another field because that's a better way to solve problems inherent in LLMs.
• micromacrofoot 1mo ago
  If I were a billionaire, I'd learn how to give away a lot more of my wealth.
• excentricus 1mo ago
  Yeah sure, learning the internals of a technology that’s being actively developed will probably teach you things that remain useful for awhile even if your learning outcomes could end up being different than what he’s implying. Whether that makes a good long-term investment is quite debatable. IMO, LeCun’s take (at the end) is far more forward-looking since it aims to gain more insight into what could come next based on what we know about the current state of the art. All that being said, the extent to which these people capitalize on our tendency to be blinded by the halo effect is incredible. A constant stream of bite-sized aphorisms… This is especially at a different level for early startup figures who happened to be in the right place at the right time and usually did little more than digitizing mundane, traditional day-to-day processes. Yet they’re treated as geniuses and prophets, with people hanging on their every word as though everything they say contains some deeper wisdom. PG and the like often strike me as broken clocks and they’re still profiting from having been very early players in the game, who had good instincts for commercialization and capitalism. ———— LeCun’s reply: > I would try to figure out why LLMs can write my essays but not clean my bedroom. Then I would study topics in college and grad school that could help solve that problem. I'll figure out a set of methods and architectures beyond LLMs that can quickly learn to perform physical tasks as efficiently as humans and animals. That last item is also what I would if I were 30, 40, 50, or 66 years old
• delis-thumbs-7e 1mo ago
  When I was 17, I wrote poetry and learned how to play guitar. I’m really happy that I did. That’s what I would do now, were I 17. I feel slightly bad for people who didn’t.
  • nphardon 1mo ago
    Same. But if I could go back, I would add skating as well. I slept on that one and spent too much time trying to be a hacker.
  • tomhow 1mo ago
    Sure, but isn't this a bit of a sneer? “I feel slightly bad for people who didn’t” has an air of superiority about it, and seems like a veiled swipe at those who lacked whatever qualities your 17-year-old self had. The guidelines have several different ways of asking us to avoid commenting like this. As to the substance of your comment, there's nothing stopping a 17-year-old from writing poetry, learning guitar, and also learning about how LLMs work, if they're so inspired. Indeed I'm sure pg would encourage it, and that he did the equivalent of all those things himself when he was young. There's plenty of evidence of that in his essays: he wrote short stories, published a scandalous school newspaper, played soccer, and studied fine art and philosophy. See: https://www.paulgraham.com/worked.html https://www.paulgraham.com/worked.html https://www.paulgraham.com/nerds.html https://www.paulgraham.com/nerds.html
    • delis-thumbs-7e 1mo ago
      > there's nothing stopping a 17-year-old from writing poetry, learning guitar, and also learning about how LLMs work, if they're so inspired. I assume the people at HN are intelligent enough to understand this without me having to explain the most obvious banalities to them. The point is why you would do anything at 17: > if they're so inspired You said it yourself. The Twitter post, to me, reads like career advice or something in that vein. One has plenty of time to run the wheel like a lab rat later, but at that very point in life you have possibility to find the thing that truly inspires you, that is its own motivation.
• kashnote 1mo ago
  Don't really understand why people are saying this is terrible advice. If you're 17, you should be learning everything. At that age, your brain is a sponge and your energy levels are the highest they will ever be. None of us know what the future of work, education, or AI is going to look like. But your best bet is to become a life-long learner. Be it LLMs, musical instruments, physics, or business.
• coolThingsFirst 1mo ago
  That's applicable only if you live in the US. if i were 17, i'd do these things: 1) LC until mediums 2) Calculus & linear algebra even if i understood nothing i'd just stare at vectors and derivatives.
• chews 1mo ago
  When you see anthropic job postings that say the job may not be available in a year... it really doesn't inspire much confidence. I say if you're 17, learn to weld, solder, and fit pipe...
• incanus77 1mo ago
  > Notice that what I would not do is try to start a startup. Instead I'd build the foundation of knowledge to base a startup on later. There's also this little-known concept called learning things for learning's sake and not always trying to capitalize on it.
• 4lx87 1mo ago
  If I were 17, I'd be going to parties, music festivals, chasing girls, and enjoying my youth. But sure, make the kids even more depressed by telling them they need to learn how to build an LLM so they can get a job working themselves to death to make Paul and friends rich.
  • dang 1mo ago
    This community is filled with people who were obsessed with microcomputers or (depending on which generation) websites during their teenage years. HN was created by and is managed by people of this type. No doubt we're a minority, and the type you describe is more common, but if you're implying there's something wrong with teenagers being intellectually curious about technology, I can't help but think of the way "nerds" used to get put down and shamed in the past.
• LarsDu88 1mo ago
  I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundational understanding, and they might become dated... like knowing how a CRT monitor work. Something of historical interest and good for learning, but not crucial to being well-rounded. There are other types of models like diffusion models right now that are showing more efficiency and have a higher ceiling for improvement. Understanding math and fundamentals are more important.
  • lern_too_spel 1mo ago
    Yann has been consistently wrong about the limits of LLMs.
    • LarsDu88 1mo ago
      He has, but I think he's right in the sense that there will be better models. It would be premature to think that the current frontier model architectures will last into the future. It would be like making DVD engineering a standard part of high school curricula
      • handoflixue 1mo ago
        > It would be like making DVD engineering a standard part of high school curricula I fail to see how PG suggesting that a 17 year old, today, should study something... is in any way a parallel to formalizing it as part of the curriculum. I would, in fact, expect PG to recommend something completely different in 10 years. The whole point of hacking is to understand the world around you, today
    • carbondrone 1mo ago
      Well good for them, I think their comment is heavily suggesting they're dissatisfied if said limits, and want to see them broken.
  • aaron695 1mo ago
    [dead]
• drblast 1mo ago
  Mr. McGuire: I just want to say one word to you. Just one word. Benjamin: Yes, sir. Mr. McGuire: Are you listening? Benjamin: Yes, I am. Mr. McGuire: Plastics. Benjamin: Exactly how do you mean? Mr. McGuire: There's a great future in plastics. Think about it. Will you think about it? --- I love this scene because it so perfectly captures what it's like to be young and given advice, however well-meaning, by an older generation living in a world that no longer exists for the young. And it's ambiguous and trite enough to be essentially useless even if the underlying idea isn't terrible.
• wildcatqz 1mo ago
  A teenager in China actually did that and got a paper accepted at ICML. The interview podcast is in Chinese, but you can ask AI to summarize it https://www.xiaoyuzhoufm.com/episode/6a8472b95aeb2a5712e8de78 https://www.xiaoyuzhoufm.com/episode/6a8472b95aeb2a5712e8de7...
• jcmoscon 1mo ago
  I think learning how to build your own agent harness framework from scratch and really understand each part of it, what are the modern components that a good agent harness are using these days, is more valuable than learn how to build LLMs, but that depends on what you want to do with your career.
• chente 1mo ago
  When I was 17, I made bad cartoons and was in a band. If I were 17, I'd spend more time learning music and design theory. I also learned PHP at this time, but that was low on the list, friendships came first.
• swozey 1mo ago
  Paul has the same problem just about every tech obsessed engineer has (including myself), he thinks everyone else loves computers too. They don't. At all. I''m a self-taught ex-bartender and I can't tell you how many grown adults in the service industry I've tried to get into computer stuff and they had 0 interest. And younger people I meet don't even own laptops. I had a genz/millennial cusp friend who wrote all her college papers on her iphone.
• analog31 1mo ago
  Ah nostalgia. At 17, I learned how to write programs in BASIC on a mainframe.
• bizstrategy10 1mo ago
  [dead]
• barbazoo 1mo ago
  > Someone asked what I'd do if I were 17. I'd learn how to build LLMs from scratch, and then train ones as powerful as I could with whatever hardware I could get access to. He's seeing a future for models running on everyday hardware just capable enough to do what the use case requires.
  • wawaWiWa2 1mo ago
    That is the idea of ai Not agi A specific model for each problem. Agi is the academia solution Software is the practical solution
• wanderingpixel 1mo ago
  Does anyone have a roadmap ?
• mrwaffle 1mo ago
  I'd say, if you were 17, you'd probably be better off learning how to socialize or at least gracefully approach and relate to people your own age. That's probably the greatest challenge of the near future, sadly.
• danbrooks 1mo ago
  I had the same inspiration! I spent a days reviewing the lecture notes for CS336: Language Modeling from Scratch - and then trained a nanoGPT-esque model in PyTorch. I'd recommend trying it for those who are curious. Computational bottlenecks become much more intuitive when you've looked at the overall process.
• claysmithr 1mo ago
  The problem is tech moves so fast that by time kid is adult llm are outdated
• steele 1mo ago
  I assume in this hypothetical Cornell just gives away degrees to passersby.
• mcculley 1mo ago
  What corpus would one be advised to use for training? Should one limit oneself to legally downloadable documents or should one follow the examples of Altman, Amodei, and Zuckerberg and use LibGen and Anna's Archive?
• hughw 1mo ago
  If I were 77 I'd do the same
• amelius 1mo ago
  And what resources would you use for that? (Besides the obvious nano gpt)
• Isaackoz 1mo ago
  If I were 61, I'd learn to not be so out of touch with the younger generation
• dirtikiti 1mo ago
  if i were 17, i'd learn a trade.
• paretolaw 1mo ago
  "...and then train..." and those dummies in openai and google wasting billions, while all they need is some 17 year old and few pizzas
• ebbi 1mo ago
  As someone who doesn't work in this field (I'm in boring old Accounting/Finance), but would like to learn this field (both to future-proof my skillset, but also because I have an interest in it), would it be wise for me to learn math? Is it something that's going to be fundamental for future technologies? I always plan to learn, but end up in a death spiral feeling like I'll invest so much time and energy only for the puck to have moved somewhere completely different. Would gladly accept any advice :)
• KerrAvon 1mo ago
  it is once again time to balance out paulg sycophancy https://idlewords.com/2005/04/dabblers_and_blowhards.htm https://idlewords.com/2005/04/dabblers_and_blowhards.htm
• tdhz77 1mo ago
  Interesting advice I will be interested in how this advice holds up in the future
• ls-a 1mo ago
  [dead]
• metalspot 1mo ago
  LLMs are just one small part of computing. it is fine to learn how they work but they are nothing magical. if you want to build a real working system using LLMs that actually makes money you still have to learn everything else. but i think his point here is that learning LLMs at 17 means hacking together some home brew rig on shitty consumer hardware and if you learn all of that you will be in a much better place than learning how to use chatgpt to automate a spam campaign for your b2b saas startup.
• zephyreon 1mo ago
  Sorry but Paul is just so out of touch with reality. I’m tired of seeing everyone on HN evangelize this man. He is very intelligent and has done some really great things but times have changed and we need to let the next generation think for themselves. They are increasingly anti-AI. Maybe that pans out, maybe it doesn’t. If I were 17, I’d pursue my dreams — whatever those may be — and learn how to connect meaningfully with my peers. Perhaps that’s naive of me or perhaps that’s what humanity needs most right now. I’d rather let the 17 year old version of me make that choice for themselves. I’m sure I’ll get torn to pieces for this but it’s frustrating to continually witness people treat a single person’s prose as the Word of God.
• exacube 1mo ago
  plug: studymode.ca/ml is an AI-generated cirricculum + chapters/lessons that teaches ML by building practical projects the lessons have svg diagrams, ai chat inline (google docs), etc apologies if this comes off as slop, but its been working great for me.
• dongliliu 1mo ago
  [dead]
• arceister 1mo ago
  One of the replies: "Why would you reinvent the wheel rather than making it better?" "To learn how wheels are made" Banger reply.
• imagetic 1mo ago
  https://github.com/huggingface/tau https://github.com/huggingface/tau
• finaard 1mo ago
  Funnily enough, I recently started on a series of demos and documentation to explain to my 10 year old daughter how LLMs work - in this case in the context of "don't use LLMs for looking stuff up if you figure out how to circumvent the school blocks yet again, or if you do, at least verify its output", but also figured it can be useful in general. Plus, if I get a 10 year old to understand that there's a slim chance I can get customers to understand that as well.
• syngrog66 1mo ago
  better priorities: save democracy; counter rise of disinfo & fraud; cure cancer; carbon climate crisis
• cbondurant 1mo ago
  Had to reread this a few times to properly parse what the hypothetical was. My first read was "if you could redo your life from age 17 what would you do". To which "figure out how to make an LLM" would be an insane answer. I think his advice here is maybe... a year or two too late to be good advice. I can't foresee the job market for machine learning experts being better than it is now in 5-6 years time when said hypothetical 17 year old would be most ready to start career hunting. Either the bubble is gonna pop and the market will be flooded with laid off AI talent. Even if I'm wrong about there being a bubble at all, I still think that in 5 years time the tech will just have matured to the point of diminishing returns on refining existing architectures. Plateauing until some PHD comes up with something as ground breaking as attention.
• panzi 1mo ago
  If I'd were 17 I'd choose a different career.
• nostrademons 1mo ago
  Yann LeCun's response is interesting. I've heard that the transformer architecture underlying LLMs is actually really effective for robotics, but progress is a lot slower here because instead of just scraping the web and Anna's Archive, you have to outfit workers with motion-capture suits that record their every move, and then have them physically do enough jobs that you can gather good training data on how the job is done. This is a little challenging when no blue-collar worker wants to be replaced by a robot. Long-term there's probably a lot more potential, and much bigger markets, in robotics than in LLMs, but it will really take the long term to get there.
• AtlasBarfed 1mo ago
  You're not 18 so you can steal as much of media as you want for training and not face real legal consequences
• 1970-01-01 1mo ago
  To build LLMs from scratch, one must first learn how to build a computer from scratch because microchip prices are crazy. Oh and before that, Sagan says you need to make a universe. :)
• bicepjai 1mo ago
  We should remember the fact that LLMs had emergence doing one mundane thing. Now think what we would have if we do our mundane thing of learning something for the sake of just learning… Emergence.
• crossroadsguy 1mo ago
  If I were 17 again, I would probably be single-mindedly focused on getting laid adequately and getting into something non-STEM. Or at least I hope so.
• quicklywilliam 1mo ago
  This is what I did, tho I’m 41
• n4pw01f 1mo ago
  Hot take - but I’d learn to build CNNs from scratch. We are only just now scratching the surface of convolutional neural networks capabilities
• lwhi 1mo ago
  I think Paul is assuming he'd also be wealthy. Wealth inequality is a huge factor in our ability to excel.
• alun 1mo ago
  I had the same thought as PG (although I'm considerably older than 17). The playing field in software development has been leveled by AI, which led me to wonder how you can get ahead of the curve. My current answer is to understand both AI and data intensive systems at a deeper level than 90%+ of people. For the AI part, I recently started working through this CMU course on Modern AI which teaches you how to build an LLM from scratch. It was posted on HN a few months ago and has been great: https://modernaicourse.org/ https://modernaicourse.org/ If anyone has other recommendations for beginners, please share them!