karpathy

10 ms·

61.
by karpathy 4y ago
I am working on a video lecture series that will step through it and "spell it out". Without it even this code can be a bit opaque for someone who is new to the field and e.g. uncomfortable with n-dimensional array manipulations o
62.
by karpathy 4y ago
Hah funny to see this on HN, it is a relatively old project but one that I continue to love and still work on. I was trying to train a GPT one day and discovered that available implementations were quite complex, spread across many files, a
63.
by karpathy 4y ago
Data is an excellent place to look at to get a sense of where the model is likely to work or not (what kinds of images), and for prompt design ideas because, roughly speaking, the probability of something working well is proportional to its
64.
by karpathy 5y ago
I agree with this. Their ranker is really good and has improved dramatically the more I watch and like. And it's not even fully garbage content, a good amount of it is interesting, useful, inspirational and educational. A rabbit hole
65.
by karpathy 5y ago
Surely You're Joking is one of my all time favorite books, for sure.
66.
by karpathy 5y ago
the plot thickens :) https://twitter.com/YuleHou/status/1407395412575592453
67.
by karpathy 5y ago
I count myself very fortunate that I find the word "work" very confusing.
68.
by karpathy 5y ago
https://twitter.com/karpathy/status/1407378320551923718 :) But more seriously, I just really love learning and worked on this on the side, in small increments in between the cracks, and purely from interest for fu
69.
by karpathy 5y ago
For the record these are some blatant & false FUD attempts.
70.
by karpathy 6y ago
You do not need any matrix calculus for deep learning and micrograd ( https://github.com/karpathy/micrograd ), which implements backpropagation for neural nets in 100 lines of code, is a proof. Everything else is just ve
71.
by karpathy 6y ago
I found that the psychological impacts of only creating a deficit via diet are over long term not negligible, and this gets worse as you get leaner. At low body fat percents (low teens and below) you'll start to feel hungry all the tim
72.
by karpathy 6y ago
Yes it’s one of my favorite thoughts, which is why I pinned it :)
73.
by karpathy 6y ago
Fun idea. GPT @ Home :D. Scatter of the inputs would be very cheap as they are tiny LongTensors (sequences of indices), but the Gather of the gradients seems like a bottleneck. These models can be quite large. Maybe each worker only communi
74.
by karpathy 6y ago
minGPT is actually quite performant too, the min refers to breadth of supported functionality (eg the absence of support for various additional conditioning, exotic masking, masked LMs, finetuning, pruning, etc).
75.
by karpathy 6y ago
Addicting and dangerous. When I was in the thick of it my nights were restless as my brain continued to spin on my factory and various ideas for its refactoring. I also suddenly can't not see our global economy and its gradual automati
76.
by karpathy 6y ago
Yes natural sugar is just barely better than added sugar (due to the fiber it's typically mixed in with). The apples we eat today are candy bars, they just happen to grow on trees. Trees shaped extensively via artificial selection for
77.
by karpathy 6y ago
Agree! (and wrote a bit more in a comment above). I think IF was the high order bit that enabled me to achieve a long-term consistent deficit. I used to track it with Zero but stopped later because there isn't much to track. I strictly
78.
by karpathy 6y ago
Agree, I didn't mention this in the post but I've become a big fan of intermittent fasting, which is a fancy way of saying "skip breakfast". I only eat from 12-8pm and most days from 12-6pm. I also found that even if I w
79.
by karpathy 7y ago
I find the simple idea appealing that I don't have to spend too much energy on who to reward or not reward. I reward anything that I find valuable enough to spend my finite attention on. If companies started to receive sufficient profi
80.
by karpathy 7y ago
As one datapoint over last ~3 weeks or so I've accumulated 20ish BAT (~$5) from a combination of 1) ads shown to me, 2) tips from people to my twitter, 3) automatic monthly contributions from other users, presumably based on them spend
81.
by karpathy 7y ago
I was also really hoping they would slip this into the update too, but looks like not yet unfortunately :( The whole login security ecosystem across the web is in a state of high entropy flux. Some websites still don't offer 2FA, some
82.
by karpathy 7y ago
oops, thank you, fixed :)
83.
by karpathy 7y ago
Very likely not for the positions above! (they all focus on Python)
84.
by karpathy 9y ago
There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that se
85.
by karpathy 9y ago
I gave this talk a while ago to a small group of attendees. It was not recorded (I saw some ask below). It's based on a document I wrote a while ago called "You suck at writing AI" (never published). The basic argument was th
86.
by karpathy 9y ago
Tesla | Autopilot Vision | Onsite | Full-time We're looking for strong ML/CV/Roboticists for the Tesla Autopilot Vision team. We're also eager to explore strong C++ generalist programmers who want to get into autonomous
87.
by karpathy 9y ago
yep :(
88.
by karpathy 9y ago
Agree, I'm pretty sure the world could really benefit from an awesome "OpenAI-2" that is more of an application-driven AI for high-societal-impact organization. We're more focused on mitigating the existential risk assoc
89.
by karpathy 9y ago
I'm not saying NLP/Vision is not important, but we approach these modalities in a very specific way. When we "work on NLP" it looks something like https://blog.openai.com/learning-to-communicate/ or
90.
by karpathy 9y ago
Our focus at OpenAI is on AGI research. Many of us believe that Vision/NLP research falls into the category of AI applications, and does not inform insights into how to achieve generally intelligent agents. Instead, the approach is to

More ›