14 ms·
Five Years of Tinygrad
- mika6996 10mo agoWhat would tinygrad replace if they continue to proceed like this?
- spiderfarmer 10mo agoPotentially PyTorch and Tensorflow.
- cyberax 10mo agoI think it has great potential for deployments on edge systems.
- vileain 10mo ago[flagged]
- spiderfarmer 10mo agoThere are lots of bubbles where Elon is still king. Those bubbles are often void of deodorant.
- vileain 10mo agoBased on the response it appears HN is one such bubble.
- spiderfarmer 10mo agoElon spent billions to buy a platform and promote his tweets. He spent billions more to create a tweaked AI model that praised him like a mad king. He only has to spend a couple thousand a month to influence comment ranking on HN.
- mycodendral 10mo agothe value is the directness, not implied origination not everyone cares about playing voldemort
- vileain 10mo agoWhat is so aggrandizingly 'direct' about calling the system you are attempting to improve 'dumb'?
- mycodendral 10mo agoyou have a more direct way?
- dang 10mo ago"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- vileain 10mo ago[flagged]
- mft_ 10mo agoAs someone with no skin the game in either direciton, it's interesting that you're so triggered by the mere mention of someone's name. Is it really so difficult to parse out different aspects of a person's existence, such that one could simultaneously appreciate some of the positives (the existence and success of SpaceX, making Tesla a success against the odds, [popularisation of] some engineering leadership concepts) while also acknowledging the negatives (of which, more latterly, there are many)? Or must the sheer emotion always cloud out rational judgement?
- pa7ch 10mo agoVery weird to market this as subscribing to "Elon process for software" I remember when defcon ctf would play Geohot's PlayStation rap video every year on the wall.
- spiderfarmer 10mo agoI hate it when ‘inspirational’ quotes are attributed to the person with the largest audience and not the people who came up with it, like in this case, the engineers at Lockheed’s Skunk Works.
- piskov 10mo agoYou would be amazed https://en.wikipedia.org/wiki/TRIZ https://en.wikipedia.org/wiki/TRIZ
- derrida 10mo agoOk this wins my “best wikipedia article of December” award
- DoctorOetker 10mo agoCheck out this cacophony of an infographic: https://upload.wikimedia.org/wikipedia/commons/b/bf/40_principles_of_TRIZ_method_720dpi.jpg https://upload.wikimedia.org/wikipedia/commons/b/bf/40_princ... No wonder the soviet union collapsed! When you need Moscow to explain the hammer sign on your corded drill, its too late for damage control.
- ramesh31 10mo agoIt's an apocryphal quote. "A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away." - Antoine de Saint-Exupéry
- vessenes 10mo ago
- deburo 10mo agoSo this is all python? I bet Chris Lattner probably approached them.
- zephen 10mo agoLattner is a smart guy, but I think Mojo might be the wrong direction. Time will tell. History has not so far been kind to projects which attempt to supplant cPython, whether they are other Python variants such as PyPy, or other languages such as julia. Python has a lot of detractors, but (despite some huge missteps with the 2-3 transition) the core team keeps churning out stuff that people want to use. Mojo is being positioned "as a member of the Python family" but, like Pyrex/Cython, it has special syntax, and even worse, the calling convention is both different than Python, and depends on the type of variable being passed. And the introspection is completely missing.
- tucnak 10mo agoHonestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while since I looked at Lux.jl for XLA integration, but I reckon it could be incredibly useful. As long as LLM's and RLVR training thereof should continue to improve, we may be able to translate loads of exiting Pytorch code eventually.
- eli_gottlieb 10mo ago> For example, Bayesian programming like Turing.jl is virtually unmatched in Python. What about numpyro? Disclaimer: I contribute to numpyro occasionally.
- tucnak 10mo agoHonestly, I'm not familiar with it. I had only played with RxInfer, if only to try the so-called "message-passing" paradigm. My grasp on probability is really lacking, in fact I picked up prediction markets and Julia to get better at it. If you don't mind me asking, what's the deal with NumPyro why you chose to work it?
- paxys 10mo agoLots of words and weird analogies to say basically nothing. What is the status of the project? What can it do? What has it achieved in 5 years? But no, let's highlight how we follow the "Elon process". As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.
- jszymborski 10mo agoFrom [0]: "When we can reproduce a common set of papers on 1 NVIDIA GPU 2x faster than PyTorch. We also want the speed to be good on the M1. ETA, Q2 next year." [0] https://tinygrad.org/#tinybox https://tinygrad.org/#tinybox
- dewey 10mo agoUsing lines of code as a metric for productivity is bad. Using it to show how simple something is, or how a refactor removed x lines of code that doesn’t need to be maintained any more isn’t such a bad thing I’d say.
- whilenot-dev 10mo agoTFA includes a time measurement though, and 5 years for 18'935 SLOC doesn't scream quite "how simple something is".
- alphazard 10mo agoYeah this is exactly right, if you can trust the contributors to not code-golf or otherwise Goodhart the LoC metric, then it's a reasonable measure of complexity. It doesn't work as well when you start mixing languages, or generating code.
- selkin 10mo agoLess LOC also doesn't imply simplicity: just look at the demoscene, which often has the former but not the latter.
- dewey 10mo ago
- timzaman 10mo agoFell bad for geohotz. Such a lovely guy, i hope he strikes it right soon
- still-learning 10mo agoSeems like he's doing fine, why do you feel bad for him?
- unixhero 10mo agoWell what is fine
- slfreference 10mo agoI hope Mitchell Hashimoto with his billions will join forces with Geohotz. I mean ghostty is fine and all. But we are talking here about minority report UI like future.
- still-learning 10mo ago>People get hired by contributing to the repo. It’s a very self directed job, with one meeting a week and a goal of making tinygrad better I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.
- ttul 10mo agoI wonder what happened to George’s old policy of requiring everyone to move to San Diego?
- georgehotz 10mo agoThat's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task. We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose. We also have a wide array of fancy multi GPU boxes that everyone on the team gets full access to (known external contributors can get some access also). I think many companies that were quick to embrace remote have walked it back, not everyone is capable of working productively remotely, nor are all types of work amenable to remote.
- dustbunny 10mo agoWhy Hong Kong? I guess you have a bunch of contributors near there?
- quux0r 10mo agoIf I recall correctly, George recently relocated there.
- octoberfranklin 10mo agoHave you run into problems with contributors who can't enter the PRC?
- measurablefunc 10mo agoIs it really "Complex"? Or did we just make it "Complicated"? - https://www.youtube.com/watch?v=ubaX1Smg6pY https://www.youtube.com/watch?v=ubaX1Smg6pY
- alphazard 10mo agoProgramming a GPU in 2025 is complex, that might be because it has been made complicated, but regardless, it is not complexity that this project can control. The fact that it competes with PyTorch in so few lines speaks to the incredibly low incidental complexity imposed by Tinygrad.
- fragmede 10mo agoIf you think you can do better, the code's right there and you're welcome to try! There's a giant pile of money at the end of the rainbow for you if you can uncomplexify it. (Love that talk tho)
- measurablefunc 10mo agoI don't care for money but you seem to so you should focus on making as much as possible.
- hrimfaxi 10mo ago> pathologizing the speaker/writer does not contribute to anything either so my recommendation is to leave those out as well.
- measurablefunc 10mo ago> There's a giant pile of money at the end of the rainbow for you
- alphazard 10mo ago> To fund the operation, we have a computer sales division that makes about $2M revenue a year. What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?
- piskov 10mo agoGeorge raised $5.1M in 2023 for Tinygrad
- vessenes 10mo agoLooking at their boxes, I'd guess 20-ish%, once the labor costs of designing the boxes is out of the picture. I imagine George doesn't take a salary, or at least takes only a small one. They have a contract with AMD as well. I'd bet the company is modestly profitable.
- bytesandbits 10mo agothat plus contracts with AMD plus they have some VC money
- piskov 10mo ago> tinygrad is following the Elon process for software. Make the requirements less dumb. The best part is no part. That’s not Elon. See Russian TRIZ https://en.wikipedia.org/wiki/TRIZ https://en.wikipedia.org/wiki/TRIZ
- gessha 10mo agoNot sure where it was from but I remember reading a comment about how every modern rocket scientist needs to come to terms with the fact that every novel idea they might have has already been proposed and tried by a Russian scientist in the 20th century.
- wiseowise 10mo agoSoviet scientist, let me correct you a bit, comrade.
- geremiiah 10mo agoThe risk for Tinygrad is that PyTorch will create a new backend for Inductor, plug in their AMD codegen stuff and walala, PyTorch still king. I mean, they could have easily just taken that route themselves instead of bothering with a new ML framework and AD engine. 99% of the work is just the AMD codegen part of the compiler. Either way, super cool project and I wish them the best.
- ellis0n 10mo agoThe main risk is that an LLM will rewrite itself and programmers will no longer be needed. I worked a bit with tinygrad and it looks quite amusing I managed to run it right away and make fixes in one of the tasks, but I decided not to commit because I was afraid of rejection. For example, the tasks are strange: $500 for two months, optimizing H.265, something that only a small group of people in the world can do. The SV is a unique place where you can meet Geo and get $5M, maintain a bunch of hardware, build a framework in 20,000 LOC and everything works well.
- georgehotz 10mo agotinygrad has a PyTorch frontend, we fully encourage the frontend to stay PyTorch and for them to integrate tinygrad at any level. Our mission is to commoditize the petaflop, all frontends are welcome. We also have a quite complete ONNX frontend.
- kaashif 10mo ago> walala Is this like voila or something else?
- peter_d_sherman 10mo ago>"We also have a contract with AMD to get MI350X on MLPerf for Llama 405B training." Anything to help AMD (and potentially other GPU/NPU/IPU etc. chip makers) catch up with NVidia/CUDA is potentially worth money, potentially worth a lot of money, potentially worth up to Billion$... Why? If we have a) Market worth Billion$ and b) A competitive race in that Market... then c) We have VALUE in anything (product, service, ?, ???) that helps any given participant capture more of that market than their competitors... (AMD (and the other lesser known GPU/NPU/IPU etc. chip vendors) are currently lagging behind NVidia's CUDA AI market dominance -- so anything that helps the others advance in this area should, generally speaking, be beneficial for all technology users in general, and be potentially profitable (if the correct deals could be struck!) by those that have the skills to do such assisting...) Anyway, wishing you well in your endeavors, Tinygrad!
- semiquaver 10mo agoIs this the guy who talked a big game about all the things he was going to fix at Twitter, then utterly failed when confronted with a real world codebase and gave up having done nothing of use?
- piskov 10mo agoHe left after realizing nothing was going to change (not like he’s in the leadership). Also half-joked how the good food went away. — George is many things but not a quitter (see comma ai for example). If someone could pull this, it’s him due to “never give up, never surrender” attitude. The shit with nvidia just needs to stop
- rent0pat 10mo agoI've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.
- Barrin92 10mo agoyes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/codegen/simplify.py https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.
- nighthawk454 10mo agoThat doesn’t look super awful to me? Hardly extreme code golfing. The far more interesting part is the order of magnitude. If they can pull off a 20k LOC with zero dependencies (implying a pretty concise project size) and it still works well on meaningful applications, that’s pretty neat. A 1000x reduction in code size and matching/exceeding in perf is worth looking at. Probably also implying a better architecture as code golf isn’t gonna get you 1000x less code. Again - their claims not mine, so we’ll see. But at that point they can triple the LOC to 60k with nothing but white space, new lines, and comments, for all I care. It won’t even add a zero.
- abacadaba 10mo agoit looks dense but perfectly readable. arguably more readable that way than if it had a bunch of extra new lines, definitions, and code blocks spreading the logic out into a larger visible area.
- nighthawk454 10mo agoI'm inclined to agree. While not always appropriate or popular, it makes some sense to me to have the visual weight/area of the code being ~proportional to it's significance. Communicates the ideas more clearly. Instead of spending the majority of space telling me what _isn't_ going to happen, and hiding what is. I often find myself wishing this was more ergonomic in languages.
- random9749832 10mo agoIf you want to "own" Nvidia, the much more realistic way of doing this then trying to compete with all the data centers that are already being built with Nvidia chips is obviously with open source models. In the case of open source models, inference is much more important to most people not training which a maxed out macbook already does a good job of.
- agentifysh 10mo agowhat have people built with tinygrad?
- piskov 10mo agoComma ai
- pehtran 10mo agoHow are ergonomics compared to pytorch, though? Adoption can be also driven by frictionless research (e.g. torch vs. tf comes to mind). Repo is missing proper docs aimed at early adopters imho
- Tepix 10mo agoWhat happened to the tinybox red (v1)? It had way better specs than red v2.
- blactuary 10mo agoDude is an absolute nut, I'll stick with PyTorch https://geohot.github.io/blog/jekyll/update/2025/04/22/a-way-forward.html https://geohot.github.io/blog/jekyll/update/2025/04/22/a-way...
- ergocoder 10mo agoThe blog seems like a regular political opinion. We can't just call people we disagree with "absolute nuts" all the time.
- blactuary 10mo agoSaying Elon is the only adult in the room is bonkers level insane. As is calling the Ds a racist party. This is merely one link, his site is full of equally nuts ranting. https://geohot.github.io/blog/jekyll/update/2023/04/11/wireheading-city.html https://geohot.github.io/blog/jekyll/update/2023/04/11/wireh... https://geohot.github.io/blog/jekyll/update/2025/10/19/solution-is-simple.html https://geohot.github.io/blog/jekyll/update/2025/10/19/solut...
- ergocoder 10mo ago> Saying Elon is the only adult in the room is bonkers level insane That doesn't sound like a "bonker level insane" in any way. Maybe you disagree with him that Elon isn't the only adult in the room but that's just disagreement. People disgreeing with you doesn't make them insane. An example of bonker level insane would be something like: the SF DA described the murder of an elder asian as "temper tantrum" because the DA wanted to downplay the crime. Now that is truly insane: https://www.foxnews.com/us/san-francisco-da-under-fire-after-citing-temper-tantrum-for-murder-of-84-year-old-asian-man https://www.foxnews.com/us/san-francisco-da-under-fire-after... ... Bruh, it was a brutal murder. Use other words please. > As is calling the Ds a racist party. I clicked on the 2 links and search for the word "racist" using the Chrome find functionality. I couldn't find that word. Not saying you are lying, but if you don't reply. You probably are. Can you reference a more specific link? because I have follow-up questions like: is he calling every single person in the Ds racist? Is he calling only a certain group? Is he calling a certain action?
- paprikanotfound 10mo agoI really hope tinygrad succeeds with their mission of commoditizing the petaflop. We're nearing a future where you own nothing and rent everything, and they are one of few companies pushing back. This combined with the focus on making models efficient through better architecture/training, not just throwing more compute at it, seems like the right direction imo.
- arjie 10mo agoI remember when he launched everything and was hoping that the AMD hardware (very capable) was just hamstrung by software. The idea that the easy part is actually the chips is something I never considered though I "know" that "Nvidia succeeded because of their software". Haha, very clever. I've got a Comma 3X and I'm thankful for the 4090 P2P work too (which is now here? https://github.com/tinygrad/open-gpu-kernel-modules https://github.com/tinygrad/open-gpu-kernel-modules) so I'm excited to see it work. Rooting for the guy. Hope it's true that the chip part is the easy work. Could be that both are "the hard work".
- stephantul 10mo agoLove this guy and how committed he is
- shinemydr 9mo agoThe only thing that George got wrong was failing to realize that he is not an engineer, but an inventor. I am saying this as a fan of his work. He is something that I would dare to call a techno artist. He is the Kasparov of sw engineering, not Karpov. His ideas are like lightning. A powerful strike that shakes the earth (or Earth? maybe). But his agenda and his goals interfered with his God given power. Ironically, you can hear him saying on a podcast with Friedman (dont' remember which episode) that one should not have a lofty goal, but then you have him talking all about such goals. The Saviour persona takes over (not saying he doesn't have that in him), but this does not suit him. It should be instinct, rather than intellect that guides him (which is the case with every artist). Think of the reason he got into hacking. From what I understand it was one summer, now long time ago, him being bored. That's it. Boredom mixed with curiosity and innate talent. What got Kubrick into making films? Him seeing how bad the films he saw were. Him thinking he can do better. And he did. A perfect example of motivation that was purely technical in nature. Problem solving skills that needed an outlet. That doesn't mean it is the only reason to get into films, far from it. In words of Kubrick himself: “Some directors are primarily interested in working with actors, others are more interested in visual composition or camera movement. I don’t think there is one correct emphasis.” We all know to which group Kubrick gravitated to (the latter of course, just like his idol Max Ophüls). My point? There was something specific that got him stared and not just: "I wanna make films". Take a look at what got him started filming 2001. It was not a book by Clarke, but this: https://en.wikipedia.org/wiki/Universe_(1960_film) https://en.wikipedia.org/wiki/Universe_(1960_film) More precisely it were the special effects he saw from a Canadian documentary in the link above. Barry Lyndon? NASA lenses. Could you say the same for Scorsese? I don't think so. He would never. Ever. Do it for those reasons. Moving on. When N.Tesla invented induction motor did he do so for the improvement of the infrastructure and improving the lives of others? No. He did it because he wanted to improve DC motor. Or namely, the fact that he did not like the sparks that came from the brushes and that were caused by electrical arcing on commutator segments. Did he care to perfect his invention? No. Mikhail Dolivo-Dobrovolsky did. N.Tesla could not care less about the materials used or the number of phases that would make the motor perform at it's best (even though he was invested in the production phase initially). Why? Because his mission was complete. He gave the idea. He built a working model. Then he moved on. Just like Kubrick, his energy came from a desire to solve a problem. No shame in moving on. Imagine Kubrick calling it quits after Lolita. Or N.Tesla devoting his energies on perfecting the induction engine. Or Elon Musk working on electric cars and not caring about space travel. Etc... Not good. Because there are people in this world made specifically for that role. That would be the Torvald's of the world (L Torvalds: "I am not a visionary. I'm an engineer. I'm happy with the people who are wandering around looking at the stars but I am looking at the ground and I want to fix the pothole before I fall in"). The Karpov's. The Marcan's. The George R.R Martin's... Not Nikola's, not Elon's or Steven King's. Not George's. And just because visionaries look at the stars does not mean they have a lofty goal. Or that they are entitled. Visionary is the wiring of the brain. Those blessed with it should not go against it. I could be wrong. I am not that technical. I am that educated. I have not achieved much in life. English is not my first language.