rahen

5 ms·

31.
by rahen 6mo ago
A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
32.
by rahen 6mo ago
I'm also writing a compiler and CS6120 from Cornell has helped me a lot: https://www.cs.cornell.edu/courses/cs6120/2025fa/self-guided...
33.
by rahen 6mo ago
I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me. There is nothing wrong using AI tools to write code, but nothing here seems to have taken more than a generic 'wri
34.
by rahen 7mo ago
Out of curiosity, what were those wonderful things you were hearing about the 11/34 back then?
35.
by rahen 7mo ago
Yes. The Cray supercomputers from the 80s were crazy good matmul machines in particular. The quad-CPU Cray X-MP (1984) could sustain 800 MFLOPS to 1 GFLOPS, and with a 1 GB SSD, had enough computer power and bandwidth to train a 7-10M-param
36.
by rahen 7mo ago
I also have a working design for a small Transformer on the original Game Boy. It has around 4000 parameters fitting in the 8 KB cartridge SRAM, where the "saved game" is the trained model. A TI-82 with its 32 KB of RAM would be e
37.
by rahen 7mo ago
I encouter two main failure modes. First, the bipolar PROMs degrade at the atomic level, the metal ions in the fuses tend to migrate or 'regrow' over decades, causing bit rot. Second, the backplanes suffer from mechanical fatigue.
38.
by rahen 7mo ago
The WASM GUI is probably the easiest way to see the Transformer in action on this machine: https://dbrll.github.io/ll-34/ There's also the original Tetris from 1984 to play.
39.
by rahen 7mo ago
That thing is a Tamagochi though, it constantly needs attention, pardon the pun. I did most of the development and tuning on ll-34 for that reason.
40.
by rahen 7mo ago
Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.
41.
Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer(github.com)
145 points by rahen 7mo ago|26 comments
42.
by rahen 8mo ago
Around the same time (1984), there was also another very cool piece of technology that often gets overlooked: the CMU WARP. It wasn’t as flashy as the Crays and the Connection Machine, but it was the first systolic array accelerator (what w
43.
Sheaf – A Functional Language for Differentiable Programs(sheaf-lang.org)
3 points by rahen 8mo ago|1 comments
44.
by rahen 8mo ago
I've been building a functional language for differentiable programming that compiles to JAX. The core idea is homoiconicity applied to ML, models are data structures that can inspect and transform themselves.
45.
by rahen 9mo ago
For those interested, this guy is revamping the Emacs widget library with something more modern and platform agnostic, based on SDL: https://appetrosyan.github.io/posts/ His posts are very insightful.
46.
by rahen 9mo ago
My only complaint regarding the Zed editor is the inability to display two panes of the sidebar one below the other. Not only is it impossible to display them together, but switching between them requires clicking a tiny button in the statu
47.
by rahen 9mo ago
Lisp isn't missing anything, it's a natural fit for AI/ML. It’s the ecosystem's tooling that needs catching up. The code hasn't reached RC yet, but I'll definitely post a Show HN once it's ready for a prev
48.
by rahen 9mo ago
I doubt it considering there are massive Clojure codebases with large teams collaborating on them every day. The lack of Lisp tooling and the prevalence of Python are more a result of inertia, low barrier to entry and ecosystem lock-in.
49.
by rahen 9mo ago
"> I think all of ML being in Python is a colossal mistake that we'll pay for for years. Market pressure. Early ML frameworks were in Lisp, then eventually Lua with Torch, but demand dictated the choice of Python because "
50.
by rahen 9mo ago
I love it, instant Github star. I wrote an MLP in Fortran IV for a punched card machine from the sixties ( https://github.com/dbrll/Xortran ), so this really speaks to me. The interaction is surprisingly good despite the
51.
by rahen 10mo ago
The reverse is true though, and I find that fascinating with Fortran. I recently learned Fortran IV to build a backpropagated neural network for the IBM 1130 (1965) and was amazed to see it compile with no warning on both the punched card c
52.
by rahen 10mo ago
You mean everywhere. It's just hidden behind abstraction layers or Fortran libraries like BLAS/LAPACK, which are used by NumPy, R, Julia, MATLAB, Excel, TensorFlow, PyTorch (for some backends), and basically anything that involves
53.
by rahen 10mo ago
Unless you need horizontal scalability or clustering, Compose + Terraform is all you need. With Compose, you get proper n-tier application containerization with immutability. By adding an infrastructure-as-code tool such as Terraform to abs
54.
by rahen 11mo ago
Part 9 elaborates on GOOL, the Lisp dialect they designed in-house to create the gameplay. This is my favorite part: https://all-things-andy-gavin.com/2011/03/12/making-crash-ba...
55.
by rahen 11mo ago
The best challenger to systemd in terms of feature parity is probably dinit: https://davmac.org/projects/dinit/ Have a look at Chimera Linux if you want to give it a try: https://chimera-linux.org/
56.
by rahen 11mo ago
The article and discussion are about runit, why bring systemd into it? Diversity in solutions is a good thing, there’s no need to feel threatened by that.
57.
by rahen 11mo ago
Author here. They call it a FORTRAN IV compiler but it uses some F66 extensions, such as proper types and functions, although it lacks some of the nicer constructs of F66 like If/Then/Else, which would have been handy. Regarding f
58.
by rahen 11mo ago
The first convolutional neural network, the Neocognitron, was AFAIK implemented on a PDP-11 as well: https://www.semanticscholar.org/paper/Neocognitron%3A-A-neur... No backpropagation back then, this only appeared arou
59.
Xortran - A PDP-11 Neural Network With Backpropagation in Fortran IV(github.com)
46 points by rahen 11mo ago|11 comments
60.
by rahen 11mo ago
> Basically what he said was that lisp ended up not working well for larger software projects with 5 or more people on them I don’t think "doesn’t work for teams of 5+" is a fair generalization. There are production Clojure and

More ›