joefourier

7 ms·

61.
by joefourier 7mo ago
You don't even need to go into the pipeline details. The 9800X3D has 8x more L2 cache, 6x more L3 cache, 2x the memory bandwidth than the now 8 years old i9 9900K. 3D V-cache is pretty cool.
62.
by joefourier 7mo ago
I'm not from that generation so that's a bit hard for me to understand. Even if you used a closed-source C compiler, wouldn't you still have been able to look at the header file, which would have been pretty self-explanatory?
63.
by joefourier 7mo ago
Oh I'm happy this has a name now! Even if it's quite silly.
64.
by joefourier 7mo ago
> HN is an interesting place. How is it interesting that in a forum with thousands of active users, someone posted a comment that disagreed with other comments from 6 years ago? Even in this thread, there's many different opinions b
65.
by joefourier 7mo ago
Surely it's more of a spectrum? From a CPU, to a TPU, to a chip that hardwires softmax attention but lets you store arbitrary weights, to one that hardwires the weights directly.
66.
by joefourier 7mo ago
That depends what kind of ASIC you’re talking about. Cerebras can run models like GLM 4.7 with 355B parameters.
67.
by joefourier 7mo ago
The first professional commercial 4K camera came out over 23 years ago, and the first smartphones and camcorders capable of 4K video were back in 2013. The Macbook Neo has a 2.5x higher multi-core Geekbench score compared to the i7-4960X&#x
68.
by joefourier 7mo ago
Fine-tuning still makes sense for cost/latency-sensitive applications. Massive context windows drastically slow down generation, and modern models' performance and instruction following ability relies heavily on a reasoning step t
69.
by joefourier 7mo ago
You don't need 8GB of RAM or less to have memory issues. Cursor + Claude Code + Slack + Discord + Spotify + a few Docker containers + YouTube and a few browser tabs is enough to overwhelm a MacBook with 24GB of RAM. Right now on my mac
70.
by joefourier 7mo ago
What's your definition of intelligence? If you exclude LLMs, you might have to exclude quite a few humans as well.
71.
by joefourier 7mo ago
When did AGI start meaning ASI? LLMs are artificial general intelligence, as per the Wikipedia definition: > generalise knowledge, transfer skills between domains, and solve novel problems without task‑specific reprogramming Even GPT-3
72.
by joefourier 7mo ago
Tensorflow is largely dead, it’s been years since I’ve seen a new repo use it. Go with Jax if you want a PyTorch alternative that can have better performance for certain scenarios.
73.
by joefourier 7mo ago
You can actually generate surprisingly coherent text with minimal finetuning of BERT, by reinterpreting it as a diffusion model: https://nathan.rs/posts/roberta-diffusion/ I don’t see a useful definition of LLM th
74.
by joefourier 8mo ago
Well in that case wouldn't that be millions of ASIs, each with contradictory goals? I'm not saying that ASI isn't an existential threat, just that it probably won't present itself like the fanciful sci-fi scenario of a s
75.
by joefourier 8mo ago
> No, it would have been called AI. A decade ago most people were not familiar with AGI as a term, that just got popularised because AI was taken over to be basically what we used to call ML. Define "most people", I don't
76.
by joefourier 8mo ago
We are already at AGI. I don’t know how you can argue that LLMs don’t meet the definition of general artificial intelligence, as opposed to narrow AI like chess engines, image classifiers, AlphaGo or self driving cars, which are trained wit
77.
by joefourier 8mo ago
ASI still runs at finite speed and is limited by its hardware, and speed of its interactions with the real world. It won’t be able to recursively improve itself overnight if it only generates 10 tokens per seconds, and a second company coul
78.
by joefourier 8mo ago
Oh yes you're correct, imaging would be the correct term for what's happening I think (aliasing is high -> low and imaging is low -> high)?
79.
by joefourier 8mo ago
The author is correct in that agents are becoming more and more capable and that you don't need the IDE to the same extent, but I don't see that as good. I find that IDE-based agentic programming actually encourages you to read
80.
by joefourier 8mo ago
The reason the nearest neighbour interpolation can sound better is that the aliasing fills the higher frequencies of the audio with a mirror image of the lower frequencies. While humans are less sensitive to higher frequencies, you still ex
81.
by joefourier 8mo ago
Odd that the author didn’t try giving a latent embedding to the standard neural network (or modulated the activations with a FiLM layer) and had static embeddings as the baseline. There’s no real advantage to using a hypernetwork and they t
82.
by joefourier 8mo ago
It absolutely is noticeable the moment you have to run several of these electron “apps” at once. I have a MacBook with 16GB of RAM and I routinely run out of memory from just having Slack, Discord, Cursor, Figma, Spotify and a couple of Fir
83.
by joefourier 8mo ago
Not go to all “ackchually” but modern GPUs can render in many other ways than rasterising triangles, and they can absolutely draw a cylinder without any tessellation involved. You can use the analytical ray tracing formula, or signed distan
84.
by joefourier 9mo ago
Have you used an LLM specifically trained for tool calling, in Claude Code, Cursor or Aider? They’re capable of looking up documentation, correcting their errors by compiling and running tests, and when coupled with a linter, hallucinations
85.
by joefourier 10mo ago
There’s a few more considerations: - You can use the GPU for training and run your own fine tuned models - You can have much higher generation speeds - You can sell the GPU on the used market in ~2 years time for a significant portion of it
86.
by joefourier 10mo ago
If you think medieval artists lacked skill, check out Villard Honnecourt’s sketchbook, especially the insects on folio 7 and Christ in Majesty on 16: https://www.medievalists.net/2024/12/sketchbook-villard-honn...
87.
by joefourier 10mo ago
M? The OP literally did train an LLM from scratch in a 3090 (except for the tokenizer), that’s what the whole post is about.
88.
by joefourier 10mo ago
Deepseek via their API also has cached context, although the tokens/s was much lower than Claude when I tried it. But for background agents the price difference makes it absolutely worth it.
89.
by joefourier 1y ago
In the case of dithering, that’s only because the monitor has insufficient resolution. Put a 1:1 Floyd steinberg dithered image on your phone, hold it at arm’s length, and unless you have superhuman vision you’ll already start having a hard
90.
by joefourier 1y ago
Beautiful demo, but I’m not sure it’s accurate to call dithering an “illusion” of more shades than is available? If you apply a low pass filter to a dithered image, and compare it to a low passed filtered thresholded, you’ll see that the “i

More ›