3 ms·

Ignore the naysayers! Any article, even the really good ones on HN, while they get positive comments, for whatever reason, always get a lot of negative ones, t

• M4R5H4LL 2mo ago
  I am also very cautious with people who tell me something impossible when I can trust my engineering skills and get a good sense that there is potentially a good outcome. In my experience, it simply means they don’t know how to do it, or are frustrated they couldn’t do it themselves and get into the spotlight.
• RetroTechie 2mo ago
  Very nice indeed. Model weights in RAM blocks distributed all over a big FPGA: should be super helpful at minimizing RAM bandwidth bottlenecks. To say nothing of latency. But model(s) implemented are clearly too small to be useful as a 'chat partner'. Tried a couple of sentences - replies is just some gibberish coming out. This really needs a bigger FPGA, or some other application(s) where a tiny LLM does actually useful work. Barring that, generated tokens/sec is kind of a meaningless measure imho.
  • imtringued 2mo ago
    Obviously at 20k tokens per second your primary goal would be some sort of time series model running at 20KHz for processing sensor data and I'd say the model might even be too big for that. You could probably run a bunch of sensors at once.