dirk94018

6 ms·

31.
Unix Reimagined for AI – Iterative Coding in a Shell Loop(linuxtoaster.com)
1 points by dirk94018 7mo ago|1 comments
32.
by dirk94018 7mo ago
We wrote the linuxtoaster inference engine, toasted, and are getting 400 prefill, 100 gen on a M4 Max w 128GB RAM on Qwen3-next-coder 6bit, 8bit runs too. KV caching means it feels snappy in chat mode. Local can work. For pro work, programm
33.
by dirk94018 7mo ago
Author here. AI as pipe, not platform. toast is sed with a brain — reads stdin, writes stdout, honors the Unix contract. We're rewriting the command line around that idea, one tool at a time. Generation is cheap now. Review is not. The
34.
A Unix Manifesto for the Age of AI(linuxtoaster.com)
6 points by dirk94018 7mo ago|1 comments
35.
by dirk94018 7mo ago
Simplicity is hard. Mark Twain's 'I would have written less had I had more time' at the end of a letter comes to mind. Software dev's tendency to build castles is great for technical managers who want to own complex syst
36.
by dirk94018 7mo ago
Simplicity is hard. Mark Twain's 'I would have written less had I had more time' at the end of a letter comes to mind. Software dev's tendency to build castles is great for technical managers who want to own complex syst
37.
by dirk94018 7mo ago
For chat type interactions prefill is cached, prompt is processed at 400tk/s and generation is 100-107tk/s, it's quite snappy. Sure, for 130,000 tokens, processing documents it drops to, I think 60tk/s, but don't qu
38.
by dirk94018 7mo ago
On M4 Max 128GB we're seeing ~100 tok/s generation on a 30B parameter model in our from scratch inference engine. Very curious what the "4x faster LLM prompt processing" translates to in practice. Smallish, local 30B-70B
39.
by dirk94018 7mo ago
Author here. This started because our C inference engine was slower than Python, which was annoying. We got it to 400 tok/s prefill, 100 tok/s generate, 1,800 lines of C++, no dependencies beyond MLX. Just not redoing work was a 1
40.
Building an Inference Engine in 1,800 Lines of C++(linuxtoaster.com)
1 points by dirk94018 7mo ago|1 comments
41.
by dirk94018 7mo ago
I'm saying the capability to reason about novel situations is in tension with guaranteeing it never produces harmful outputs. We are talking about contradictory design constraints.
42.
by dirk94018 8mo ago
Isn't a safe and reliable intelligence an oxymoron?
43.
by dirk94018 8mo ago
Don't nerf the models. We don't know what we are losing. DOW said it out loud.
44.
by dirk94018 8mo ago
toast is sed with a brain. I got tired of cut and paste and made my own tool. Then I decided to let the AI drive and tried toast | bash, pretty good but AIs are terrible at escaping, got annoyed and wrote a shell for AI to use called jam. W
45.
by dirk94018 8mo ago
Crawler. Heh.. never thought of it that way.
46.
by dirk94018 8mo ago
This is exactly right. We hit the same wall. Our solution was to re-imagine Unix at https://linuxtoaster.com , and either pipe through jq etc or just start rewriting tools that do that. A good tool shouldn't be verbose out o
47.
by dirk94018 8mo ago
The Pentagon seems to see this as a procurement issue, we bought a tool, don't tell us how to use it, and Anthropic seems concerned that the tool's nature is shaped by the constraints put on it, and we don't really understand
48.
by dirk94018 8mo ago
Cool
49.
by dirk94018 8mo ago
This is exactly why local inference matters. Every query you send to a cloud API is another data point. Your prompts contain your code, your logs, your thought process — arguably more identifying than your HN comments. The paper shows deano
50.
by dirk94018 8mo ago
TexMacs is great. However, I use LaTeX regularly. Used to keep a cheat sheet of commands I'd forget between documents. Today I can describe what I want in plain English, pipe it through toast, and get the LaTeX back. LaTeX, vim, sed, a
51.
by dirk94018 8mo ago
Hydrogen has been the future as long as I have been paying attention to electric cars. There are many problems with it, including Hydrogen is the smallest molecule. It leaks through seals, embrittles metals, and has terrible energy density
52.
by dirk94018 8mo ago
Early BSD VM pre-allocated swap backing for every anonymous page — you couldn't allocate virtual memory without a swap slot reserved for it, even if the page was never paged out. When a process forks, the child needed swap reservations
53.
by dirk94018 8mo ago
Hmm. Interesting thought but maybe not everyone without a car needs a bus? My favorite way to get around SF is an EUC. On the side walk, I'm a pedestrian, moving at 2 miles per hour. On the road I'm a car, moving at 30-40 miles pe
54.
by dirk94018 8mo ago
I remember that. A few weeks later ran a script to count all the websites on the Internet.. 324 at that time.
55.
by dirk94018 8mo ago
Aren't LLMs supposed to write machine code directly, no more programming languages at all, any day now? Joking aside, programming languages are a good mental exercise. Forth was my first language after assembly. Didn't like the st
56.
by dirk94018 13y ago
Those European university tests copied by Google, then copied by SV startups, mostly deliver two things, making the interviewer feel superior and filtering for submissive coders. All managers dream of robots that can read minds and work for