jcgrillo

5 ms·

31.
by jcgrillo 4mo ago
Yeah despite the conceptual statelessness, there is quite a bit of state that hangs around though--KV cache and context. I still haven't been able to find anything concrete in docs about how these are isolated. In any case it's cl
32.
by jcgrillo 4mo ago
undefined
33.
by jcgrillo 4mo ago
Isn't everything these days? It's all gotten so gross.
34.
by jcgrillo 4mo ago
I realize the context is where all the retained information is, I guess given how insecure the attempts at preventing injections appear to be I (maybe unfairly) assumed the efforts to keep contexts isolated are similarly lacking. I haven&#x
35.
by jcgrillo 4mo ago
Even assuming the agent is properly sandboxed, and all the services it interacts with treat its commands with appropriate suspicion, don't we still run the risk the agent itself will leak information across sessions? The only way I can
36.
by jcgrillo 4mo ago
Yes, sorry I should have been more specific. A classification task seems totally safe, and like it plays well to LLMs strengths. You also have all kinds of options if it goes wrong, and bounded consequences. What I'm talking about is s
37.
by jcgrillo 4mo ago
Information density is one part, precision is another. Papers are often presenting work at the frontier of the field, which is by nature not well understood yet, and competitive. To have something worthy of publication is to have something
38.
by jcgrillo 4mo ago
The security products industry has a long and storied history of using national security folks in marketing or marketing-adjacent ways (putting them on boards to lend an air of legitimacy, etc). I don't doubt that such people know a lo
39.
by jcgrillo 4mo ago
If that's the case then user-facing products that can take any useful action are strictly off the table.
40.
by jcgrillo 4mo ago
100%. Anyone who is feeding unsanitized input to an LLM is doing it wrong. It'd be just like letting users craft their own SQL queries. I think the security aspect raises an interesting (if awkward) question: How do you sanitize inputs
41.
by jcgrillo 4mo ago
It's more nuanced than either extreme. But regardless of the root cause, if you have engineers duplicating work left and right something has gone wrong. Their labor is not being used efficiently. EDIT: LLM or not, this is still true. I
42.
by jcgrillo 4mo ago
> engineers engineer around them with their own solutions When that happens there's a major engineering leadership failure currently in progress, even if engineering leadership isn't aware of it.
43.
by jcgrillo 4mo ago
I don't have a good intuition about the what makes a problem sensitive to tuning, is it possible there are a large number of control problems where proper tuning doesn't matter much? Or the deleterious effects of improper tuning (
44.
by jcgrillo 4mo ago
I saw a crazy stat somewhere like some large fraction of the industrial PID controllers that are actually deployed in plants are completely untuned--that is, just running default settings. I think it was a book about tuning PID controllers,
45.
by jcgrillo 4mo ago
PSA: do not forget to properly format the cover of your TPS reports
46.
by jcgrillo 4mo ago
It's endlessly fascinating to me how long it's taking for GPUs to be adopted for this kind of workload. I remember all the excitement around the GPGPU era, OpenCL, and eventually CUDA. That was like 15yr ago! Yes, GPUs can do a
47.
by jcgrillo 4mo ago
I feel like based on the amount of power these people wield in the world I should be afraid of them, but then I read this: > registrants returned again and again to the same theme: that AI will reorder work, war, education, and belief wi
48.
by jcgrillo 4mo ago
I just wonder how bad it will have to get before they decide we need to be reigned in. Going by how the 20th Century went, if it's anything like that we're in for a rough time before we get there--like, to a place where software e
49.
by jcgrillo 4mo ago
The computer does not see things like you see things ;)
50.
by jcgrillo 4mo ago
It's shocking. I can't understand why people repeatedly fall for it.
51.
by jcgrillo 4mo ago
Made my day lol. Brilliant.
52.
by jcgrillo 4mo ago
FaaS are really interesting, because the tradeoffs seem "jagged" (to use a word I learned recently). Like, IIUC this is more or less what's happening when a cloud function is invoked: 1. Some event triggers the function (HTTP
53.
by jcgrillo 4mo ago
I remember reading somewhere about an HPC "rule of thumb" that was like 1GB RAM/1GHz of processing is about the right amount for most applications.. Don't recall where that came from or what the exact reasoning was. That
54.
by jcgrillo 4mo ago
I swapped the engine on my lawn mower ~3yr ago because it no longer had any compression and was very hard to start. The new engine has an electric starter. I didn't have time to fabricate a proper battery tray so I ratchet strapped a 2
55.
by jcgrillo 4mo ago
Machine telemetry data (logs, metrics, traces) are shipped around as highly redundant utf-8 strings. I'm sure there are plenty of other examples of 2+ orders of magnitude improvements that aren't even low hanging fruit, they'
56.
by jcgrillo 4mo ago
> The main place RAM usage is going to get optimized is on the server side Agree, but it's going to be a long, hard, potentially ultimately unsuccessful uphill slog. I think the main obstacle is basically 2 decades of inertia. Appro
57.
by jcgrillo 4mo ago
> The analysis that I want is on a studied cost/value basis somehow. Absolutely. The AI ROI question is the entire ballgame. Earlier today I heard the term "jagged technology" for the first time, and it's an extremely
58.
by jcgrillo 4mo ago
I stand corrected! That's very interesting.
59.
by jcgrillo 4mo ago
Out of touch leadership and management combined with a culture of self-congratulatory "innovation" where companies just plagiarize eachother in an endless loop... It doesn't surprise me at all, tbh.
60.
by jcgrillo 4mo ago
> Only when you implement by hand can you feel if the implementation of your design is painful or not, and only this signal can tell you if your design if truly good or not. I think this touches obliquely on a point I keep coming back to

More ›