ijk

7 ms·

61.
by ijk 7mo ago
For a while the complaint was that Lego was making too many big, specialized pieces, so I'm amused that the current complaint seems to be that they're using too many small generic ones.
62.
by ijk 7mo ago
I have discovered that the measure of good documentation is not whether your team writes documentation, but is instead determined by whether they read it.
63.
by ijk 7mo ago
This is my current problem: I can get work to pay for a Claude Max subscription, but for personal use or to learn how to use it that's a big price tag. I worry that we're returning to an era of renting core development tools. Afte
64.
by ijk 8mo ago
I find this fascinating, as I keep observing that there are pretty widespread differences between what people believe copyright does and what the law actually says.
65.
by ijk 9mo ago
It's interesting that mining gold and silver is similar to printing more money; we usually think of inflation as represented by debasing coinage but there's been a few circumstances where flooding the local economy with precious m
66.
by ijk 10mo ago
I'm no expert, but having read some archeological papers that do make conclusions like that, the evidence is often quite compelling and well-supported. The context we find something in can convey a lot of data, and conclusions that are
67.
by ijk 10mo ago
Is there a changing taste hypothesis? It's honestly the first time I've heard that suggested as the explanation, versus the more plausible to me idea of reconstruction from incomplete evidence.
68.
by ijk 10mo ago
Eh, that's overstating the case. There's clearly some aesthetics that are more appealing to more people but for many architectural movements in particular the reason that they look that way is for the way that specific ideological
69.
by ijk 10mo ago
Interestingly to me, generative AI is often used to get results that commit the opposite error compared with these statues: they are, essentially, too confident in their choice of details. For any random topic, the average member of the pub
70.
by ijk 11mo ago
I was hoping that this would be about Llama 1 and comparison with GPT-contaminated models.
71.
by ijk 11mo ago
Unfortunately, I am also worried that is the case. There was an era where there were a lot of completely free sites, because they were mostly academic or passion projects, both of which are subsidized by other means. Then there were ads. Ba
72.
by ijk 11mo ago
Buying a book scanner and frequenting used book stores seems like a past time to start that'll pay off in the long term.
73.
by ijk 1y ago
There is an awful lot of "looking for my keys under the street light" going around these days. I've seen a bunch of projects proposed that are either based on existing data (but have no useful application of that data) or hav
74.
by ijk 1y ago
Not rainforest, but rather savanna [1]. The Arabian desert is technically considered to be part of the Sahara, climate-wise, and participes in the same cycle [2]. This article is about researching evidence for ehat those transitions looked
75.
by ijk 1y ago
Not strictly true: while this was previously believed to be the case, Anthropic demonstrated that transformers can "think ahead" in some sense, for example when planning rhymes in a poem [1]: > Instead, we found that Claude pl
76.
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning(arxiv.org)
2 points by ijk 1y ago|0 comments
77.
by ijk 1y ago
So, what I think most people don't realize is that the amount of computation an LLM can do in one pass is strictly bounded. You can see that here with the layers. (This applies to a lot of neural networks [1].) Remember, they feed in t
78.
by ijk 1y ago
There's been a few attempts at training a backspace token, though. e.g.: https://arxiv.org/abs/2502.04404 https://arxiv.org/abs/2306.05426
79.
by ijk 1y ago
Adding knowledge works, depending on how to define knowledge and works ; given sufficient data you can teach an LLM new things [1]. However, the frontier models keep improving at a quick enough rate that it's often more effective ju
80.
by ijk 1y ago
That's consistent with other research I've seen, where varied presentation of the data is key to effective knowledge injection [1]. My assumption, based on the research is that training on different prompts but the same answer giv
81.
by ijk 1y ago
Some of that is, or at least was, down to the training: extending the context window but not training on sufficiently long data or using weak evaluation metrics caused issues. More recent models have been getting better, though long context
82.
by ijk 1y ago
Yes, I was using it for structured outputs before the dedicated structured outputs got their act together.
83.
by ijk 1y ago
My sense is they need to go back and update previous docs; they release a lot of software updates and a lot of notebooks showing how to use the features, but the two might fall out of sync. Would that match your observations?
84.
by ijk 1y ago
It's a little more subtle than that: They're approximating the language used by someone describing the taste of chocolate; this may or may not have had any relation to the actual practice of eating chocolate in the mind of the o
85.
by ijk 1y ago
Spans labeled as 'unknown' when I definitely labeled them in the code is probably the most annoying part of Phoenix right now.
86.
by ijk 1y ago
Assembling 6GB of training data is actually rather impressive, given the constraints.
87.
by ijk 1y ago
I find that for more "intuitive" evaluations, reasoning tends to hurt more than it helps. In other words, if it can do a one-shot classification correctly, adding a bunch of second guessing just degrades the performance. This may
88.
by ijk 1y ago
I'm really just looking for a subset of XML so that's probably sufficient. For me, the advantage that Pydantic AI has right now is that it's easy to do ingestion/validation of the generated text, since I've already
89.
by ijk 1y ago
I tend to find that using LLMs for interpretation and classification is often more useful for a given business task than wholesale generation.
90.
by ijk 1y ago
Of course "reads like" is part of the problem. The models are very good at producing something that reads like the kind of document I asked for and not as good at guaranteeing that the document has the meaning I intended.

More ›