oofbey

11 ms·

211.
by oofbey 9mo ago
You can control some simple things that way. But the subtle stylistic choices that many teams agree on are difficult to articulate clearly. Plus they don’t always do everything you tell them to in the prompts or rule files. Even when it’s e
212.
by oofbey 9mo ago
Solid MAHA advice right there. Just eat French fries fried in beef tallow, avoid all vaccines, and whatever ails you will surely go away. Medicine isn’t actually that complicated. /s
213.
by oofbey 9mo ago
Wouldn’t matter. The execs of these companies are unlikely to be subject to excessive policing. Systemic bias being what it is.
214.
by oofbey 9mo ago
It’s not incompetence. This is simply not caring. If they had any interest in fixing this they would have. It just wasn’t at all important to them.
215.
by oofbey 9mo ago
It’s nonsensical to call it “zero shot” when a sample of the voice is provided. The term “zero shot cloning” implies you have some representation of the voice from another domain - e.g. a text description of the voice. What they’re doing is
216.
by oofbey 9mo ago
We cannot consider this report unbiased considering the authors are selling the product.
217.
by oofbey 9mo ago
They have a strange business model. Their chips are massive. So they necessarily only sell them to large customers. Also because of the way they’re built (entire wafer is a single chip) no two chips will be the same. Normally imperfections
218.
by oofbey 9mo ago
It is real in that it exists. It is not real in the sense that almost nobody has access to them. Unless you work at one of the handful of organizations with their hardware, it’s not a practical reality.
219.
by oofbey 9mo ago
I think that was pretty clear even when this paper came out - even if you could find these sub networks they wouldn’t be faster on real hardware. Never thought much of this paper, but it sure did get a lot of people excited.
220.
by oofbey 9mo ago
What language would you recommend? Or if none qualify what do you think is missing?
221.
by oofbey 9mo ago
That’s closer still. But even closer would be: x = tokenize(input) i = 0 finish = 0 do { p, x = layers(x) finish += p } while(finish < 0.95 && i++ < t_max); output = lm_head(x) Except t
222.
by oofbey 9mo ago
I think it follows naturally from speech. People say "order ten thousand" pretty naturally. Big-O notation is often vocalized as "order". But technically it's a a clash of ideas when written that way.
223.
by oofbey 9mo ago
Arstechnica used to have different kinds of upvotes for "funny" vs "insightful" - I forget exactly all of them. But I found it awesome. I wanted to and could read the insightful comments, not the funny ones. A couple
224.
by oofbey 9mo ago
Interesting. Thanks for the reference! It's hard to know which way this will go. Discrete/text reasoning has many advantages. Safety as you note. Interpretability, which is closely related. Interoperability - e.g. the fact tha
225.
by oofbey 9mo ago
I’m sad SO died, even if they deserved it.
226.
by oofbey 9mo ago
I think many would agree that this policy was the single biggest moderation failure of the site. And it would Have been so easy to fix. But management believed fewer high quality answers were better. Management was wrong.
227.
by oofbey 9mo ago
In some environments this is a hard requirement, and will be hard to break. Places where the code is know to have big impact / blast radius and can’t be wrong. In other environments (most startups founded in the last six months) no hum
228.
by oofbey 9mo ago
What is a bad idea? Allowing reasoning to happen in continuous space instead of discrete token space? This paper can be seen as a variant of the Coconut models (continuous chain of thought). Continuous reasoning is certainly more efficient
229.
by oofbey 9mo ago
But as we get LLMs to write more of our code, shouldn’t we be moving to languages like Rust, where the compiler is very strict and has lots of checks against subtle bugs? In this sense Python is the worst possible language for vibe coding,
230.
by oofbey 9mo ago
Training doesn’t encourage the intermediate steps to be interpretable. But they are still in the same token vocabulary space, so you could decode them. But they’ll probably be wrong.
231.
by oofbey 9mo ago
Yeah if layers() is a shortcut for layer4(layer3(layer2(layer1(input)))). But sometimes it’s only output = layers(input) Or output = layers(layers(input)) Depends on how difficult the token is.
232.
by oofbey 9mo ago
They literally sold: https://arstechnica.com/gadgets/2021/06/stack-overflow-sold-...
233.
by oofbey 9mo ago
I agree there was some natural slow down as the corpus grew - the obvious questions were answered. But if the community was healthy, that should not have caused growth to stop. New technologies get created all the time, each starting with z
234.
by oofbey 9mo ago
How did Joel sell out? Curious as I’m not aware of any monetary changes. I watched Joel several times support completely brain dead policies in the meta discussions which really set the rules and tone. So my respect there is low.
235.
by oofbey 10mo ago
If discussions had a more modern UI with threads or something then the difference might be real. But AFAICT it’s the same set of functionality, so it’s effectively equivalent to a tag.
236.
by oofbey 10mo ago
Good point there isn’t some magic unlock. Although maybe they were hoping to find one. But a key difference with corn is that corn has been farmed for thousands of years. We know without a doubt there aren’t any low hanging fruit to make it
237.
by oofbey 10mo ago
It’s very natural. Python is fantastic for going from 0 to 1 because it’s easy and forgiving. So lots of projects start with it. Especially anything ML focused. And it’s much harder to change tools once a project is underway.
238.
by oofbey 10mo ago
I think both points are fair. Python is slow - you should avoid it if speed is critical, but sometimes you can’t easily avoid it. I think the list itself is super long winded and not very informative. A lot of operations take about the same
239.
by oofbey 10mo ago
What are the problems with pea protein and lead?
240.
by oofbey 10mo ago
Why do you think it’s not a VC idea? VC is necessary to scale up to large volume. It’s easy for me to believe that insect protein can be a good business at high volume but not low. At volume you can get economies of scale and efficiency and

More ›