daxfohl

35 ms·

571.
by daxfohl 2y ago
But how on earth do you train it? With regular LLMs, you get feedback on each word / token you generate, as you can match against training text. With these, you've got to generate hundreds of tokens in the thinking block fiest, an
572.
by daxfohl 2y ago
That's what I was thinking too, though with an image you could do a convolution layer and, idk, maybe that makes it imagine visually. Or actually, the reasoning is backwards: the convolution layer is what (potentially) makes that part
573.
by daxfohl 2y ago
I wonder what it would look like in multi modal, if the reasoning part was an image or video or 3D scene instead of text.
574.
by daxfohl 2y ago
Half of my team's work right now is changing our abstractions because a couple clients want something slightly different, without breaking existing clients. The other half of my team's work is KLO forced on us by our dependencies
575.
by daxfohl 2y ago
Abstractions are a failure in most libraries too. Invariably something will come up that breaks the abstraction. Either new instances come up that don't quite match, or clients want new functionality that doesn't apply to all inst
576.
by daxfohl 2y ago
I agree with everything except there being a business opportunity there. Whatever ends up being the fix, all the big players will incorporate it into their own IDEs within a couple months. Unless the fix is something that is significantly d
577.
by daxfohl 2y ago
Or DSLs like cucumber for acceptance tests. Cute for simple things, but for anything realistic, it's more convoluted than convenient.
578.
by daxfohl 2y ago
I wonder how much of this is "I don't want to sell this Tesla I couldn't afford, so instead I'll change my whole political ideology"
579.
by daxfohl 2y ago
I think there's no better proof than this that they stole a big chunk of OpenAI's model.
580.
by daxfohl 2y ago
Haven't formed thoughts on the content yet, but happy to see a company launching something non-AI for a change.
581.
by daxfohl 2y ago
How do you do GDPR takedowns?
582.
by daxfohl 2y ago
It's because everybody in big tech has a tesla, and it's a lot easier to change political ideologies than sell your car.
583.
by daxfohl 2y ago
tbh i'll probably just forget both things.
584.
by daxfohl 2y ago
I'm bewildered how things changed so fast. Big Tech was conservative public enemy number 1 just a couple years ago. And vice-versa. So now what? Will Democrats start cozying up to the oil oligarchs? Sounds ridiculous, but ... do they h
585.
by daxfohl 2y ago
Nice try, Zuck.
586.
by daxfohl 2y ago
Maybe buried in that proof is the key insight needed to prove P != NP. Elegance would make it more likely that this insight is pulled out and identified.
587.
by daxfohl 2y ago
Though, say we finally solve chess. There's not always going to be a "why" the suggested move is the best. There might be heuristics, or even some set of rules in the end game or certain situations that a human's brain c
588.
by daxfohl 2y ago
Hash *keys* are 8 bytes (assume 64-bit machine). But the Set *buckets*[1] are a masked subset of that, starting at 3 bits and growing with the size of the set. Meaning if you store 16, 8, 32, 24, 0 all in the same set, they'll be in t
589.
by daxfohl 2y ago
Not only bucketed together, which means they could still be disambiguated in C by the full hash, but they'll actually share the full hash, which means they'd have to jump back into Python code to check `__eq__` to disambiguate IIU
590.
by daxfohl 2y ago
Is it "just" a problem of a few missing definitions? It reminds me of like a class that has a hundred fields and subfields and lambda types that each have their own hundreds of fields and subfields and lambda types, etc. Typically
591.
by daxfohl 2y ago
More explicitly, since buckets in Python's Set implementation are the bottom N bits of the hash (where N depends on set size), this ensures linear sequences of ints (which are the most common type of int set) are all in different bucke
592.
by daxfohl 2y ago
Another question is why do ints largely hash to themselves? Most hashing guides recommend hashes be far more distributed IIUC.
593.
by daxfohl 2y ago
Oh. No, not sure.
594.
by daxfohl 2y ago
True, and this even happens in a brain the size of a flea's. Which makes one wonder, what is it that makes processing and reconciling millions of visual signals per second "easy", but reasoning through a simple sudoku near im
595.
by daxfohl 2y ago
Isn't that what the original resnet project disproved? Rather than trying to hand-manicure what the NN should look for, just make it deep enough and give it enough training data, and it'll figure things out on its own, even better
596.
by daxfohl 2y ago
There's a _lot_ of smoke and mirrors. Paste a sudoku into chatgpt and ask it to solve. Amazing, it does it perfectly! Of course that's because it ran a sudoku-solving program that it pulled off github. Now ask it to solve step by
597.
by daxfohl 2y ago
And you know that you'll come back in five years and nothing will have changed, except five new services that do half of what the vision required, most systems still depend on what was originally there, the team has tripled in size and
598.
by daxfohl 2y ago
Yup, or EM pitches the big five year vision rearchitecture because that's what builds their empire, even though a cron job that takes a couple days to write would solve the business need.
599.
by daxfohl 2y ago
The challenge here is a lot of managers don't know how to grade you based on things you successfully avoided.
600.
by daxfohl 2y ago
Enshittification

More ›