scarmig

21 ms·

391.
by scarmig 2y ago
A while back, I did an experiment: drawing a typing keyboard. It's something I see and use every day, so it should be easy, right? Well, I was able to, but it didn't come from first principles or even visualizing it. It came from
392.
by scarmig 2y ago
I agree this is true if your goal is to inform. If, however, your goal is to increase clicks, decrease support tickets, and get engagement, I'd be surprised if being technically descriptive and accurate is better than dumbing it down t
393.
by scarmig 2y ago
I wouldn't be surprised if the sole training data for autocompletes was google3. It's an absolutely massive codebase, using the libraries and patterns Googlers use, and more or less entirely safe to train on. Any training data bey
394.
by scarmig 2y ago
Was OpenAI at all affiliated with YC? I think they were entirely independent organizations.
395.
by scarmig 2y ago
Kind of, but that's not inherent to the over reliance on memorization. I suspect using the top tier models from any of OAI, Anthropic, or Google would have resulted in much less embarrassing results, and I believe they're all prim
396.
by scarmig 2y ago
For the purpose of generalization, I think that's more treating a symptom than the root cause. The better approach IMO would be finding architectures that heavily penalize the formation of memorizing and interpolating circuits. E.g muc
397.
by scarmig 2y ago
Not about that quote in particular, but the original grokking paper from Powers et al came about because they accidentally left a training job to run too long (at least as related by gwern).
398.
by scarmig 2y ago
What they share is a subversion of the naive framework that ML works simply by performing gradient descent over a loss landscape. Double descent subverts it by showing that learning isn't monotonic in parameter count; grokking subverts
399.
by scarmig 2y ago
Since I first learned about grokking, I've had a strong suspicion that getting a handle on it and figuring out how to aid it should be the central question in AI. We are currently stuck in a local minimum, where memorizing circuits per
400.
by scarmig 2y ago
I think it's a joke; imagine you spent 24 hours a day looking at the sky.
401.
by scarmig 2y ago
Network pruning. In ANNs, pruning 90% of the weights without substantial loss isn't unheard of. I guess this may be analogous: continuous pruning and fine tuning over a lifetime. Though, is removing 90% of the brain more analogous to 9
402.
by scarmig 2y ago
The larger network contains exponentially more subnetworks. 10x the size contains far more than 10x subnetworks (although it'd also take more than 10x as long to train).
403.
by scarmig 2y ago
The halting problem doesn't mean you can never decide if something cycles etc, just that you can't always decide. As it stands, my guess is that the LLM would always confidently make a decision, even if it were wrong, and then pol
404.
by scarmig 2y ago
Most journalists have never visited Israel (though I believe Gessen has). As far as who knows more, the base level facts are well-known: two ethnic groups both want a piece of land with lots of religious and cultural attachments; the weaker
405.
by scarmig 2y ago
If your ostensible purpose is being sidelined by decision makers, trying to fight back is often a good option, but sometimes you fail. Admitting failure and focusing on other approaches is the right choice at that point.
406.
by scarmig 2y ago
Starting something without a very good plan or being unable to execute on it is a sign of bad judgment.
407.
by scarmig 2y ago
Google does not care to start a war where every company has to form explicit legal agreements with every other company to scrape their data. Maybe if they got really desperate, but right now they have no reason to be.
408.
by scarmig 2y ago
Yeah. There are lots of things we can do with existing capabilities, but in terms of progressing beyond them all of the frontier models seem like they're a hair's breadth from each other. That is not what one would predict if LLMs
409.
by scarmig 2y ago
It's not a lie to provide the best argument for something; it'd only be a lie if you looked at the best argument for something and declared it true by fiat. Imagine I've realized someone I'm talking to is a flat Earther,
410.
by scarmig 2y ago
Unsurprisingly, "poor people" are not targeted for help.
411.
by scarmig 2y ago
Yes: for most obese people, to become not obese and stay so for the rest of their life, lifelong Ozempic is the only thing that works at scale. Behavioural interventions more or less don't work (on an individual level they occasionally
412.
by scarmig 2y ago
> Google doesn't pay it's shareholders dividends Google just started paying a dividend.
413.
by scarmig 2y ago
There's always tradeoffs. Ozempic probably has some long term side effects, but are they worse than obesity? Probably not. (And an intervention based on ozempic is far more likely to cause lasting change than one based on a prescribed
414.
by scarmig 2y ago
Those mutations would probably be favored by evolution, if anything. They would attract less attention and allow more focus on reproduction.
415.
by scarmig 2y ago
Parasites do the same thing: hijack a piece of hardware and use it to reproduce. Computer viruses have even formed a kind of symbiotic ecosystem with blackhats: the scammer provides resources to help the virus reproduce, and the virus provi
416.
by scarmig 3y ago
Sometimes valid email addresses will be rejected as invalid, and sometimes invalid email addresses are still successfully delivered. Validation guarantees nothing, and at most it should be a UI cue.
417.
by scarmig 3y ago
Anecdote: as a bisexual guy (now married to a woman), dating women was much, much harder than dating men. This extends beyond sexual encounters to actual relationships: for every relationship opportunity with a woman, I'd guesstimate I
418.
by scarmig 3y ago
Taxes, too: the government will be taking over a third of your winnings for a big payout.
419.
by scarmig 3y ago
Sure. Is more money in the US spent on gambling, or on prestige goods?
420.
by scarmig 3y ago
With a designer handbag, you get something that's worth $50 for a cost of $500. That's a 100% chance of a $450 loss. But, you say, it gives me pleasure! Well, so does gambling. And spending money for pleasure is fine, so long as y

More ›