robbedpeter

8 ms·

91.
by robbedpeter 4y ago
Part of the scaling benefit comes from the efficiency of training. LSTM and other recurrent architectures take more bits per parameter and the attention mechanism doesn't optimize for context - each lstm neuron uses gating individually
92.
by robbedpeter 4y ago
A McKenna style heroic dose in silent darkness is something I'd recommend any for human, at least once in their life, if they're not at risk of schizophrenia. The experience is profoundly and deeply human. In a therapeutic context
93.
by robbedpeter 4y ago
Color blindness is a common injury among incautious radio engineers who stood in front of transmitters too many times. Dish Network's satellite uplink station in Wyoming can "burn" away cloud cover when needed, and unfortunat
94.
by robbedpeter 4y ago
There are a multitude of such YouTube videos on Linux support for anything you can think of, as well as innumerable support vectors like forums, discord channels, irc, and so forth. Overload with all the available distributions and desktop
95.
by robbedpeter 4y ago
The distribution of ages wasn't hugely different except at the ends of the bell curve. Now, many more people make it past infancy and live past 75.
96.
by robbedpeter 4y ago
I don't think a lot of these critics realize how capable gpt-3 is of convincingly outsmarting most people in most contexts. The failures are glaring, and you'll no doubt catch them over time, but transformers are a sea change in a
97.
by robbedpeter 4y ago
For me, the experience gave me the realization that everything is a hallucination, that all of your every day experience and thoughts are constructs, and that knowledge becomes as fundamental and intimate as the sensation of breathing or to
98.
by robbedpeter 4y ago
Timing is important for some things, and seconds matter sometimes. The option would be nice.
99.
by robbedpeter 4y ago
Oh, you and your silly little math. /s Really, though, things like this should be taught and demonstrated from k-12. A kindergartener can understand the cognitive dissonance when the Monty Hall problem is demonstrated, especially if an
100.
by robbedpeter 4y ago
Diapers, maybe, or the litter / pan would have to be under a constant slight suction? A big enough hamster wheel to give the cat gravity? Someone's gonna have to answer this question in the next decade when rich space tourists wan
101.
by robbedpeter 4y ago
In limited, often zero or one shot probing of the model, yes. Do multiple generations and recursive passes over the output to have the model select and iterate on a target and the utility goes way up. You can coax great output from small mo
102.
by robbedpeter 4y ago
The math the models are doing are similar to rote rule chaining as opposed to calculation. The errors made look like kludged together lookups. I wonder if you could sequence the training of a model so that you could reinforce calculations o
103.
by robbedpeter 4y ago
Spiking comes with persistence baked in, so anything done with them has an implicit sequence and temporal context. Like LSTM, it automatically means the architecture is going to handle some problems better than a naive perceptron. Transform
104.
by robbedpeter 4y ago
Smaller models with better performance are beginning to arrive. Things like RETRO, better training data, longer training time, and scale optimization will have these models on phones and desktops doing crazy things in the near future.
105.
by robbedpeter 4y ago
I think of the logic directing input through various different models and recursion in multiple pass inference as similar to executive function in a brain. Models are getting better and smaller, to the point you can use things like gpt-neo
106.
by robbedpeter 4y ago
Men would often be the authority on finances in a household, in Western culture. The notion is looked down on these days, but until the 70s, men would be the "breadwinner" of the family, and the wife would have limited influence o
107.
by robbedpeter 4y ago
If anything, if you had a dog 100+ years ago, it's because it provided value to you, in the arena of herding, hunting, work, and protection. Survival value, not anything close to modern notions of pets. That means your dog was a valuab
108.
by robbedpeter 4y ago
There is no such consensus. Transformers navigate problem spaces with various mechanisms that include recursion, and multi-pass inference means the depth can be arbitrary. This means that models pick up on the functions that generate answe
109.
by robbedpeter 4y ago
Elon's robots might actually work out, at least in software. This type of methodology, doing meta-cognitive programming by linking together different models, is awesome. They're constructing low resolution imitations of brains - g
110.
by robbedpeter 4y ago
Putin is a paranoid tyrant who should be put down like a rabid wolf. Nobody should care what Putin thinks or wants. After he's gone let's see what shakes out.
111.
by robbedpeter 4y ago
China is run by gangsters, essentially. They make Mexican cartels look like Mother Theresa on some levels.
112.
by robbedpeter 4y ago
Yes. We carry supercomputers in our pockets, it's nice being able to use them and not be subject to connection issues or server outages, or developer whim.
113.
by robbedpeter 4y ago
"Cultural upcycling" has a nice ring to it.
114.
by robbedpeter 4y ago
Iterating over past failures is how progress happens. There will be negative consequences from anything, but the reason for geothermal is that it doesn't have the negative consequences fossil fuels do, and it's cleaner than wind o
115.
by robbedpeter 4y ago
Vitrification of the rock is what it's labeled in the pictures and some of the notes I've seen.
116.
by robbedpeter 4y ago
You can get ad-free content. Things like substack, YouTube premium, Netflix, some newspapers, lots of patreon clients, etc. That ads still exist in other places where you've ostensibly paid for the content doesn't invalidate the o
117.
by robbedpeter 4y ago
MBA's using macs telling their ux and ui "engineers" to make things better by copying Apple.
118.
by robbedpeter 4y ago
It favors megacorporations, who can afford to eat any short term losses as new unfavorable conditions drive smaller businesses under, eliminating competition, and then prices adjust upwards when the local market has been sufficiently captur
119.
by robbedpeter 4y ago
Not necessarily. The biblical flood and a majority of flood myths around the world, go back much farther than that. Many seem to have been inspired by events during the Younger Dryas period, around 12,000 years ago. There's a "bla
120.
by robbedpeter 4y ago
There are also lots of cases where blatantly illegal activity is gotten away with right out in the open because law enforcement don't even know to look, or how to look. Ponzi schemes, pump and dump shitcoins, fake bids on nfts, and so

More ›