Straw

8 ms·

91.
by Straw 3y ago
No, all of these use the same fundamental architecture with minor tweaks, such as the dynamic gate for mamba or an outer product paramterization of the values for RWKV-v5
92.
by Straw 3y ago
The SSMs papers and blogs always have unnecessarily complicated explanations. At this point I almost wonder if its to hide how simple the underlying algorithms are, or to make them seem fancy. SSMs are doing exponentially weighted moving av
93.
by Straw 3y ago
I'm familiar with the NN-as-GP papers. In practice we don't train infinite width neural networks!
94.
by Straw 3y ago
This keeps popping up but while technically true, its essentially nonsense- normally when people talk about kernel machines, the kernel doesn't depend on the data, or at least not very much- we might use a Gaussian kernel and tune the
95.
by Straw 3y ago
The validation curves will look identical. These models are far too small to overfit to the training set. With a large enough model and many epochs, you can certainly get overfitting, but for one epoch val/train curves look exactly the
96.
by Straw 3y ago
Chinchilla's death has been greatly exaggerated. This article makes the same mistake as in the original GPT-3 scaling law of extrapolating from mid-training loss curves- but most of the loss improvement in the middle of training comes
97.
by Straw 3y ago
Sure, you can sell solar for cheaper when the sun shines. But if we want to consider replacing the entire grid, we need to look at the cost including storage and not accounting for slack in other generation sources!
98.
by Straw 3y ago
According to this recent source, its currently 100-130 $/MWh: https://www.lazard.com/media/2ozoovyg/lazards-lcoeplus-april... And that's for 100MW solar with 50MW of 4-hour storage
99.
by Straw 3y ago
I'm having trouble believing that its that cheap- battery prices have not fallen 8x in the last two years. I believe the report I was reading was from 2021.
100.
by Straw 3y ago
Perhaps California taxpayers make up the difference?
101.
by Straw 3y ago
Unfortunately, it has significant costs as well as benefits: -What makes you think we'll get huge economic growth from building very expensive infrastructure? -An excellent goal, but we can get this for example with gas turbines which
102.
by Straw 3y ago
OpenAI | GPU Software Engineer | CUDA/Triton | Onsite (hybrid) | SF, California | VISA We're looking for engineers with GPU programming experience/knowledge to help write high performance kernels and improve the Triton compil
103.
by Straw 3y ago
Although I agree wholeheartedly with the economic logic, I have a couple of minor counterpoints: 1. Constant 2% inflation at least doesn't have the negative effects of unpredictable inflation, its just a 2% tax. 2. Certain prices appea
104.
by Straw 4y ago
Unfortunately its not very good at longer context lengths, which sort of defeats the point of efficient scaling with context. See https://twitter.com/arankomatsuzaki/status/16390003799784038... Its also not really
105.
by Straw 4y ago
Bicycle Thieves, very sad. The owner of the bike he steals lets him off the hook at the end, but he's still desolate.
106.
by Straw 4y ago
For inference, sure. Not yet for training. Weight quantization is much easier than weight + activation.
107.
by Straw 4y ago
A community built, evolving one for characters: https://en.akinator.com/ Its genuinely hard to find someone (real or fictional) who it can't find!
108.
by Straw 4y ago
I've been thinking about trying to build something from scratch with a similar spirit, but very different methods, but I also doubt that my predictions will be any good without far more time investment than I have available. As you say
109.
by Straw 4y ago
How difficult is it to get investors when you can show your model beats the market consistently? Of course, they have to check your not trading a strategy with extreme tail risk, but here it sounds like that's not the case?
110.
by Straw 4y ago
This article makes a number of misleading claims I think stemming from a misunderstanding of inflation- which does not mean "price increase" but rather "a general increase in the prices of goods and services in an economy&quo
111.
by Straw 4y ago
Isn't this the original article? Which doesn't actually look at IQ tests.
112.
by Straw 4y ago
Source?
113.
by Straw 4y ago
IQ in humans strongly correlates with almost any measure of performance you pick, so I think its definitely telling us some measure of capability even if not the whole story. I imagine people with ADHD do have trouble with some types of tas
114.
by Straw 4y ago
Nowhere do they discuss an actual IQ test! Last I checked, even the largest models fail hard on IQ tests, both visual and verbal. Of course they do find the underlying issue that benchmarks often don't test as much as we think they do.
115.
by Straw 5y ago
How much was SCP affected by TCP buffer size tuning?
116.
Training a Random Gaussian Generator(cp4space.hatsya.com)
3 points by Straw 5y ago|0 comments
117.
by Straw 5y ago
How does this compare to Japanese steels such as HAP40 and Aogami White/Blue/Super?
118.
by Straw 5y ago
Read "How to Win Friends and Influence People"- its old but human interaction hasn't changed that much. Its also just a matter of practice- as I'm sure you know from ML, no amount of reading can substitute for trying som
119.
by Straw 5y ago
I can't find any source in there suggesting a lack of effectiveness. The underlying source linked by the article mentions a reduction of 80% in symptomatic disease, which is responsible for most transmission. I can't find anything
120.
by Straw 5y ago
All the data suggests that vaccines significantly reduce transmission, both by decreasing the likelihood of infection and onward transmission from vaccinated individuals. Additionally, anything that incentivizes more people to get the vacci

More ›