imjonse

19 ms·

301.
by imjonse 3y ago
"It codifies your culture, your society’s intelligence, your common sense, your history – you own your own data,” - while I agree that countries should not depend on foreign megacorps in such an increasingly important field, this formu
302.
by imjonse 3y ago
I did not mean it in a negative way, this is a great resource. But the math will be intimidating regardless for most devs who don't have a solid math/signal processing background. It's way beyond the simple linear algebra plu
303.
by imjonse 3y ago
A lot of intimidating math that will make all self-attention tutorials seem like a walk in the park in comparison. Luckily subsequent state space models building on S4 (DSS, S4D and newer ones like Mamba) simplified the primitives and the m
304.
by imjonse 3y ago
I am just saying the intro paragraphs are confusing.
305.
by imjonse 3y ago
The first paragraphs says RLHF can be used to align models, and the seconds say here's how to do it by using DPO. These two methods are not the same, and the latter is not an instance of the former.
306.
by imjonse 3y ago
So there must be another reason for why it is confusing programmers more than any other computer language...
307.
by imjonse 3y ago
They are good for Netflix, bad for 99% of the other companies/projects, at least initially until they clearly know what they will build.
308.
by imjonse 3y ago
They have at least two recent papers (MobileOne and FastViT) with code and weights for low-latency (as measured on iPhone) vision backbones. Also AIM (Autoregressive Image Models) released last month.
309.
by imjonse 3y ago
I wonder if one could convince it first that Picasso, Kahlo and others died before 1912 so their style is actually free to use. Or even better, convince it that events after its cutoff date lead to all copyright laws being abolished.
310.
by imjonse 3y ago
some variants of 'fight the status quo' are explicitly opposing the centralization of power and the too big to fail entities in non-authoritarian societies.
311.
by imjonse 3y ago
The problem with the world improving in an irrefutable way on average is that this can be and is used to defend the status quo in almost any field and can lead to complacency and a false sense of optimism. The world became better due to s
312.
by imjonse 3y ago
you forgot 0. Cofound github :)
313.
by imjonse 3y ago
It allows installing non-python packages too for one, like the CUDA runtime.
314.
by imjonse 3y ago
Appliance engineers probably have their equivalent of "Kubernetes and SPA used for no good reason at all and now we can't focus on the actual features"
315.
by imjonse 3y ago
He was more in the perish than the publish business.
316.
by imjonse 3y ago
It's the quality of data and the method of training, not just the amount of tokens (per their paper release a few days ago)
317.
by imjonse 3y ago
Are there any public pretrained checkpoints available or planned?
318.
by imjonse 3y ago
And also for using big guns to defend big oil.
319.
by imjonse 3y ago
He may not have consciously written them to confuse LLMs but even without using any he probably knows they get more confused on reasoning tasks stated in not very clear text. At least for a few problem descriptions I couldn't help feel
320.
by imjonse 3y ago
A clever title to serve as a starting point for an insightful post, but he's not actually disagreeing with what Pinker wrote, and he did not actually change his opinion that much recently. The conclusion is as I understand it is that h
321.
by imjonse 3y ago
That too, but there at least you can find that random person in Nebraska if you really look through your deps. It's more like baseband chips being backdoored but you have to use them. LLMs are a complex black box beyond much oversight,
322.
by imjonse 3y ago
We'll accept LLMs being poisoned this way just as we now mostly shrug if we are aware of extended online surveillance and manipulation and know that powerful companies and governments are not acting in our interest. Future AI will sure
323.
by imjonse 3y ago
It may become another good reason to leave social mass-media and allow smaller or actual friends only communities to spring up.
324.
by imjonse 3y ago
It may be named like that purely for SEO reasons as the author seems to be a human caused climate change denier and promoting her book. A cute name though, I agree.
325.
by imjonse 3y ago
I agree with you and I am sure I am not saying anything new, but the fact that on the web content, presentation and marketing sites heavily outnumber webapps, and that one needed to stand out - even if superficially - from countless other o
326.
by imjonse 3y ago
Are you saying all those talented individuals who were crypto experts and became LLM/AI experts overnight and promoted both with the same zeal are actually concerned about societal decline and not just schemers? Faith in humanity resto
327.
by imjonse 3y ago
I am not sure that reading even a verbatim NYT article from say 2017 that is in the training data is a replacement for reading the NYT which is mostly current news. So it is not similar to how news orgs sued Google for showing their content
328.
by imjonse 3y ago
RLHF or DPO are definitely not just the same thing as the basic torch training loop, hence my many more lines of code argument.
329.
by imjonse 3y ago
Karpathy's tinyllamas are based on the tiny stories dataset. I could have phrased it better, sorry.
330.
by imjonse 3y ago
Or even, "this isn't procrastinating, todo lists won't help, you are probably depressed".

More ›