toxik

27 ms·

331.
by toxik 3y ago
I saw LeCun give a keynote speech at ICML 2018, he did not mention planning but talked basically only about semi and self supervised learning.
332.
by toxik 3y ago
multiprocessing is almost never a good idea in my experience.
333.
by toxik 3y ago
It isn’t threadsafe in current Python.
334.
by toxik 3y ago
Surely you are joking? Your version is four times longer and repeats a large section of itself. The way to avoid control flow here is to simply find the rainbow first, take the sequence up until then, filter by yellow, and thingify the rem
335.
by toxik 3y ago
Of course, it’s a Google intern.
336.
by toxik 3y ago
Good tip! I might try this variable width idea. Seems a bit out there.
337.
by toxik 3y ago
Such an utter disgrace for Denmark and the Danish people. To sacrifice all your credibility in the Nordic and European community, to be the lapdog of a US intelligence agency. Absolutely pathetic.
338.
by toxik 3y ago
That’s not true and that was the entire point of my comment. It’s like playing chess, you’re not predicting your next best move, you’re searching for it.
339.
by toxik 3y ago
It is not accurate to say that an LLM like ChatGPT predicts anything. It is trained to maximize a score function, so it is more like trying to win a game where the moves are word choices.
340.
by toxik 3y ago
A lot of people use three screens. I don’t, I use one, but people do.
341.
by toxik 3y ago
That’s exactly my point, it ISN'T self explanatory when it’s mathematical code like this, expressed in terms of addition and multiplication. Understanding the dot product as “sum of multiplications” is an extremely shallow understandin
342.
by toxik 3y ago
I sincerely doubt you possess any kind of geometric understanding of what is going on in that code just from reading the first principles variant of cosine similarity.
343.
by toxik 3y ago
The cosine similarity is not inherently better suited for linear properties whatever that means, it’s just the cosine of the angle between two vectors. If the vectors are unit length, then it’s just the projection of one to the other.
344.
by toxik 3y ago
I disagree that your version is more readable. If you don’t know linear algebra, the code is inscrutable, and if you do, dot(x,y)/norm(x)/norm(y) is about as close to the math as you can get.
345.
by toxik 3y ago
The guy spamming all black is funny, as if it's going to run out of ink. It's a thermal printer.
346.
by toxik 3y ago
Hm, this method seems odd to me. Unbiased yes but then rounded to nearest float. Consider each floating point value a bin. Then this will fill bins where the coverage is sparse faster. Eg say the possible floats are 1, 10, 100 and so on - y
347.
by toxik 3y ago
Kinda sounds like it was a speedhack
348.
by toxik 3y ago
Using `i`, `j`, and `k` as index variables is not bad practice.
349.
by toxik 3y ago
Nobody questions the logical consistency of the system, I think. Presently, we use 24-hour time (AM/PM or otherwise), so the loss in precision is enormous going from 24 to 10. Think about it. A tenth of a standard hour is 6 standard mi
350.
by toxik 3y ago
I disagree, it’s not equal but different. The French famously tried to switch to a decimal system for angles before, but failed in no small part because of the relatively few unique prime factors. Being able to divide evenly by three turns
351.
by toxik 3y ago
I love how the example is super confusing. You sleep 3.33… metric hours. You fall asleep 9:50 metric time. You'd think we can just interchange the . and : so you should wake up 12:83…, or 2:83… the next day. But no, it's apparentl
352.
by toxik 3y ago
`ii` it is then
353.
by toxik 3y ago
I think about it this way p(th) = p(t) p(h) p(ht) = p(h) p(t) Hence p(th) = p(ht) regardless of coin imbalance as long as both events actually will happen. QED.
354.
by toxik 3y ago
This is simply not true, the pause token insertion is not situation dependent. If the model could say "please run me again", that would be groundbreaking and it has been looked into. The problem is how do you train such a model? I
355.
by toxik 3y ago
You still haven’t answered my one question: how is this different than just increasing embedding size? I can even think of a sketch proof to show this equivalence for single heads. The paper should have answered this question and well. It i
356.
by toxik 3y ago
Maybe something to do with the Transformers are mesa optimizers thing?
357.
by toxik 3y ago
I think I don’t understand what you are trying to say. Yes, FLOPs per token is the number we care about. Adding ~10x FLOPs and saying “look it’s better” seems odd. What’s worse, memory use is quadratic in sequence length so adding 10x token
358.
by toxik 3y ago
This isn’t exactly news. The question is if it’s better use of FLOPs than just having a larger network or embedding.
359.
by toxik 3y ago
That’s what I figured, you always lose in the end.
360.
by toxik 3y ago
I flushed a flush for this reason :(

More ›