thesz

24 ms·

451.
by thesz 2y ago
> short of regressing to in-order, non-speculative cores I guess you are referring to a GPU cores here. It is a joke but can hint that in-order non-speculative cores are powerful computers nonetheless.
452.
by thesz 2y ago
Intellect Amplifier?
453.
by thesz 2y ago
If you have been training in different gear than is supplied to you by host of the event then you are at disadvantage. Take a look at the variety of weightlifting shoes, as an example of gear for a specific sport.
454.
by thesz 2y ago
I do not know. I just pointed out that progressive taxation makes UBI more sensible in the "more equitable society" sense. But I do not like the notion of "more equitable society" at all. I do not think it is fair or use
455.
by thesz 2y ago
With a progressive taxation, the gap between high and low income narrows. Low income people may not even notice a change in tax bracket, if any such thing happen, high income people will give back a substantial part of that additional incom
456.
by thesz 2y ago
I want to migrate from C++ because these languages are fashionable and hip and have more developers available and because migration with regular expressions is easy, apparently. Actually, a colleague of mine asked me why people keep using b
457.
by thesz 2y ago
> The engineering cost of migrating code bases is trending to 0 I work with code base of >750K LOC C++ that is 12+ years old and would like to migrate it to something fashionable like Futhark or Python. So, please, tell me mor
458.
by thesz 2y ago
Here's the link about neural scaling law: https://en.wikipedia.org/wiki/Neural_scaling_law Can you make a napkin calculation of how much better training should be, how larger models and datasets should be to overc
459.
by thesz 2y ago
The paper refers to the paper of Meijer et al from 1991: https://maartenfokkinga.github.io/utwente/mmf91m.pdf Paramorphisms' section is at page 5. It refers to the 1990 paper by Lambert Meertens that your referenc
460.
by thesz 2y ago
> You call that censorship but we know that allowing hate to be spewed unconditionally is itself a form of silencing others. How is that? And who are we?
461.
by thesz 2y ago
As someone with highly unpronoceable nickname said, my only complaint is that only first order methods are used. Second order methods are fun, actually. I like them. ;)
462.
by thesz 2y ago
This is why I mentioned batch gradient line search. You can combine it with conjugate gradient. And small LeNet (I think it is first convolutional network that obtained good score on MNIST) is orders of magnitude bigger than KAN's in t
463.
by thesz 2y ago
I can only state that KAN paper used LBFGS and that LBFGS is remarkably different from SGD. One of the differences is a dynamic learning rate guided by approximation of the local curvature.
464.
by thesz 2y ago
From the article: Our careers are defined by the highest moments of its biggest upside swings. The "highest moment of biggest upside swing" can be likened to winning a jack pot. A not solid career or life advice. In my opin
465.
by thesz 2y ago
The original paper [1] used LBFGS [2], it is quasi-second-order optimization algorithm. [1] https://arxiv.org/pdf/2404.19756 - "Both MLPs and KANs are trained with LBFGS for 1800 steps in total." [2] http
466.
by thesz 2y ago
https://cacm.acm.org/news/nist-post-quantum-cryptography-can... "Belgian researchers have cracked the SIKE cryptographic algorithm, a fourth and final-round candidate that the U.S. National Institute of Standards
467.
by thesz 2y ago
It appears that sourdough is relatively healthy: https://pubmed.ncbi.nlm.nih.gov/18317680/ I found that sourdough from local restaurant did not raised blood glucose level significantly (well below the threshold of high
468.
by thesz 2y ago
The algoithm itself is about the use of golden ratio for interval search. One can use subdivision, other can use ternary division. And another one can use golden ratio.
469.
by thesz 2y ago
Then you possibly will find this useful: https://en.wikipedia.org/wiki/Golden-section_search
470.
by thesz 2y ago
This can be seen as encoding ranges that span each character. E.g., "asdoasd" needs to encode just 3 for 'o', remove 'o' from "asdoasd" and encode 0 and 2 for 'a', then 0 and 1 for 's&#
471.
by thesz 2y ago
Let me quote the first one, which is meta-analysis: ". Of the articles published prior to the COVID-19 pandemic, 79% (n = 19) demonstrated that work-from-home increased productivity and performance whereas 21% (n = 5) showed mixed or n
472.
by thesz 2y ago
This is obvious in retrospect - instead of making LLMs work better, LLM's handlers invented various techniques to make LLMs to look like they work better, one such example is summarization. Next gen LLMs then get trained on that data.
473.
by thesz 2y ago
GrokFast strongly reminds me of Stochastic Average Gradient Descent: https://www.cs.ubc.ca/~schmidtm/Courses/540-W19/L12.pdf Both use averaging.
474.
by thesz 2y ago
My experience suggests that LLMs become not less accurate, but less helpful. Two years ago they output a solution for my query [1] right away, now they try to engage user to implement that thing. This is across the board, as far as I can
475.
by thesz 2y ago
> So, you're betting on no significant cost reduction of compute hardware? Seems implausible to me. This is not about compute, but about data. https://arxiv.org/abs/2404.04125 "...our study reveals an expo
476.
by thesz 2y ago
"Doing" is also non-normally distributed, because you cannot do something in a negative amount of time.
477.
by thesz 2y ago
Thanks, very insightful. "Dictionary Encoding is effective across data types (even for floating-point values) because most real-world data have low NDV ratios. Future formats should continue to apply the technique aggressively, as in P
478.
by thesz 2y ago
Of course. But nobody tells me that I can hit a hard limit and then I need a second Parquet file and should have some code for that. The situation looks to me as if my "Favorite DB server" supports, say, only 1.9 billions records
479.
by thesz 2y ago
I do not know a good one. A former colleague of mine is now working on a memory-mapped log-structured merge tree implementation and it can be a good alternative. LSM provides elasticity, one can store as much data as one needs, it is static
480.
by thesz 2y ago
Parquet is not HDFS. It is a static format, not a B-tree in disguise like HDFS. You can have compressed Parquet columns with 8192 entries being a couple of tens bytes in size. 600 columns in a row group is then 12K bytes or so, leading us t

More ›