pama

33 ms·

511.
by pama 2y ago
We know the desired random distribution of the 3rd order real polynomials (a product of uniform random distributions say), and I believe we can work out the distribution of the subset that have linear roots only and the fraction of random r
512.
by pama 2y ago
Thanks. The funky distribution of b has to be exactly the one that results in uniform (or the desired) random distribution of the real coefficients of the 3rd order real polynomial that is the resulting polynomial when you multiply your ex
513.
by pama 2y ago
The data suggests that the gender variance relates with the societal mixing of genders. The low population countries might simply have large variance due to noise but for all countries in the 10s of millions population or higher there appe
514.
by pama 2y ago
That’s why you have to do a double induction (I’m not a mathematician so my terminology may be wrong) and also keep proving that those special polynomials that are a product of a complex linear term with a polynomial of real coefficients al
515.
by pama 2y ago
Say that it is true that for degree N-1 polynomials the largest root is real with more than 0.5 probability. Then for degree N, either you multiply a degree N-1 with real coefficients by a linear (first order) polynomial with a real root th
516.
by pama 2y ago
Thanks. Finding the right combo to balance the largest root if it were non-real and end up with real coefficients seems harder than balancing a smaller root, ie the polynomial with the larger real root also has an n-1 degree polynomial wit
517.
by pama 2y ago
I am not sure why the point that reals appear to be more likely would be counterintuitive. If instead of uniform random real coefficients in an interval one picked uniform random roots in a circular disk centered at the origin of the compl
518.
by pama 2y ago
The full quote was not related to visual distraction (3 lines of config remove it all in Emacs) but rather: > I used Emacs a lot so my Ctrl key is remapped to CapsLock. I think such Control key position is way better and ergonomic, so co
519.
by pama 2y ago
Much faster yet than stable pytorch 2.3 (46% on A100, as per the tweet), and much much faster yet compared to pytorch 2.2, which was the stable version a couple weeks ago. Also llm.c is much faster yet when the performance comparison is on
520.
by pama 2y ago
An AI that can help referee papers to advance human knowledge doesn’t need to have lots of human qualities. I think it suffices if a) it has the ability to judge correctness precisely, and b) it expresses a degree of surprise (low log likel
521.
by pama 2y ago
I think information gain will be easy to measure in principle with an AI in the near future: if the work is correct, how unexpected is it. Anything trivially predictable based on published literature, including exact reproduction disguised
522.
by pama 2y ago
I feel you. Here are some thoughts from the other side of the fence: Social media banning aims to preserve anonymity when the reviews are blind. It is hard to convincingly keep anonymity for many submissions, but an effort to keep it is st
523.
by pama 2y ago
Sorry to hear all this (after writing my other sibling comment). Please don’t lose faith in the review process. It is still useful. Until the AGI can be better reviewers, which is hopefully not too far in the future.
524.
by pama 2y ago
I agree with almost all you said except that Twitter is better than top conferences, and I take a contrarian view that reviewers slow down AGI with requests for additional experiments. Without going into specifics, which you can probably gu
525.
by pama 2y ago
As you learn the key bindings keep thinking about how to use Emacs as your operating system and ask questions to experienced users in forums or to LLMs. Read the manual, slowly, over a period of weeks, then reread it. Eventually learn a bit
526.
by pama 2y ago
Also the cost of drug discovery, which in this case amounted to some doctor suggesting let’s mix those previously approved drugs and see what happens, compared to 5–10 years of research, failures, and systematic work by chemists and biologi
527.
Large Scale Transformer Model Training with Tensor Parallel(pytorch.org)
2 points by pama 3y ago|0 comments
528.
by pama 3y ago
Unless the date in the comment is wrong and I missed some important update in 2024, this statement is not correct. Apple Books a chunk of the latest tech for a period of many months every now and then but there is a long tail of companies
529.
by pama 3y ago
FYI: This does not work with iPhone lockdown mode.
530.
by pama 3y ago
I personally think that the model artifacts are simply programs with tons of constants. Many math routines have constants in their approximations and I don’t expect the source to include the full derivation for these constants all the time.
531.
by pama 3y ago
Isn’t a simple interpretation of this type of license that some people get the open source license and others get the commercial license? Almost like a switch statement for licenses. If you belong in the category that gets the commercial o
532.
by pama 3y ago
IIRC this model had finished pretraining in the summer of 2022.
533.
by pama 3y ago
I see. You are right. I also don’t think grok would be friendly to the home user.
534.
by pama 3y ago
Haha. Apologies. I hope I didn’t accidentally make anyone waste their time. If they did I’m sure there are people interested in hiring such people anyways so maybe it’s a time invested well in the end.
535.
by pama 3y ago
Sure but the point of the comment was SRAM. There is some confusion in a subset of the ML people about hardware memories, their latencies, and bandwidths. We don’t all need to write kernels like Tri Dao to make transformers efficient on GP
536.
by pama 3y ago
The H100 has 50MB SRAM (L2 cache) and does just fine. https://docs.nvidia.com/launchpad/ai/h100-mig/latest/h100-mi...
537.
by pama 3y ago
This would be the code to beat. Ideally with only 8 cores but any number of cores is also very interesting. https://github.com/gunnarmorling/1brc/discussions/710
538.
by pama 3y ago
Having write access to storage or spawning persistent daemons is an extra requirement and that is often not available in practice when evaluating contest code :-) This is a fun project for learning CUDA and I enjoyed reading about it——I jus
539.
by pama 3y ago
The AMD Ryzen 4800U has 8 cores total so the author follows the contest restriction. This CPU supports hyperthreading. (I’d be very interested in seeing hyperoptimized CUDA code using unlimited GPU cores FWIW.)
540.
by pama 3y ago
The performance report followed the initial request: run 6 times and remove the best and worst outliers, so the mmap optimization is fair game. Agreed that the C code has room left for some additional optimization.

More ›