pama

18 ms·

361.
by pama 2y ago
I think it is a good idea to slow down on hydrogen as a fuel for cars for a while, because, unfortunately, it does not work well within our modern civilization. Even tiny amounts of elemental hydrogen can cause serious corrosion on certain
362.
by pama 2y ago
The HN title shortening is a bit misleading. The paper systematically analyzes the burden of this known effect on 184 countries.
363.
by pama 2y ago
Agree that service in NYC these days is worse than it was about 15–20 years ago. At the time I didnt know that the MTA was the center of a political power play between city and state, depending on whimsical politicians in two centers to coo
364.
by pama 2y ago
Your company could locally host LLMs; you wont get chatGPT or Claude quality, but you can get something that would have been SOTA a year ago. You can vet the public inference codebases (they are only of moderate complexity), and you control
365.
by pama 2y ago
In your experience, why einx over einops+einsum?
366.
by pama 2y ago
Einsum immediately clicked with me because in my past advanced classical mechanics courses such concise contractions of multi-index creatures were really the only way to make quick sense of complicated problems without the limitations of lo
367.
ICLR: In-Context Learning of Representations(arxiv.org)
2 points by pama 2y ago|0 comments
368.
by pama 2y ago
If anyone from epoch.ai is reading this, it would be nice to link the toplevel result for o3 to this page.
369.
by pama 2y ago
These numbers are all mindboggling. I understand that the modern specs for EUV dont mean wire width, but if with this future NIL we truly get down to 8nm wide wires, perhaps we should start counting the number of atoms across the width of
370.
by pama 2y ago
Thanks. I don’t think we disagree on major points. Maybe there is a communication barrier and it may be on me. I came from a computational math/science/statistics background to ML. These next token prediction algorithms are of cou
371.
by pama 2y ago
“It was a world where a man would propose assassinating politicians, only to spark a debate about logistics.”
372.
by pama 2y ago
I should have emphasized the words "nontrivial ways" in my previous response to you. I didn't mean to emphasize compression and definitely not memorization, just the ability to also learn algorithms that can be evaluated by
373.
by pama 2y ago
Why does the limit on computational complexity of single decoder transformers matter for obtaining superhuman coding ability? Is there a theory of what level of complexity is needed for the task of coding according to a spec? Or the complex
374.
by pama 2y ago
The discussion cannot go deeper than the current level, unfortunately. One thing to not forget when thinking about decoder transformer models is that there is no limitation to having parts of the output / input stream be calculated by
375.
by pama 2y ago
I am very familiar with these and other clustering methods in modern ML, and have been involved in inventing and publishing some such methods myself in various scientific contexts. The paper I cited above only used 3 nearest neighbors as on
376.
by pama 2y ago
I recall early, incomplete speculation about transformers not solving Boolean circuit value problems; what did you think of this work? https://arxiv.org/abs/2402.12875v3
377.
by pama 2y ago
Thanks for the tip, though I’m not sure how complexity theory will explain the impossibility of superhuman results. The main advantage ML methods have over humans is that they train much faster. Just like humans, they get better with more t
378.
by pama 2y ago
I agree. However, my point is that they have to compress information in nontrivial ways to achieve their goal. The typical training set of modern LLMs is about 20 trillion tokens of 3 bytes each. There is definitely some redundancy, and t
379.
by pama 2y ago
My main concern with the simplification of memorization or near neighbor interpolation that is commonly assumed for LLMs is that these methods are ineffective at scale and unlikely to be used by decoder transformers in practice. That paper
380.
by pama 2y ago
To the downvoters: I am curious if the downvoting is because of my speculation, or because of the difference in understanding of decoder transformer models. Thanks!
381.
by pama 2y ago
> I wish people would understand what a large language model is. I think your view of llm does not explain the learning of algorithms that these constructs are clearly capable of, see for example: https://arxiv.org/abs&#x
382.
by pama 2y ago
I wanted to like kitty and tried it many times. It is subtle issues that break Emacs now and then, like breaking the display alignment for some zero-width joined emoji. Iterm2 on a MacBook is snappy for me. With remote work, the latency for
383.
by pama 2y ago
Being able to easily set it up so it sends key combinations to remote servers at will (including ctrl/meta/shift combos) was my main reason; this enabled my seamless interactions with remote Emacs servers. 24bit color and italics
384.
by pama 2y ago
Iterm is better documented. Try finding how to pass Ctrl-Meta-key; better yet, throw in a shift as well. Setting up 24bit color, supporting italics, and allowing Emacs keystrokes to go through to the remote server at will is a quality of l
385.
by pama 2y ago
This workshop contribution is OK, and the benchmark is somewhat valuable even without the rephrasing part of the problems, but the rephrasing (of only a small number of problems) sometimes genuinely makes the problem more confusing to human
386.
by pama 2y ago
Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default op
387.
by pama 2y ago
The risk of getting diagnosed with cancer fades over 80 years old. Tumor cells might take many years, or decades to develop into a diagnosable form of cancer, so much of the story relates to things that happen well before the 80+ years det
388.
by pama 2y ago
What you use now is a simple KNN classifier, and if it works well enough, perhaps no need to go much further. If you need to squeeze out a couple additional percentage points maybe try a different simple and robust ML classifier (random for
389.
by pama 2y ago
After failing with three reasonable ideas, they solved the problem with an idea that previously would have seemed unlikely to work (get similarity to past known nits and use a threshold of 3 similar nits above a certain cutoff similarity sc
390.
by pama 2y ago
It is made by a relatively trivial change to CDCA (chenodeoxycholic acid), which in turn is made in the liver by cholesterol in a number of steps. I love the simplicity of stepwise modifications of complex molecules inside the body with the

More ›