pama

13 ms·

211.
by pama 1y ago
Pixi inherited some ot the bad designs from conda, and conda clearly hadnt been close to solving python packaging either and was digging itself into a black hole over time.
212.
by pama 1y ago
These tools together solve a fraction of the problen. The other parts of the problem are interfacing with classic c, c++ libraries and handling different hardware and different OSes. It is not even funny how tricky it is to use the same GPU
213.
by pama 1y ago
The math in the original paper is questionable. By leaving free the choice of divergence in Eq 3, Eq 4 has no practical value except when said divergence is zero exactly.
214.
by pama 1y ago
It is the other way around. If the data is causal and presented in the causal order, it is impossible to beat the loss of a pure auto-regressive model because it has the correct probability distribution for the dataset. Language data is mos
215.
by pama 1y ago
In the case of the access to private data, I think that the concern I mentioned is not fully alleviated by simply cutting off exposure to untrusted content. Although the latter avoids a prompt injection attack, the company is still vulnera
216.
by pama 1y ago
Agreed on all points. What should one make of the orthogonal risk that the pretraining data of the LLM could leak corporate secrets under some rare condition even without direct input from the outside world? I doubt we have rigorous ways t
217.
by pama 1y ago
If you want to go down the rabbit hole of the state of the art, I recommend the EAGLE3 paper: https://arxiv.org/abs/2503.01840
218.
by pama 1y ago
Thanks.
219.
by pama 1y ago
Why so much power for a simple NAS? Wouldnt a Pi5 be sufficient?
220.
by pama 1y ago
That’s a great reminder that what humans consider random and what is a truly random sets of states with high entropy are typically different things. In this case out of 43 quintillion combinations, there’s only 48 that fit the human imposed
221.
by pama 1y ago
Or simply upgrade to the top of tree version of Emacs that now has native mouse support in the terminal.
222.
by pama 1y ago
I think that a powerful application of AI is the breaking of apps. Vibe breaking rather than vibe coding. AI can help immitate nasty users, find edge cases, hack systems, and break them under load. If you have a non-graphical intermediate
223.
Personal Superintelligence(meta.com)
1 points by pama 1y ago|2 comments
224.
by pama 1y ago
An alternative and perhaps easier way to estimate the relative importance of the GPU cost vs the electricity cost is to estimate how many years of constant use of the GPU at full power you need for the cost of industrial-scale electricity t
225.
by pama 1y ago
Anyone here has tips for the code and hardware setup to get best per-GPU throughput on H200 or B200 hardware for large reasoning traces and inputs of around 10k–40k tokens? Is there an equivalent effort to sglang’s optimization of the V3&#
226.
by pama 1y ago
This sounds very reasonable to me. When considering top tier labs that optimize inference and own the GPUs: the electricity cost of USD 5000 at a data center with 4 cents per kWh (which may be possible to arrange or beat in some counties in
227.
by pama 1y ago
> If I make the plausible but not necessarily correct assumption that OpenAI's API prices reflect the cost of electricity, none of their models are even remotely that cheap This assumption is very wrong. The primal cost factor in in
228.
by pama 1y ago
> > V3/R1 scale models as a baseline, one can produce 720,000 tokens On what hardware? At how many tokens per second? But most importantly, at what quality? The hardware is the GB200 NVL72 by NVidia. This is for the class of 671B
229.
by pama 1y ago
Agreed. And that near zero work has a near zero energy cost. In addition, silicon inference (combining hardware and software advances) continues to be optimized and become more energy efficient at a rapid rate. There exists an unfounded my
230.
by pama 1y ago
Silicon is already more efficient for inference than the brain. If we use centralized decoding of the V3/R1 scale models as a baseline, one can produce 720,000 tokens (a wild guess for the tokens humans could produce in 24 hours) usin
231.
by pama 1y ago
I would simply call it a healthy level of curiosity :-)
232.
by pama 1y ago
I agree vague language in popular press is sometimes annoying. “Off by 17 orders of magnitude” would be off by 136 billion years, so not that much for sure. Assuming you want to be able to test the plant and or maintain it once per year, 43
233.
by pama 1y ago
Read till the end—what you ask is the last table.
234.
by pama 1y ago
> I can't find a single open source codebase, actively used in production, and primarily maintained and developed with AI This popular repo (35.6k stars) documents the fraction of code written by LLM for each release since about a y
235.
by pama 1y ago
I didnt read your draft paper, but your premise in HN sounds a bit off to me. AGI does not assume the ability for finding or learning an optimal solution to every problem (with that assumption it would be trivial to prove it impossible in m
236.
by pama 1y ago
I use a personally-adjusted variant of M-x shell for most of my last 20 years of Emacs use (after trying various hacks including things mentioned in this article). I end up having about 20–100 named shell buffers on each machine, different
237.
by pama 1y ago
Cool. Would it be possible to eliminate that little vocab format conversion requirement for the vocab I see in the test against tiktoken? It would be nice to have a fully compatible drop in replacement without having to think about details.
238.
by pama 1y ago
V3 is number 5 in your list. R1-0528(free) is number11 and R1(free) is number15. Openrouter separates the free (in the top 20 list you shared) vs paid (further down) instances of V3 and R1, and of course it doesnt count the direct connectio
239.
by pama 1y ago
Not sure what you are referring to—do you have a pointer to a technical writeup perhaps? In training and inference MLA has way less flops than MHA, which is the gold standard, and way better accuracy (model performance) than GQA (see compar
240.
by pama 1y ago
I had related questions and checked out the project a bit deeper though I havent tested it seriously yet. The project did start work over a year ago based on relevant papers, before vllm or sglang had decent solutions; it might still be add

More ›