pama

13 ms·

241.
by pama 1y ago
Is your aim targetting the inference at scale or specialized/new/simpler inference pipelines? Sglang and vllm have disaggregated prefix and decoding serving (eg https://docs.vllm.ai/examples/online_serving&#x2
242.
by pama 1y ago
The best places to work at are full of people that are intrinsically motivated. “Good job” is implied. Feedback, including criticism, is expected, as it helps improve things further or recognise perfection/good enough. Academics do hav
243.
by pama 1y ago
Maybe this is the better link: https://www.nvidia.com/en-us/geforce/news/rtx-5050-desktop-g...
244.
by pama 1y ago
The performance figure in the link clearly says that it it’s a significant improvement to the 3050.
245.
by pama 1y ago
The output of the verilog optimizer is different every time. The output of a fab is different in every batch. Each chip in a batch is different from others in that batch. Quality control drops the fraction of truly poor chips, and hardware
246.
by pama 1y ago
Your experience with fabs must be somewhat limited if you think that the state of the art in fabs produces deterministic results. Please lookup (or ask friends) for the typical yields and error mitigation features of modern chips and try to
247.
by pama 1y ago
Pricing is a hard problem. Theoretically, if companies occasionally raise prices dramatically once something is useful, they sometimes can create early demand and more testers for future product releases. Ofc they have to be careful to avo
248.
by pama 1y ago
“While we strive to maintain consistent pricing between preview and stable releases to minimize disruption, this is a specific adjustment reflecting Flash’s exceptional value, still offering the best cost-per-intelligence available.”
249.
by pama 1y ago
Thanks. Done.
250.
by pama 1y ago
First world problems: can I email you and specify a later shipping date?
251.
by pama 1y ago
Please read their report. There is no bragging. It just tries to document performance and clarify a misconception. The concept that LLM inference may not be profitable or may be energy inefficient has been a constant song of misinformation
252.
by pama 1y ago
The current version of sglang allows inference with the R1 model at a cost that is very close to the rate that DeekSeep claimed (using H100s, not exactly the DeepSeek compute). Their claim is almost validated by replication at this point so
253.
by pama 1y ago
Please read the DeepSeek analysis of their API service (linked in this article): they have 500% profit margin and they are cheaper than any of the US companies serving the same model. It is conceivable that the API service of OpenAI or Anth
254.
by pama 1y ago
How did the pelicans of point releases of V3 and of R1 (R1-0528) do compared to the original versions of the models?
255.
by pama 1y ago
Maybe I'm a hoarder. My timestamped histories from four machines in four different locations added up to 898028 lines --- I kept most of my shell commands, including errors and iterative attempts to get a good command, since early 2
256.
by pama 1y ago
Lots of the numbers here are confusing in relation to other numbers… “The researchers found that just under 3% of people in the UK, France and Germany, and 4% of those in the US have MASH, but diagnosis rates were below 18%. That means abou
257.
by pama 1y ago
Thanks. As an EU citizen am I exempt from this order? How does the judge or the NYTimes or OpenAI know that I am an EU citizen?
258.
by pama 1y ago
You may have already noticed that many providers have separate, much lower, prices for offline inference.
259.
by pama 1y ago
Thanks! It seems to me that the performance gain here was due to a smaller vocab size. This type of change is almost guaranteed to backfire for larger models and larger datasets / lower loss requirements, so it probably is not very u
260.
by pama 1y ago
What exact attempt with which Tokenmonster vocab do you refer to? Sometimes it is hard to conclude much from these efforts. For example, having a smaller vocabulary is typically only useful for small models where the compute cost of the sof
261.
by pama 1y ago
https://x.com/deepseek_ai/status/1928061589107900779?s=46
262.
DeepSeek-R1-0528 performance improvements(huggingface.co)
14 points by pama 1y ago|3 comments
263.
by pama 1y ago
Any untrue/invalid idea is bad. So it is not always subjective. “Tigers are friendly pets and every apartment in SF should have one” is an obviously bad idea.
264.
by pama 1y ago
I have nothing against random anecdotes per se, but a lot of academic code does not correctly optimize computation on GPU hardware. If you can estimate by pen and paper how many FLOPs/s your code was using because of the main operation
265.
by pama 1y ago
Nice image of your poster!
266.
by pama 1y ago
Another intuition is simply that anytime your causal relationships in the training data are sequential you are having a lower probability of getting the correct token at a certain position because you have less of the causal information lea
267.
by pama 1y ago
You cannot argue successfully in court that the copy of the binary compiled code is not a copy of a substantial portion of the software. The fix is very trivial. This should not be an open issue.
268.
by pama 1y ago
Except Singapore no longer has a large amount of jungle to build on.
269.
by pama 1y ago
Not sure I understand—healpix starts from the rhombic dodecahedron and then bisects the generalizations of the 12 squares each time. Where do octahedra come into play?
270.
Modular Platform 25.3: 450K+ Lines of Open Source Code and Pip Packaging(modular.com)
1 points by pama 1y ago|0 comments

More ›