gdiamos

11 ms·

121.
by gdiamos 1y ago
The NVIDIA stock price keeps going up because accuracy/intelligence is more valuable than efficiency Scaling laws let you spend more transistors and watts on intelligence Do you want more tokens or smarter tokens?
122.
by gdiamos 1y ago
As someone who listened to the music, it’s surprising to see this as the top comment. Yes their lyrics are dark. That was the point. Eating animals isn’t what comes to mind for me. I also rationalize it that 100s of millions of animals are
123.
by gdiamos 1y ago
That’s only if you are planning to jump 100 more times.
124.
by gdiamos 1y ago
probability is memoryless. If you have been base jumping for 20 years, you have the same risk on your next jump as someone trying it for the first time.
125.
by gdiamos 1y ago
How does it handle images?
126.
by gdiamos 1y ago
Nice work anton et al. I hope you continue the 50-100M parameter models. I think there is a case for models that finish fast on CPUs in solve by llm test cases.
127.
by gdiamos 1y ago
Thanks, I missed a factor of 1000x, it should be a million billion
128.
by gdiamos 1y ago
Yes I agree you can tune autoregressive LLMs You can also tune diffusion LLMs After doing so, the diffusion LLM will be able to generate more tokens/sec during inference
129.
by gdiamos 1y ago
This sounds like a strawman. GPUs can do 1 million trillion instructions per second. Are you saying it’s impossible to write a test that finishes in less than one second on that machine? Is that a fundamental limitation or an incredibly ine
130.
by gdiamos 1y ago
I think the LLM dev community is underestimating these models. E.g. there is no LLM inference framework that supports them today. Yes the diffusion foundation models have higher cross entropy. But diffusion LLMs can also be post trained and
131.
by gdiamos 1y ago
I used to think I needed to type faster. As I get older I spend more of my coding time on walks, at the whiteboard, reading research, and running experiments
132.
by gdiamos 1y ago
Go with anthropic
133.
by gdiamos 1y ago
Great write up. We use vLLM kv cache and continuous batching as a foundation for requests in ScalarLM and also add batching optimizations in a centralized queue and by adding explicit batching support in our client. https://www.s
134.
by gdiamos 1y ago
Wow - the title is "basic facts" - but it should be "key insights" You wouldn't believe how many PhDs I've met who have no idea what a roofline is.
135.
by gdiamos 1y ago
I mean broke compared to Meta or OpenAI. AI research used to be fringe and not well funded. Back in those days, 99.9% of hardware was Xeon.
136.
by gdiamos 1y ago
startups and researchers are broke, just like Geoff Hinton in 2006 - https://blog.waqasrana.me/assets/papers/hinton2006.pdf
137.
by gdiamos 1y ago
The future part didn’t land for me at all. To me it just goes completely off the rails. “”” 2035: AI is 10,000 times smarter than the smartest human.7 It composes “A Brief, Exhaustive and Completely Correct History of Robotics” which is muc
138.
by gdiamos 1y ago
Was this thing written by ChatGPT?
139.
by gdiamos 1y ago
It's surprising to me that the field is willing to invest this much in mega-kernels, but not models that generate multiple tokens in parallel...
140.
by gdiamos 1y ago
An alternative to prefix caching?
141.
by gdiamos 1y ago
Congrats on the great run and the future. Jemalloc was an inspirational to many memory allocators.
142.
by gdiamos 1y ago
Finetuning is deep learning training. It's pretty difficult to get right. Andrej's 2019 blog laments on some of the reasons why it is hard and I can relate to a lot of this - https://karpathy.github.io/2019/04
143.
by gdiamos 1y ago
It's pretty frustrating to spend weeks on finetuning and end up with a model that says: "SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT
144.
by gdiamos 1y ago
explicit is better than implicit
145.
by gdiamos 1y ago
Where is the diffusion LLM inference framework so people can run these models?
146.
by gdiamos 1y ago
If you have two methods that both improve accuracy, why not stack them?
147.
AMD MI300X Memcpy Peer Deep Dive(scalarlm.com)
1 points by gdiamos 1y ago|0 comments
148.
by gdiamos 1y ago
It doesn’t exist. There is an opportunity to build it. I think the problem with this gen of companies isn’t tech, it’s culture.
149.
by gdiamos 2y ago
If the cost of labor plummets I wonder if it will accelerate the timeline for AI, automation, and robotics. I gave this a shot at home a month ago by cutting my kids off from toys online and getting them a 3D printer. They have already pri
150.
by gdiamos 2y ago
But I thought semiconductor manufacturing was high margin?

More ›