gdiamos

10 ms·

151.
by gdiamos 2y ago
Why do you say that? Because the components are going to become more expensive? Because the US labor is more expensive? Do you know how it breaks down?
152.
by gdiamos 2y ago
If I wanted to source semiconductors from the US to avoid tariffs, how would I go about doing that? Don’t micron, intel, and GF have US fabs?
153.
by gdiamos 2y ago
ScalarLM.com
154.
by gdiamos 2y ago
It’s good for the community to move beyond drama and build
155.
by gdiamos 2y ago
Nice to see a return to open source in models and training systems.
156.
by gdiamos 2y ago
Here’s a true story I find funny about scaling laws at Baidu. From 2016-17 I did a projection using our scaling law equation with my coauthors about how many GPUs it would take to train an LLM with a step function in capability. Joel Hest
157.
by gdiamos 2y ago
Gold
158.
by gdiamos 2y ago
Sure, but why would one prefer tanh instead of normalization layers if they have the same accuracy? I suppose normalization kernels have reductions in them, but how hard are reductions in 2025?
159.
by gdiamos 2y ago
What are the practical implications of this?
160.
by gdiamos 2y ago
I’m sure there are ways of bypassing scaling laws, but I think we need more research to discover and validate them
161.
by gdiamos 2y ago
max output size is always limited by the inference framework in autoregressive LLMs eventually they run out of memory or patience
162.
by gdiamos 2y ago
Diffusion LLMs also follow scaling laws - https://proceedings.neurips.cc/paper_files/paper/2023/file/3...
163.
by gdiamos 2y ago
I think these models would get interesting at extreme scale. Generate a novel in 40 iterations on a rack of GPUs. At some point in the future, you will be able to autogen a 10M line codebase in a few seconds on a giant GPU cluster.
164.
by gdiamos 2y ago
Bidirectional seq2seq models are usually more accurate than unidirectional models. However, autoregressive models that generate one token at a time are usually more accurate than parallel models that generate multiple tokens at a time. In d
165.
by gdiamos 2y ago
Scaling laws just say that accuracy improves predictably - https://arxiv.org/pdf/1712.00409 That’s what we put in our 2017 paper. It does mean that there is a simple way to keep building smarter LLMs. I have never seen
166.
by gdiamos 2y ago
That’s how clouds try to lock you in, by making you use a custom tool that is different for the sake of being different. If you use standard tools you don’t have this problem. Containers running on VMs is standard. A mesh of microservices t
167.
by gdiamos 2y ago
I got into a fight with them kind of like Trump and Zelensky just did. Not a technical reason. Sometimes in business the deal falls through I was on the receiving end and didn’t appreciate it.
168.
by gdiamos 2y ago
Meh, as a developer who lived through the 2000-2010 era Microsoft, it’s easy to come up with a laundry list of reasons to hate on Azure. But I tried Azure for my most recent startup because I was offended by AWS, and GCP did not have enough
169.
by gdiamos 2y ago
Once the shift to diffusion LLMs starts, it will be swift. They are much more computationally efficiency and scalable.
170.
by gdiamos 2y ago
Great idea
171.
by gdiamos 2y ago
The problem is no differentiation If you have to do more work to port for no benefit, it doesn’t make sense except to commoditize it
172.
by gdiamos 2y ago
There are at least three technical reasons why none of these accelerators have worked: 1) systolic arrays without SIMT or general purpose multicore are not flexible enough, creating a flawed foundation 2) everyone targeting tsmc is building
173.
by gdiamos 2y ago
They are derived from statistical laws unlike moores law https://en.m.wikipedia.org/wiki/Chernoff_bound I agree with you that they require data
174.
by gdiamos 2y ago
Some interesting notes about the project if folks want to dig deeper: There is a dev mode that runs on a laptop: - https://docs.cray-lm.com/deployment/laptop There is a unified dockerfile for building vLLM and the
175.
by gdiamos 2y ago
Some of the authors are available to discuss the current code or the roadmap.
176.
Craylm: Open-source unified LLM training and inference for R1(blog.cray-lm.com)
9 points by gdiamos 2y ago|2 comments
177.
by gdiamos 2y ago
This argument ignores scaling laws
178.
by gdiamos 2y ago
It goes up at least until LLMs match humans - ie until an LLM can write Windows
179.
by gdiamos 2y ago
GEMM has been the workhorse of machine learning. It’s amazing how we’ve ratcheted up the TFLOPs over the years. I wonder what other algorithms allow hardware optimization like this.
180.
by gdiamos 2y ago
The instruction set is, not necessarily the implementation.

More ›