gdiamos

20 ms·

271.
Training LLMs in the Wild Wild West – AMD multi-node(lamini.ai)
2 points by gdiamos 3y ago|0 comments
272.
by gdiamos 3y ago
Technologies usually follow a learning curve eg wrights law, moores law, LLM scaling laws AI seems to add another dimension of explosive hype
273.
by gdiamos 3y ago
That would be strong terms, but then they would need to find a way to spend $1B. Here's a proposal. Send $500M to NVIDIA for GPUs. Send $100M to AMD to balance the odds. Spend $100M to build a new chip as a hail mary. Spend the rest
274.
by gdiamos 3y ago
These were a huge pain in the ass when I tried this 20 years ago on Ocelot. Eventually one of the NVIDIA engineers just asked me to join and I did. :-P
275.
by gdiamos 3y ago
https://github.com/lamini-ai/lamini-sdk/tree/main/03_RAG You can implement RAG in 80 lines of python and 0 SaaS libraries. It's extremely easy. 1. Load your data as a giant string (streaming) 2. Chu
276.
by gdiamos 3y ago
Not literally infinite, but Llama2 scale models can handle about 10 trillion tokens.
277.
by gdiamos 3y ago
I think we are saying the same thing. The key ingredients are: reasoning(skills) + knowledge + important bits/facts The best systems have all of these
278.
by gdiamos 3y ago
I really don’t get this sentiment - why not do both? Retrieval allows looking up facts - eg in a Google search Finetuning allows reasoning using new knowledge. Humans do both.
279.
by gdiamos 3y ago
Glad to see that more people outside the big ai labs are figuring out how to do fine tuning. Some open source LLM authors also seem to have figured it out. I think many users get put off it because just pushing a button doesn’t work and th
280.
by gdiamos 3y ago
There’s a difference between the computation requirements of long context lengths and the accuracy of the model on long context length tasks.
281.
by gdiamos 3y ago
Prototyping is one scenario I have seen this in. Prototyping is iterative - you experiment with the chunk size, chunk content, data sources, data pipeline, etc. every change means regenerating the embeddings Another one is where the data
282.
by gdiamos 3y ago
This is exactly what I do. No one talks about how many GPUs you need to generate enough embeddings that you need to do something else. Here's some back of the envelope math. Let's say you are using a 1B parameter LLM to generate
283.
by gdiamos 3y ago
“Larry and Sergey were smart guys who recognized they didn’t know shit about corporateness and quickly got into a pattern of hiring and empowering psychotic pricks who were presumably “good at business”.” That sounds pretty negative about p
284.
Open source ICD11 Mistral trained on AMD GPUs [video](youtube.com)
2 points by gdiamos 3y ago|1 comments
285.
by gdiamos 3y ago
I open sourced all of the code for this: https://github.com/lamini-ai/lamini-sdk/tree/greg.cpt-gpt It's based off of this SDK: https://github.com/lamini-ai/lamini-sdk I walked throu
286.
by gdiamos 3y ago
It’s great to win, until you lose.
287.
by gdiamos 3y ago
When did that happen? I used to write a lot of C++, but it's been years since I've had to use it everyday.
288.
by gdiamos 3y ago
From the article: "The crux is that while vector search is better along some axes than traditional search, it's not magic. Just like regular search, you'll end up with irrelevant or missing documents in your results." RA
289.
by gdiamos 3y ago
If anyone wants access to mi250 or mi300 today, contact us here: https://www.lamini.ai/contact If you are a LLM startup blocked on compute, send me a dm https://www.linkedin.com/in/gregory-diamos-1a8b90
290.
by gdiamos 3y ago
I host them here: https://app.lamini.ai/playground You can play with them, tune them, and download the weights It isn’t exactly the same as open source because weights != source code, but it is close in the sense that it is
291.
Good old fashion ML is dying
20 points by gdiamos 3y ago|16 comments
292.
by gdiamos 3y ago
LLMs popularized zero-shot learning, or “prompt engineering” which is drastically easier to use and more effective than labeling data. You can also retrofit “prompt engineering” onto good old fashion ML like text classifiers. I wrote a lib
293.
by gdiamos 3y ago
Jensen isn't a normal CEO
294.
by gdiamos 3y ago
Where are they going to get the compute or the data?
295.
by gdiamos 3y ago
I thinks it’s sad that LLMs have become so hostile to builders. It doesn’t have to be this way.
296.
by gdiamos 3y ago
Doesn’t work
297.
by gdiamos 3y ago
Fast switching among many LoRAs is already supported in Lamini. Great paper talking through some of the design issues and optimizations, eg Orca and custom fused GPU kernels. I’d love to see more research in this direction.
298.
by gdiamos 3y ago
I was in the same situation as you, and then I tried it a few years ago. I realized that front end software is actually pretty shallow. Design is important, deep, and requires talking to users, but once you have a design it is pretty straig
299.
by gdiamos 3y ago
Researchers are welcome to try our platform, and give us feedback Send me a note on linkedin, and I can grant expanded functionality, eg access to more models. We generally believe in the value of AI research and will be supportive where po
300.
by gdiamos 3y ago
We showed this in our original scaling law paper: https://arxiv.org/pdf/1712.00409.pdf The slope of the power law is determined by the problem and dataset. Compute, parameter count, and data move you along the curve.

More ›