gdiamos

19 ms·

301.
by gdiamos 3y ago
Happy to answer any questions about this.
302.
by gdiamos 3y ago
I really want to see a cleaned 10T token dataset for LLMs
303.
by gdiamos 3y ago
Terrible advice. If no one can understand your work, then no one will use it.
304.
by gdiamos 3y ago
It's available now with an enterprise license, because we validate that it is running correctly on a system we help configure. We will open source pieces of it over time. Our strategy is open source functional core. Eventually we will
305.
by gdiamos 3y ago
I completely agree. I wasted a lot of time just assuming that ROCm would work on Radeon, just like it does for CUDA.
306.
by gdiamos 3y ago
We kept hearing 52 weeks for new shipments.
307.
by gdiamos 3y ago
Completely agree, I'd love to see some of the innovations from HPC move over into their LLM stack. We are working on it, but it takes time. Contributions to foundational layers like ROCBlas, pytorch, slurm, Tensile, huggingface, etc wo
308.
by gdiamos 3y ago
Instinct has much better SW support today than Radeon, so you would need to send MI210s/etc . I think it's at the point where if you are comfortable with GEMM kernels, setting up SLURM, etc it is usable. But if you want to stay a
309.
by gdiamos 3y ago
See the memory size comparison (GB) in this table: https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...
310.
by gdiamos 3y ago
Yeah getting around the no public cloud thing was really annoying. We had to build our own datacenter. On the plus side, it was drastically cheaper and now we can just slot in machines. I would prefer that a tier 1 cloud made MI GPUs avail
311.
by gdiamos 3y ago
We get good utilization at scale as well. Typically 30-40% of peak at the full application level for training and inference. Perf isn't the biggest problem though, many AI chips can do this or a bit better on benchmarks, if you invest
312.
by gdiamos 3y ago
Available in orders of up to 10,000 GPUs today - no shortage More than 10x cheaper than allocating machines on a tier 1 cloud - AWS, Azure, GCP, Oracle, etc More memory - 128GB HBM per GPU - means bigger models fit for training/inferen
313.
by gdiamos 3y ago
Back in the day, using CUDA was really hard. It got better as more people built on it and it got battle tested.
314.
by gdiamos 3y ago
Relevant, we deployed Lamini on hundreds of MI200 GPUs. Lisa tweet: https://x.com/LisaSu/status/1706707561809105331?s=20 Lamini tweet: https://x.com/realSharonZhou/status/1706701693684154
315.
by gdiamos 3y ago
We built a new horizontally scalable LLM inference server that is optimized for AMD. API docs: https://lamini-ai.github.io/API/completions/ Put a load balancer in front of it and it will scale to as many GPUs as
316.
by gdiamos 3y ago
Hosted here (on our own AMD cluster): https://www.lamini.ai Docs here: https://lamini-ai.github.io We can run on any machine that can run docker in a dev mode. It won't be very fast for very big models, but you
317.
by gdiamos 3y ago
The hard part about using any AI Chips other than NVIDIA has been software. ROCm is finally at the point where it can train and deploy LLMs like Llama 2 in production. If you want to try this out, one big issue is that software support is h
318.
AI startup Lamini bets future on AMD's Instinct GPUs(theregister.com)
22 points by gdiamos 3y ago|6 comments
319.
How to use a LLM to classify text(github.com)
2 points by gdiamos 3y ago|1 comments
320.
by gdiamos 3y ago
Use it to build a tool routing agent https://github.com/lamini-ai/llm_routing_agent
321.
by gdiamos 3y ago
For example, here is a cat/dog classifier trained using prompts. Cat prompt: Cats are generally more independent and aloof. Cats are also more territorial and may be more aggressive when defending their territory. Cats are self-groomin
322.
by gdiamos 3y ago
Train a new classifier with just a prompt. ./train.sh --class "cat: CAT_DESCRIPTION" --class "dog: DOG_DESCRIPTION" ./classify.sh 'woof' {'data': 'woof', 'prediction': &
323.
Laminify: Instantly classify data with Lamini and Llama 2(github.com)
2 points by gdiamos 3y ago|2 comments
324.
by gdiamos 3y ago
The strength of my faith in GPT-4 has been getting stronger every year since 2016.
325.
Cramming an LLM platform into one container(lamini-ai.github.io)
3 points by gdiamos 3y ago|1 comments
326.
by gdiamos 3y ago
Getting this to work was an adventure. I had two experiences that shaped this. * One was building an LLM optimized training datacenter at Baidu. * The other was deploying software as a medical device in three US hospital networks. The form
327.
by gdiamos 3y ago
How can you edit or remove guardrails without fine tuning?
328.
by gdiamos 3y ago
Wasn't that right around the time of Fukushima? https://en.wikipedia.org/wiki/Fukushima_nuclear_disaster
329.
by gdiamos 3y ago
Cloud has a surprisingly high margin
330.
1.109B times faster serving of finetuned LLMs(lamini.ai)
2 points by gdiamos 3y ago|1 comments

More ›