5 ms·
Bringing Up DeepSeek-V4-Flash on AMD MI300X
- benlm 4mo agoNice work! Would DeepSeek V4 Pro on 8xMI300X work with these patches?
- mezark 4mo agowe think so - but haven't tested it ourselves
- kkm 4mo agoAlso the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword https://github.com/doublewordai/vllm-amd-blog-doubleword
- mezark 4mo agoWe at doubleword are bullish for AMD for low-interactivity inference - it does just take a bigger lift on the software side...
- brcmthrowaway 4mo agoAre you long AMD?
- latchkey 4mo agoInteresting that you ask that as AMD hits another ATH.
- brcmthrowaway 4mo agoThen you are definitely long on AMD.
- latchkey 4mo agoMore accurately... I'm long on a viable alternative to the current monopoly. We have two OS's for phones (android and ios), there is no reason why we shouldn't have the same for all AI hardware and software. The only one even close, is AMD.
- boxking 4mo agohello,sir, I want bulk order Asrock BC-250, is it still available ?
- keynha 4mo ago[dead]
- maCDzP 4mo agoI train on AMD MI250X and managed to get Gemma 4 31B to work - but it took a lot of work on the software side.
- kkm 4mo agoThis is very interesting, planning to write about it?
- latchkey 4mo agoNice work and thanks for being a customer. (CEO Hot Aisle)
- erichocean 4mo agoI wish you guys could partner with Modular to get Mojo inference working on your hardware, e.g. https://www.modular.com/models/deepseek-v4-pro https://www.modular.com/models/deepseek-v4-pro
- latchkey 4mo agoNot sure I understand. If they support MI300x, their self-hosted will run on our hardware.
- erichocean 4mo agoIf it was that easy, I wouldn't have commented. It's not, which is why it would be nice if they did the actual work (on your hardware). I would 100% pay $16/hr to run a self-hosted instance, but I won't spend thousands of dollars to (maybe) get it working (my time + the hardware).
- latchkey 4mo agoOk, sure. Valid. Have you asked them to support V4? https://docs.modular.com/max/models/ https://docs.modular.com/max/models/ I agree with you though, serving up inference is secret sauce for a lot of teams and not everyone publishes how to do it because of the costs involved in doing so. They need an ROI.
- jing09928 4mo ago[flagged]
- edg5000 4mo agoChecked out this company about a year ago and they only offered small models. Now I see they have GLM-fp8/Kimi and DeepSeek V4 Pro. Since workloads are predominantly cached input, I'm surprised to see no separate price for cached input vs uncached. I hope the prices will drop significantly; with these prices you'll end up with thousands in monthly costs quickly. Hopefully more hardware companies will be on the market in the coming years. If the Chinese eventually start competing with the current memory makers, maybe that will help.
- zftnb666 4mo ago[flagged]
- mezark 4mo agoHi! Co-founder of Doubleword here - we've hugely increased the number of models that we offer (partly thanks to work that we've done on hotswapping https://blog.doubleword.ai/fast-sglang-starts https://blog.doubleword.ai/fast-sglang-starts. We're kind of known for our low prices - our prices (our main usage is for our high throughput API - the async tier) is significantly below average openrouter prices - but cached prices is coming soon which will lower them even more :)
- edg5000 4mo agoWhat kind of workloads are you primarily seeing from users? I´d guess coding harness-type stuff where you have repeated calls with lots of cache hits. Or is it more like bulk OCR or invoice processing?
- alfiedotwtf 4mo agoIt’s just weird Deepseek released a model that was not compatible with any of the usual engines. Without derez’s new project just to support DSv4, how long until it’s actually viable in llama :(