3 ms·
WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build,
by swalsh 2mo ago
WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.
- venusenvy47 2mo agoFor your setup, do you have both 3090's in parallel for the inference of the model?
- swalsh 2mo agoYes they run in parallel via LMStudio (250k context)
- XCSme 2mo agoWhy slow? I see ~50tps on a single 3090
- apitman 2mo agoYeah make sure you're using MTP and potentially tensor parallelism.
- swalsh 2mo agoyeah, i consider that slow.
- XCSme 2mo agoOh, ok, that's like the average tps for most AI providers
- app13 2mo agoWhats your setup? I have a single 3090 and am struggling to get it purring
- XCSme 2mo ago5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama. I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com
- shdh 2mo agoHow did you undervolt the 3090?
- XCSme 2mo agoMsi afterburner, you go around 900mv curve editor, raise it up to normal clock frequency, and it uses like 260w instead 300w for same performance. There's some YouTube guides for it. I also undervolted my new 5070ti, same tdp, around 260w instead of 300w and like 8% better performance.
- polyterative 1mo agoI remember the days of crypto mining memory intensive tasks do not suffer from TDP reduction as much as game rendering.It's so counterintuitive.I remember getting 96% performance at less than half of TDP with my old AMD card.