3 ms·
Under the hood, we're supporting multiple models that can be selected, but haven't optimized all the quantizations possible (the space is moving fast). The ran
by maiybe 2y ago
Under the hood, we're supporting multiple models that can be selected, but haven't optimized all the quantizations possible (the space is moving fast).
The range is 1GB - 24GB, depending on model selection, but would be amazing to push lower than that. 24GB is high end as only the NVIDIA XX90s can support those.
- jsheard 2y ago1-2GB might be workable if the model still performs adequately at that level, but anything more than that sounds very hard to justify for as long as the median Steam user and console baseline (Xbox Series S) only have 8GB of VRAM to go around.
- maiybe 2y agoDepends on the fidelity of the graphics, but I agree with you that the smaller the VRAM usage, the broader base we can support on e.g. Steam. 1GB - 2GB would be the sweet spot for all game types, which 1B parameter quantized models can hit. There is some evidence that next gen consoles will feature AMD NPUs, and I suspect there will be more available RAM. There's definitely positive tailwinds that will change the hardware landscape over time.
- deleted 2y ago[deleted]