3 ms·
Except your toy model or toy version of the model will barely work to talk to you, much less write code. I've done this experiment with a much beefier set of GP
by AstralStorm 1y ago
Except your toy model or toy version of the model will barely work to talk to you, much less write code.
I've done this experiment with a much beefier set of GPUs (3080 10 GB + 3060 12 GB) allowing me to run one step up bigger model.
It's not even comparable to free tiers. I have no idea how big the machines or clusters running that are, but they must be huge.
I was very unimpressed with the local models I could run.
- blooalien 1y agoI dunno. Maybe you're expecting too much of them? They're obviously not gonna be like those massive data-center LLMs, but I've had some pretty good "brainstorming" sessions about code and documentation with Qwen and Gemma, and the latest vision-capable Qwen models do a really decent job of extracting data and text out of images (often more accurately than typical OCR engines) and even describing or captioning images, and for writing code, I've had good success with asking it to write me docstrings and type hints, and common "boilerplate" code, and even some pretty solid function requests that often come out exactly as I expected it to (as long as it's fed good "context" to work with, like some basic code-style rules, and access to library documentation and existing codebase).