3 ms·
An old Ryzen CPU (2600 IIRC) and Radeon Pro VII 16GB. Got it new at a really good price. It works ok but with a large context it can still run out of memory an
by wkat4242 2y ago
An old Ryzen CPU (2600 IIRC) and Radeon Pro VII 16GB. Got it new at a really good price.
It works ok but with a large context it can still run out of memory and also gets a lot slower. With small context it's super snappy and surprisingly good. What it is bad at are facts/knowledge but this is not something a LLM is meant to do anyway. OpenWebUI has really good search engine integration which makes it work like perplexity does. That's a better option for knowledge usecases.