4 ms·
USB device using a Raspberry Pi Zero W 512MB RAM and single-core ARMv6. It looks like a regular USB drive when plugged in. Used llamacpp but had to remove arch
by andrewfromx 2y ago
USB device using a Raspberry Pi Zero W 512MB RAM and single-core ARMv6. It looks like a regular USB drive when plugged in.
Used llamacpp but had to remove architecture-specific optimizations. Took over 12 hours on the Pi Zero's old CPU!
Successfully runs three different LLM models: Tiny Stories (15M parameters), Lamini (77M parameters), Smollm (135M parameters)
The smallest model generates tokens at ~200ms per token, while larger models take about 2.5 seconds per token. When you create a new file on the USB drive, it detects the filename and automatically generates content!