3 ms·
You need at minimum a stock operating system install of: - Linux 2.6.18+ (arm64 or amd64) i.e. any distro RHEL5 or newer - MacOS 15.6+ (arm64 or amd64, gpu on
by jart 3y ago
You need at minimum a stock operating system install of:
- Linux 2.6.18+ (arm64 or amd64) i.e. any distro RHEL5 or newer
- MacOS 15.6+ (arm64 or amd64, gpu only supported on arm64)
- Windows 8+ (amd64)
- FreeBSD 13+ (amd64, gpu should work in theory)
- NetBSD 9.2+ (amd64, gpu should work in theory)
- OpenBSD 7+ (amd64, no gpu support)
- AMD64 microprocessors must have SSSE3. Otherwise llamafile will print an error and refuse to run. This means, if you have an Intel CPU, it needs to be Intel Core or newer (circa 2006+), and if you have an AMD CPU, then it needs to be Bulldozer or newer (circa 2011+). If you have a newer CPU with AVX or better yet AVX2, then llamafile will utilize your chipset features to go faster. No support for AVX512+ runtime dispatching yet.
- ARM64 microprocessors must have ARMv8a+. This means everything from Apple Silicon to 64-bit Raspberry Pis will work, provided your weights fit into memory.
I've also tested GPU works on Google Cloud Platform and Nvidia Jetson, which has a somewhat different environment. Apple Metal is obviously supported too, and is basically a sure thing so long as xcode is installed.
- mercutio2 3y agoApple Security will be excited to reach out to you to find out where you got a copy of macOS 15.6 :) I'm guessing this should be 13.6?
- gary_0 3y agoThis is jart we are talking about. Perhaps, having made code Actually Portable in space, now she is doing time.
- jart 3y ago15.6 is a Darwin kernel version from 2018. It's the number `uname -a` reports. We should probably just switch to using XNU version numbers, which are in the 10000s now, so there's no confusion. I'm reasonably certain it works that far back, but I currently lack the ability to spin up old MacOS VMs for testing. Caveat emptor anyone not running MacOS on a recent version.
- anst 3y agoTime to go amd, poor old me, Intel MB Air 2018 (zsh: exec format error, Darwin Kernel Version 22.2.0, MacOS Ventura 13.1).
- deleted 3y ago[deleted]