2 ms·
With ggml the mmap part is the default. It isn't a panacea though [0]. Note that most runtimes (like MLX, ONNX, TensorFlow, JAX/XLA etc) will employ a number of
by ignoramous 2y ago
With ggml the mmap part is the default. It isn't a panacea though [0]. Note that most runtimes (like MLX, ONNX, TensorFlow, JAX/XLA etc) will employ a number of techniques for efficient inference and mmap is just one part of it.
[0] https://news.ycombinator.com/item?id=35455930 https://news.ycombinator.com/item?id=35455930