6 ms·

Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?

huseyinkeles 4y ago
I'm getting bunch of errors on an Intel mac, starts with the following, which suggests that it's about the CPU architecture, but maybe I'm wrong. sysctl: unknown oid 'hw.optional.arm64'
- cuttothechase 4y ago
  I am getting the same error on an Intel Mac.
- strikingloo 4y ago
  same problem, haven't found a solution. Please let me know if you do
  - strikingloo 4y ago
    Okay for what is worth, I was getting that as a warning but the ./chat executable works anyway. So give it a try if it is being compiled!
antimatter15 4y ago
It inherits from ARM NEON optimizations in llama.cpp but I believe it should also work on x86.
- gpderetta 4y ago
  it built in a couple of seconds with no issues on my x86 machine. Downloading weights now.
- gorbypark 4y ago
  Someone contributed AVX2 support a few days after the initial release, so llama.cpp (and the alpaca.cpp fork) should work on most x86 CPUs as well now.
- inaka10100101 4y ago
  Can we run it in colab. It isn't working for me.
mil22 4y ago
Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 system_info: n_threads = 4 / 32 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000 == Running in chat mode. == - Press Ctrl+C to interject at any time. - Press Return to return control to LLaMa. - If you want to submit another line, end your input in '\'. > What is an alpaca? An Alpaca (Vicugna pacos) is one of two species in the genus Vicuña, which are both domesticated animals from South America and Central Asia that have been bred for their fine wool. >
- deleted 4y ago
  [deleted]
- windstar81 4y ago
  [dead]
- windstar81 4y ago
  [dead]
- windstar81 4y ago
  [dead]
- windstar81 4y ago
  [dead]
lxe 4y ago
It does run on either with decent speed. Probably can even do it on a rasberry pi 4.
unshavedyak 4y ago
Works fine on my x86. Running Linux (NixOS). Ryzen 5950x.

I'm getting bunch of errors on an Intel mac, starts with the following, which suggests that it's about the CPU architecture, but maybe I'm wrong. sysctl: unknown oid 'hw.optional.arm64'

cuttothechase 4y ago
I am getting the same error on an Intel Mac.
strikingloo 4y ago
same problem, haven't found a solution. Please let me know if you do
- strikingloo 4y ago
  Okay for what is worth, I was getting that as a warning but the ./chat executable works anyway. So give it a try if it is being compiled!

antimatter15 4y ago

It inherits from ARM NEON optimizations in llama.cpp but I believe it should also work on x86.

gpderetta 4y ago
it built in a couple of seconds with no issues on my x86 machine. Downloading weights now.
gorbypark 4y ago
Someone contributed AVX2 support a few days after the initial release, so llama.cpp (and the alpaca.cpp fork) should work on most x86 CPUs as well now.
inaka10100101 4y ago
Can we run it in colab. It isn't working for me.

mil22 4y ago

Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 system_info: n_threads = 4 / 32 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000 == Running in chat mode. == - Press Ctrl+C to interject at any time. - Press Return to return control to LLaMa. - If you want to submit another line, end your input in '\'. > What is an alpaca? An Alpaca (Vicugna pacos) is one of two species in the genus Vicuña, which are both domesticated animals from South America and Central Asia that have been bred for their fine wool. >

deleted 4y ago
[deleted]
windstar81 4y ago
[dead]
windstar81 4y ago
[dead]
windstar81 4y ago
[dead]
windstar81 4y ago
[dead]

lxe 4y ago

It does run on either with decent speed. Probably can even do it on a rasberry pi 4.

unshavedyak 4y ago

Works fine on my x86. Running Linux (NixOS). Ryzen 5950x.