4 ms·
To use with llama.cpp on CPU and 8GB RAM git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && cmake -B build && cmake --build build python3
by diimdeep 3y ago
To use with llama.cpp on CPU and 8GB RAM
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && cmake -B build && cmake --build build
python3 -m pip install -r requirements.txt
cd models && git clone https://huggingface.co/openlm-research/open_llama_7b_preview_200bt/ && cd -
python3 convert-pth-to-ggml.py models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights 1
./build/bin/quantize models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights/ggml-model-f16.bin models/open_llama_7b_preview_200bt_q5_0.ggml q5_0
./build/bin/main -m models/open_llama_7b_preview_200bt_q5_0.ggml --ignore-eos -n 1280 -p "Building a website can be done in 10 simple steps:" --mlock
- gigel82 3y agoYou the real MVP! Though I'm getting this error on an Intel macbook (Monterey); it works fine on a Windows11 box: python3 convert-pth-to-ggml.py models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights 1 Loading model file models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights/pytorch_model-00001-of-00002.bin Traceback (most recent call last): File "/l/llama.cpp/convert-pth-to-ggml.py", line 11, in <module> convert.main(['--outtype', 'f16' if args.ftype == 1 else 'f32', '--', args.dir_model]) File "/l/llama.cpp/convert.py", line 1129, in main model_plus = load_some_model(args.model) File "/l/llama.cpp/convert.py", line 1055, in load_some_model models_plus.append(lazy_load_file(path)) File "/l/llama.cpp/convert.py", line 857, in lazy_load_file raise ValueError(f"unknown format: {path}") ValueError: unknown format: models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights/pytorch_model-00001-of-00002.bin
- kdtsh 3y agoI get the same error on an M series MacBook (Ventura). However from the repo README.md it looks like make should work instead of cmake, I’ll give that a try.
- sebastianhoitz 3y agoI had the same issue and then noticed that I need git lfs - otherwise just cloning the repo will not download the weights.
- greatpatton 3y agoAfter getting the model with git lfs I get: Loading model file models/open_llama_7b_preview_200bt/open_llama_7b_preview_200bt_transformers_weights/pytorch_model-00001-of-00002.bin Traceback (most recent call last): File "convert-pth-to-ggml.py", line 11, in <module> convert.main(['--outtype', 'f16' if args.ftype == 1 else 'f32', '--', args.dir_model]) File "/Volumes/mac/Dev/llama.cpp/convert.py", line 1145, in main model_plus = load_some_model(args.model) File "/Volumes/mac/Dev/llama.cpp/convert.py", line 1071, in load_some_model models_plus.append(lazy_load_file(path)) File "/Volumes/mac/Dev/llama.cpp/convert.py", line 865, in lazy_load_file return lazy_load_torch_file(fp, path) File "/Volumes/mac/Dev/llama.cpp/convert.py", line 737, in lazy_load_torch_file model = unpickler.load() TypeError: 'staticmethod' object is not callable
- mportela 3y agoThanks for the tip! After running `brew install git-lfs && git lfs install` on my Macbook, I was able to run the model.