4 ms·

I wonder if llm would work?

simonw 3y ago
Base https://llm.datasette.io/ https://llm.datasette.io/ should work just fine, but it's basically a thin Python wrapper around the OpenAI Python library which then makes HTTP requests to their API. The bigger challenge would be LLM with plugins. Those add all sorts of extra dependencies to help LLM run models directly - PyTorch, Transformers, llama-cpp-python etc. I'd be surprised to see "llm install llm-gpt4all" work for example, since that pulls in compiled Python wheels and I'm not sure it would know which architecture to use. pip install DOES work: $ ./python -m pip install httpx Defaulting to user installation because normal site-packages is not writeable Collecting httpx Downloading httpx-0.25.0-py3-none-any.whl (75 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.7/75.7 kB 2.7 MB/s eta 0:00:00 ... Successfully installed anyio-4.0.0 h11-0.14.0 httpcore-0.18.0 httpx-0.25.0 sniffio-1.3.0 $ ./python Python 3.11.4 (heads/pypack1:65ac8ac, Oct 16 2023, 02:35:05) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import httpx >>> httpx <module 'httpx' from '/Users/simon/.local/lib/python3.11/site-packages/httpx/__init__.py'> $ ls ~/.local/lib/python3.11/site-packages anyio anyio-4.0.0.dist-info h11 h11-0.14.0.dist-info httpcore httpcore-0.18.0.dist-info httpx httpx-0.25.0.dist-info sniffio sniffio-1.3.0.dist-info That ~/.local/lib/python3.11/ directory didn't exist until I ran the ./python -m pip install command. Unfortunately "./python -m pip install llm" didn't quite work, I got this: ./python -m pip install llm Defaulting to user installation because normal site-packages is not writeable Collecting llm Downloading llm-0.11.1-py3-none-any.whl (36 kB) Requirement already satisfied: click in /zip/Lib/site-packages (from llm) (8.1.6) Collecting openai (from llm) Downloading openai-0.28.1-py3-none-any.whl (76 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.0/77.0 kB 31.0 MB/s eta 0:00:00 Collecting click-default-group>=1.2.3 (from llm) Downloading click_default_group-1.2.4-py2.py3-none-any.whl (4.1 kB) Collecting sqlite-utils>=3.35.0 (from llm) Downloading sqlite_utils-3.35.1-py3-none-any.whl (67 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.2/67.2 kB 4.9 MB/s eta 0:00:00 Collecting sqlite-migrate>=0.1a2 (from llm) Downloading sqlite_migrate-0.1b0-py3-none-any.whl (10.0 kB) Requirement already satisfied: pydantic>=1.10.2 in /zip/Lib/site-packages (from llm) (1.10.12) Requirement already satisfied: PyYAML in /zip/Lib/site-packages (from llm) (6.0) Collecting pluggy (from llm) Downloading pluggy-1.3.0-py3-none-any.whl (18 kB) Collecting python-ulid (from llm) Downloading python_ulid-2.2.0-py3-none-any.whl (9.4 kB) Requirement already satisfied: setuptools in /zip/Lib/site-packages (from llm) (68.0.0) Requirement already satisfied: pip in /zip/Lib/site-packages (from llm) (23.1.2) Requirement already satisfied: typing-extensions>=4.2.0 in /zip/Lib/site-packages (from pydantic>=1.10.2->llm) (4.7.1) Collecting sqlite-fts4 (from sqlite-utils>=3.35.0->llm) Downloading sqlite_fts4-1.0.3-py3-none-any.whl (10.0 kB) Collecting tabulate (from sqlite-utils>=3.35.0->llm) Downloading tabulate-0.9.0-py3-none-any.whl (35 kB) Requirement already satisfied: python-dateutil in /zip/Lib/site-packages (from sqlite-utils>=3.35.0->llm) (2.8.2) Requirement already satisfied: requests>=2.20 in /zip/Lib/site-packages (from openai->llm) (2.31.0) Collecting tqdm (from openai->llm) Downloading tqdm-4.66.1-py3-none-any.whl (78 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 41.9 MB/s eta 0:00:00 Collecting aiohttp (from openai->llm) Downloading aiohttp-3.8.6-cp311-cp311-macosx_11_0_arm64.whl (343 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 343.5/343.5 kB 3.5 MB/s eta 0:00:00 Requirement already satisfied: charset-normalizer<4,>=2 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2.0.4) Requirement already satisfied: certifi>=2017.4.17 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2023.7.22) Collecting attrs>=17.3.0 (from aiohttp->openai->llm) Downloading attrs-23.1.0-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 41.9 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 (from aiohttp->openai->llm) Downloading multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl (29 kB) Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->openai->llm) Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB) Collecting yarl<2.0,>=1.0 (from aiohttp->openai->llm) Downloading yarl-1.9.2-cp311-cp311-macosx_11_0_arm64.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.3/61.3 kB 427.1 kB/s eta 0:00:00 Collecting frozenlist>=1.1.1 (from aiohttp->openai->llm) Downloading frozenlist-1.4.0-cp311-cp311-macosx_11_0_arm64.whl (46 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.7/46.7 kB 14.8 MB/s eta 0:00:00 Collecting aiosignal>=1.1.2 (from aiohttp->openai->llm) Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Requirement already satisfied: six>=1.5 in /zip/Lib/site-packages (from python-dateutil->sqlite-utils>=3.35.0->llm) (1.16.0) Installing collected packages: sqlite-fts4, tqdm, tabulate, python-ulid, pluggy, multidict, frozenlist, click-default-group, attrs, async-timeout, yarl, sqlite-utils, aiosignal, sqlite-migrate, aiohttp, openai, llm ERROR: Could not install packages due to an OSError: [Errno 2] ENOENT/2/No such file or directory: '/Users/simon/.local/bin/sqlite-utils'
- ahgamut 3y ago
  Yes, so pip works because the Python APE has OpenSSL built-in. pip install requires modifying the APE, so I end up installing pure-Python libraries as follows: mkdir Lib ./python -m pip download httpx unzip ./*.whl -d ./Lib mv python python.com # in case the next step doesn't work zip -r ./python.com Lib mv python.com python Installing CPython extensions like this is an unsolved problem, but I think there might be some interesting workarounds possible.
- deleted 3y ago
  [deleted]
AMICABoard 3y ago
Yes they do! https://github.com/jart/cosmopolitan/tree/master/third_party/ggml https://github.com/jart/cosmopolitan/tree/master/third_party... and https://github.com/trholding/llama2.c https://github.com/trholding/llama2.c

Base https://llm.datasette.io/ https://llm.datasette.io/ should work just fine, but it's basically a thin Python wrapper around the OpenAI Python library which then makes HTTP requests to their API. The bigger challenge would be LLM with plugins. Those add all sorts of extra dependencies to help LLM run models directly - PyTorch, Transformers, llama-cpp-python etc. I'd be surprised to see "llm install llm-gpt4all" work for example, since that pulls in compiled Python wheels and I'm not sure it would know which architecture to use. pip install DOES work: $ ./python -m pip install httpx Defaulting to user installation because normal site-packages is not writeable Collecting httpx Downloading httpx-0.25.0-py3-none-any.whl (75 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.7/75.7 kB 2.7 MB/s eta 0:00:00 ... Successfully installed anyio-4.0.0 h11-0.14.0 httpcore-0.18.0 httpx-0.25.0 sniffio-1.3.0 $ ./python Python 3.11.4 (heads/pypack1:65ac8ac, Oct 16 2023, 02:35:05) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import httpx >>> httpx <module 'httpx' from '/Users/simon/.local/lib/python3.11/site-packages/httpx/__init__.py'> $ ls ~/.local/lib/python3.11/site-packages anyio anyio-4.0.0.dist-info h11 h11-0.14.0.dist-info httpcore httpcore-0.18.0.dist-info httpx httpx-0.25.0.dist-info sniffio sniffio-1.3.0.dist-info That ~/.local/lib/python3.11/ directory didn't exist until I ran the ./python -m pip install command. Unfortunately "./python -m pip install llm" didn't quite work, I got this: ./python -m pip install llm Defaulting to user installation because normal site-packages is not writeable Collecting llm Downloading llm-0.11.1-py3-none-any.whl (36 kB) Requirement already satisfied: click in /zip/Lib/site-packages (from llm) (8.1.6) Collecting openai (from llm) Downloading openai-0.28.1-py3-none-any.whl (76 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.0/77.0 kB 31.0 MB/s eta 0:00:00 Collecting click-default-group>=1.2.3 (from llm) Downloading click_default_group-1.2.4-py2.py3-none-any.whl (4.1 kB) Collecting sqlite-utils>=3.35.0 (from llm) Downloading sqlite_utils-3.35.1-py3-none-any.whl (67 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.2/67.2 kB 4.9 MB/s eta 0:00:00 Collecting sqlite-migrate>=0.1a2 (from llm) Downloading sqlite_migrate-0.1b0-py3-none-any.whl (10.0 kB) Requirement already satisfied: pydantic>=1.10.2 in /zip/Lib/site-packages (from llm) (1.10.12) Requirement already satisfied: PyYAML in /zip/Lib/site-packages (from llm) (6.0) Collecting pluggy (from llm) Downloading pluggy-1.3.0-py3-none-any.whl (18 kB) Collecting python-ulid (from llm) Downloading python_ulid-2.2.0-py3-none-any.whl (9.4 kB) Requirement already satisfied: setuptools in /zip/Lib/site-packages (from llm) (68.0.0) Requirement already satisfied: pip in /zip/Lib/site-packages (from llm) (23.1.2) Requirement already satisfied: typing-extensions>=4.2.0 in /zip/Lib/site-packages (from pydantic>=1.10.2->llm) (4.7.1) Collecting sqlite-fts4 (from sqlite-utils>=3.35.0->llm) Downloading sqlite_fts4-1.0.3-py3-none-any.whl (10.0 kB) Collecting tabulate (from sqlite-utils>=3.35.0->llm) Downloading tabulate-0.9.0-py3-none-any.whl (35 kB) Requirement already satisfied: python-dateutil in /zip/Lib/site-packages (from sqlite-utils>=3.35.0->llm) (2.8.2) Requirement already satisfied: requests>=2.20 in /zip/Lib/site-packages (from openai->llm) (2.31.0) Collecting tqdm (from openai->llm) Downloading tqdm-4.66.1-py3-none-any.whl (78 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 41.9 MB/s eta 0:00:00 Collecting aiohttp (from openai->llm) Downloading aiohttp-3.8.6-cp311-cp311-macosx_11_0_arm64.whl (343 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 343.5/343.5 kB 3.5 MB/s eta 0:00:00 Requirement already satisfied: charset-normalizer<4,>=2 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2.0.4) Requirement already satisfied: certifi>=2017.4.17 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2023.7.22) Collecting attrs>=17.3.0 (from aiohttp->openai->llm) Downloading attrs-23.1.0-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 41.9 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 (from aiohttp->openai->llm) Downloading multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl (29 kB) Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->openai->llm) Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB) Collecting yarl<2.0,>=1.0 (from aiohttp->openai->llm) Downloading yarl-1.9.2-cp311-cp311-macosx_11_0_arm64.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.3/61.3 kB 427.1 kB/s eta 0:00:00 Collecting frozenlist>=1.1.1 (from aiohttp->openai->llm) Downloading frozenlist-1.4.0-cp311-cp311-macosx_11_0_arm64.whl (46 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.7/46.7 kB 14.8 MB/s eta 0:00:00 Collecting aiosignal>=1.1.2 (from aiohttp->openai->llm) Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Requirement already satisfied: six>=1.5 in /zip/Lib/site-packages (from python-dateutil->sqlite-utils>=3.35.0->llm) (1.16.0) Installing collected packages: sqlite-fts4, tqdm, tabulate, python-ulid, pluggy, multidict, frozenlist, click-default-group, attrs, async-timeout, yarl, sqlite-utils, aiosignal, sqlite-migrate, aiohttp, openai, llm ERROR: Could not install packages due to an OSError: [Errno 2] ENOENT/2/No such file or directory: '/Users/simon/.local/bin/sqlite-utils'

ahgamut 3y ago
Yes, so pip works because the Python APE has OpenSSL built-in. pip install requires modifying the APE, so I end up installing pure-Python libraries as follows: mkdir Lib ./python -m pip download httpx unzip ./*.whl -d ./Lib mv python python.com # in case the next step doesn't work zip -r ./python.com Lib mv python.com python Installing CPython extensions like this is an unsolved problem, but I think there might be some interesting workarounds possible.
deleted 3y ago
[deleted]

AMICABoard 3y ago

Yes they do! https://github.com/jart/cosmopolitan/tree/master/third_party/ggml https://github.com/jart/cosmopolitan/tree/master/third_party... and https://github.com/trholding/llama2.c https://github.com/trholding/llama2.c