4 ms·
Yes, you are correct. LMQL has its own interpreter and runtime, that executes and translates your query programs into multiple LLM calls, required to satisfy yo
by lbeurerkellner 3y ago
Yes, you are correct. LMQL has its own interpreter and runtime, that executes and translates your query programs into multiple LLM calls, required to satisfy your overall workload. It is inherently multi-part and automates several consecutive calls to the LLM, based on the query program and prompt template that you provide.
Re Node.js: We are actively investigating LMQL use from other languages than Python (see also https://github.com/eth-sri/lmql/issues/1 https://github.com/eth-sri/lmql/issues/1). The current interpreter is quite closely tied to a Python environment, meaning to use it from Node.js, you will at least have to host a Python interpreter as a subprocess for now. We are actively looking into gRPC/inter-process communication though, hoping to improve on this a bit in the near future.