5 ms·
This looks really interesting, great work! Is there somewhere in the docs that describes the essence of how LMQL works? My initial assumption was that it's a co
by killthebuddha 3y ago
This looks really interesting, great work! Is there somewhere in the docs that describes the essence of how LMQL works? My initial assumption was that it's a compiler that transforms LMQL queries into prompts, but then I realized that there also needs to be some kind of query execution engine to do prompt chaining, retries, etc. More generally, I'm trying to figure out how difficult it would be to use LMQL from Node.js.
- lbeurerkellner 3y agoYes, you are correct. LMQL has its own interpreter and runtime, that executes and translates your query programs into multiple LLM calls, required to satisfy your overall workload. It is inherently multi-part and automates several consecutive calls to the LLM, based on the query program and prompt template that you provide. Re Node.js: We are actively investigating LMQL use from other languages than Python (see also https://github.com/eth-sri/lmql/issues/1 https://github.com/eth-sri/lmql/issues/1). The current interpreter is quite closely tied to a Python environment, meaning to use it from Node.js, you will at least have to host a Python interpreter as a subprocess for now. We are actively looking into gRPC/inter-process communication though, hoping to improve on this a bit in the near future.