remilouf

8 ms·

31.
by remilouf 3y ago
Thank you for the pointer. The best part of posting on HN is the long list of related work you get in response.
32.
by remilouf 3y ago
The Finite-State Machine we walk on during the generation process does not suffer from this problem so we can still output correct JSON, if that’s what you’re asking.
33.
by remilouf 3y ago
I’m sorry that our software made you so angry. It was a side project led by two people independently from the rest of the company.
34.
by remilouf 3y ago
They’re checking regex partial matches for each possible completion, which is intensive indeed. You can look at the Figure 2 in our paper (link in original post) for a simple comparison with MS guidance which shows the difference.
35.
by remilouf 3y ago
IanCal said it all. But for alternative approaches that also use LLM (with miniKanren) you can check https://arxiv.org/abs/1809.02840
36.
by remilouf 3y ago
We can add an integration to llama.cpp, please open an issue on the repo if you’re interested!
37.
by remilouf 3y ago
Agree, working on a Colab with a "better" model as we speak.
38.
by remilouf 3y ago
We're unfortunately only human and didn't catch every single paper on the topic while writing the draft. Thanks for bringing it to our attention.
39.
by remilouf 3y ago
It can be used with any open source model (if you can get the logits), and to some extent with OpenAI's API. Here is an example with `transformers`: https://github.com/normal-computing/outlines#efficient-json-...
40.
by remilouf 3y ago
Figure 2 in our paper ( https://arxiv.org/abs/2307.09702 ) shows the difference between guidance and outlines to generate a sequence that is valid to a regex. Jsonformer uses the same technique as guidance. Extrapolate t
41.
by remilouf 3y ago
Perhaps I didn’t explain clearly enough in the original post?
42.
by remilouf 3y ago
Figure 2 in our paper ( https://arxiv.org/abs/2307.09702 ) shows the difference for a single regex.
43.
by remilouf 3y ago
Indeed, this remains an empirical question.
44.
by remilouf 3y ago
Thanks for bringing clownfish and relm to my attention! afaik other libraries loop over the entire vocabulary at every step of the generation. We on the other hand build an index at initialization by looping once over the vocabulary. Then g
45.
by remilouf 3y ago
Thank you! Hope this helps and opens many applications :)
46.
by remilouf 3y ago
Indeed. And we're able to update the mask with a dictionary lookup instead of looping over the entire vocabulary (slow!).
47.
by remilouf 3y ago
LQML (and guidance https://github.com/guidance-ai/guidance ) are much more inefficient. They loop over the entire vocabulary at each step, we only do it once at initialization.
48.
by remilouf 3y ago
Thanks! We have extended the approach to grammar-based sampling. We describe the approach in the paper linked above. The following PR is relevant: https://github.com/normal-computing/outlines/pull/178
49.
by remilouf 3y ago
We can extend our approach to grammar-based sampling, as explained in the paper linked above. Relevant PR: https://github.com/normal-computing/outlines/pull/178 Our method is much more efficient. llama.cpp lo
50.
by remilouf 3y ago
Two differences: (1) This feature only requires regex-guided generation. We have a PR for BNF sampling that is about to be merged. (2) ggml loops over the entire vocabulary (~50k tokens) at each step, which introduces a noticeable overhead,
51.
by remilouf 3y ago
That's what we noticed as well, and we were not satisfied with the `guardrails` approach of just rejecting invalid outputs. The method makes the interface robust.
52.
by remilouf 3y ago
Thanks for bringing this library to my attention! From my understanding, TypeChat proceeds by (1) generating (2) attempting validation (3) if it fails, call the LLM again to fix the output (4) etc. Our method on the other guarantees that
53.
Show HN: LLMs can generate valid JSON 100% of the time(github.com)
854 points by remilouf 3y ago|303 comments
54.
by remilouf 7y ago
I’ve had the same frustration recently and coded a working prototype in Elixir. I’d be happy to share.
55.
by remilouf 7y ago
For me culture fit is working culture fit. I’m always looking for someone who can get on board with the way we work as a team. I don’t care if we can become friends or not, if you are willing to work long hours or not (I’d rather not) as lo
56.
by remilouf 7y ago
Back in my days as an academic I would follow the same pattern every day: start working early from 7 till 10, take a long walk to the lab have a couple informal meetings with colleague, have lunch. Work there for a couple hours (my « availa
57.
by remilouf 7y ago
This. Having a tiling windows manager also helped me focusing on the task at hand somehow.
58.
by remilouf 7y ago
Having used Erlang for a side project recently: pattern matching and supervision trees.
59.
by remilouf 7y ago
I think it was intended to show you the landscape, give you enough tools and background knowledge so you can go and explore the literature by yourself. Years after it’s publication it still does a really good job at it.
60.
by remilouf 7y ago
What we call gut feeling is often a subconscious interpretation of the person’s body language, physical appearance and tone of voice. It’s always good to try to explicit why you had that feeling before basing your decision on it. I often th

More ›