3 ms·

What are the benefits over https://github.com/microsoft/guidance/ https://github.com/microsoft/guidance/ ?

sandkoan 3y ago
We enable conforming to arbitrary context free grammars in addition to regex patterns, and have a bunch of speed optimizations, as well. Though it may not seem too fast right now on account of the hundreds of simultaneous requests we're getting :)
- swyx 3y ago
  100s of RPS? did you have a successful launch elsewhere or something? bc this repo currently only has 21 stars, which taking normal correlation into account does not imply that level of traffic
  - sandkoan 3y ago
    We have folks playing around with it mostly through the playground / raw HTTP endpoints as opposed to the Python API. And we've got some batch jobs running, which adds further traffic.
behnamoh 3y ago
guidance is a dead project. it worked well as a hobby side project by MS researchers, but it clearly isn't a long-term solution as new LLMs are introduced.
- amkkma 3y ago
  What about LMQL?
  - darkteflon 3y ago
    I hadn't heard of LMQL before - have you tried building with it and, if so, would you be willing to share your experience?
- darkteflon 3y ago
  Would you be able and willing to provide a bit more supporting evidence for this statement? We’ve been considering trialing guidance for a few weeks but won’t bother if it’s going nowhere.
  - nestorD 3y ago
    I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201 https://github.com/microsoft/guidance/discussions/201
    - darkteflon 3y ago
      Okay interesting, thanks. There does seem to be some recent activity on the `pythonic` branch but yeah, it does look like the open issues have been going ignored for a while now.
  - IanCal 3y ago
    If it helps I tried using it and the basic examples just straight up didn't work at all and regularly got broken in different ways. Even if it's going somewhere it was unusable for a serious project. I moved to custom code and left it on the "watch this project" list for the future.
    - deleted 3y ago
      [deleted]
    - darkteflon 3y ago
      It does help, thank you. We’re going to put it on pause for now, too.

We enable conforming to arbitrary context free grammars in addition to regex patterns, and have a bunch of speed optimizations, as well. Though it may not seem too fast right now on account of the hundreds of simultaneous requests we're getting :)

swyx 3y ago
100s of RPS? did you have a successful launch elsewhere or something? bc this repo currently only has 21 stars, which taking normal correlation into account does not imply that level of traffic
- sandkoan 3y ago
  We have folks playing around with it mostly through the playground / raw HTTP endpoints as opposed to the Python API. And we've got some batch jobs running, which adds further traffic.

behnamoh 3y ago

guidance is a dead project. it worked well as a hobby side project by MS researchers, but it clearly isn't a long-term solution as new LLMs are introduced.

amkkma 3y ago
What about LMQL?
- darkteflon 3y ago
  I hadn't heard of LMQL before - have you tried building with it and, if so, would you be willing to share your experience?
darkteflon 3y ago
Would you be able and willing to provide a bit more supporting evidence for this statement? We’ve been considering trialing guidance for a few weeks but won’t bother if it’s going nowhere.
- nestorD 3y ago
  I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201 https://github.com/microsoft/guidance/discussions/201
  - darkteflon 3y ago
    Okay interesting, thanks. There does seem to be some recent activity on the `pythonic` branch but yeah, it does look like the open issues have been going ignored for a while now.
- IanCal 3y ago
  If it helps I tried using it and the basic examples just straight up didn't work at all and regularly got broken in different ways. Even if it's going somewhere it was unusable for a serious project. I moved to custom code and left it on the "watch this project" list for the future.
  - deleted 3y ago
    [deleted]
  - darkteflon 3y ago
    It does help, thank you. We’re going to put it on pause for now, too.