3 ms·

Ramon here, the other cofounder of Pretzel! Quick update: Based on some early feedback, we're already working on adding support for local LLMs and Claude Sonnet

Ramon here, the other cofounder of Pretzel! Quick update: Based on some early feedback, we're already working on adding support for local LLMs and Claude Sonnet 3.5. Happy to answer any questions!

westurner 2y ago
braintrust-proxy: https://github.com/braintrustdata/braintrust-proxy https://github.com/braintrustdata/braintrust-proxy LocalAI: https://github.com/mudler/LocalAI https://github.com/mudler/LocalAI E.g promptfoo and chainforge have multi-LLM workflows. Promptfoo has a YAML configuration for prompts, providers,: https://www.promptfoo.dev/docs/configuration/guide/ https://www.promptfoo.dev/docs/configuration/guide/ What is the system prompt, and how does a system prompt also bias an analysis? /? "system prompt" https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22system%20prompt%22&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
- prasoonds 2y ago
  Thank you for the links! We'll take a look. At the moment, the system prompt is hardcoded in code. I don't think it should bias analyses because our goal is usually returning code that does something specific (for eg "calculate normalized rates for column X grouped by column Y") as opposed to something generic ("why is our churn rate going up"). So, the idea would be that the operator is responsible for asking the right questions - the AI merely acts as a facilitator to generate and understand code. Also tangentially, we do want to allow users some king of local prompt management and allow easy access, potentially via slash commands. In our testing so far, the existing prompt largely works (took us a fair bit of hit-and-trial to find something that mostly works for common data usecases) but we're hitting limitations of a fixed system prompt so these links will be useful.
marcklingen 2y ago
Big fan of LiteLLM Proxy and LiteLLM Python SDK to connect to various local models. Might be helpful here as well
- prasoonds 2y ago
  Thanks Marc! We'll check them out. So far, with our limited experience with local models, we'd simply been thinking to use Ollama so thanks for the heads up!
  - windexh8er 2y ago
    Adding some love for local model support. A lot of security use cases are demanding local models. Would love to see local support soon as this looks interesting. I agree that I think starting out with Ollama is probably hits 80% of what people would be looking for out of the box.

Ramon here, the other cofounder of Pretzel! Quick update: Based on some early feedback, we're already working on adding support for local LLMs and Claude Sonnet 3.5. Happy to answer any questions!

westurner 2y ago

braintrust-proxy: https://github.com/braintrustdata/braintrust-proxy https://github.com/braintrustdata/braintrust-proxy LocalAI: https://github.com/mudler/LocalAI https://github.com/mudler/LocalAI E.g promptfoo and chainforge have multi-LLM workflows. Promptfoo has a YAML configuration for prompts, providers,: https://www.promptfoo.dev/docs/configuration/guide/ https://www.promptfoo.dev/docs/configuration/guide/ What is the system prompt, and how does a system prompt also bias an analysis? /? "system prompt" https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22system%20prompt%22&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

prasoonds 2y ago
Thank you for the links! We'll take a look. At the moment, the system prompt is hardcoded in code. I don't think it should bias analyses because our goal is usually returning code that does something specific (for eg "calculate normalized rates for column X grouped by column Y") as opposed to something generic ("why is our churn rate going up"). So, the idea would be that the operator is responsible for asking the right questions - the AI merely acts as a facilitator to generate and understand code. Also tangentially, we do want to allow users some king of local prompt management and allow easy access, potentially via slash commands. In our testing so far, the existing prompt largely works (took us a fair bit of hit-and-trial to find something that mostly works for common data usecases) but we're hitting limitations of a fixed system prompt so these links will be useful.

marcklingen 2y ago

Big fan of LiteLLM Proxy and LiteLLM Python SDK to connect to various local models. Might be helpful here as well

prasoonds 2y ago
Thanks Marc! We'll check them out. So far, with our limited experience with local models, we'd simply been thinking to use Ollama so thanks for the heads up!
- windexh8er 2y ago
  Adding some love for local model support. A lot of security use cases are demanding local models. Would love to see local support soon as this looks interesting. I agree that I think starting out with Ollama is probably hits 80% of what people would be looking for out of the box.