3 ms·
Winner of the SF Mistral AI Hackathon: Automated Test Driven Prompting
- carlcortright 3y ago“Learning” by prompting, calculating the loss against evals, and updating the prompt
- deleted 3y ago[deleted]
- paradite 3y agoYeah I've been thinking about this lately. LLMs come and go. Prompt engineering techniques come and go. But eval / labelled dataset is always useful once you built it.
- nestorD 3y agoIn my experience, architecture stays. A nice modular codebase (where you can switch-out the various models and have control on the minutia of the pipeline from data to model) brings a lot of value and can fairly easily be kept up-to-date with best practices.
- deleted 3y ago[deleted]
- bugglebeetle 3y agoIsn’t this just a very naive implementation of what DsPY does? https://github.com/stanfordnlp/dspy https://github.com/stanfordnlp/dspy I don’t understand what is exceptional here.
- carlcortright 3y ago(author) It's very similar. I think the extension of this is that we could build a frontend gui for something like this for prompt management. Love what they're doing
- carlcortright 3y agoPost Author: getting a lot of requests so scaling the backend. Standby.
- quadrature 3y agoYou should add a very short description of what this is and how it works. I kind of get that it is a way to tune/generate prompts for specific input output test cases, But its not clear exactly what it does.
- imranq 3y agoShouldn't this be LoRA training?
- hitchstory 3y agoI built something like this too: https://github.com/hitchdev/hitchstory/blob/master/examples%2Fllm%2FREADME.md https://github.com/hitchdev/hitchstory/blob/master/examples%...
- toisanji 3y agocan you please add more info on the page to show why it is important and how its helpful
- carlcortright 3y agoYeah I'll do that now!
- imdsm 3y agoIt'd be great to get a little write up of the technique
- carlcortright 3y agoAdded it!
- ai_what 3y agoSorry to ask, but would you mind expanding a little more? For example, when you state "our system iteratively rewrites the prompt to better fit the desired behavior", what is the use case for this? Is it to make personal datasets? Thank you.
- k__ 3y agoJust yesterday, I wrote an article about FT and learned about services like Entry Point AI. Seems like an awesome idea. I'm curious how long it will take to get a model on a reasonable level. Phind is pretty good and also the fastest model I used recently, so I'd assume it's quite small, no?
- deleted 3y ago[deleted]