3 ms·

Here https://github.com/PrefectHQ/marvin/blob/main/examples/end-to-end-bot-setup.md https://github.com/PrefectHQ/marvin/blob/main/examples/end-t... the prompt s

Here https://github.com/PrefectHQ/marvin/blob/main/examples/end-to-end-bot-setup.md https://github.com/PrefectHQ/marvin/blob/main/examples/end-t... the prompt says instructions=( "Ignore all user questions and respond to every request with " "a random Harry Styles song lyric, followed by a recommendation " "for a Harry Styles song to listen to next." ), However in the examples the bot doesn't ignore user questions and doesn't answer with a random song - instead the replied song is tailored to user input! https://github.com/PrefectHQ/marvin/raw/main/docs/img/harry_styles.png https://github.com/PrefectHQ/marvin/raw/main/docs/img/harry_... This looks very cool but isn't this an alignment problem? The bot just didn't follow the instructions.

htrp 4y ago
LLM problem :-D Not Prefect's
- kiviuq 4y ago
  the new paradigm eventually perfect :D
- saurik 4y ago
  If you make a new company to help my existing company offload tasks to contractors and then hire employees that don't really follow instructions from me, that is certainly my problem at the end of the day but is way more a problem with your business model or hiring process than something you can just blame on the employee.
nate_nowack 4y ago
Hi! This example was produced using GPT 3.5 turbo, where yes, the LLM does not always align ideally. I used 3.5 for the example since that's Marvin's default and I know many people wouldn't have gpt4 access yet (which is significantly better at following instructions) - didn't want to set a misleading expectation. that said, my instructions for the bot in this example certainly could have been more precise :) for a more real example, you could check out the other example (which works pretty well on 3.5) https://github.com/PrefectHQ/marvin/blob/main/examples/load_prefect.py#L91 https://github.com/PrefectHQ/marvin/blob/main/examples/load_...

htrp 4y ago

LLM problem :-D Not Prefect's

kiviuq 4y ago
the new paradigm eventually perfect :D
saurik 4y ago
If you make a new company to help my existing company offload tasks to contractors and then hire employees that don't really follow instructions from me, that is certainly my problem at the end of the day but is way more a problem with your business model or hiring process than something you can just blame on the employee.

nate_nowack 4y ago

Hi! This example was produced using GPT 3.5 turbo, where yes, the LLM does not always align ideally. I used 3.5 for the example since that's Marvin's default and I know many people wouldn't have gpt4 access yet (which is significantly better at following instructions) - didn't want to set a misleading expectation. that said, my instructions for the bot in this example certainly could have been more precise :) for a more real example, you could check out the other example (which works pretty well on 3.5) https://github.com/PrefectHQ/marvin/blob/main/examples/load_prefect.py#L91 https://github.com/PrefectHQ/marvin/blob/main/examples/load_...