6 ms·

100 prompts doing what? Something like more selective, focused extraction of structured fields?

by pizza 2y ago

keeganpoppen 2y ago
a combination of lots of things, with the general theme being focused prompts good at individual, specific subtasks circuits… off the top of my head: - that try to extract the factual / knowledge content and try to update the rest of the system (e.g. if the user chats you to not send notifications after 9pm, ideally you’d like the whole system to reflect that. if they say they like the color gold, you’d like the recommendation system to know that.) - detect the emotional valence of the user’s chat and raise an alert if they seem, say, angry - speculative “given this new information, go over old outputs and see if any of our assumptions were wrong or apt and adjust accordingly” - learning/feedback systems that run evals after every k runs to update and optimize the prompt - systems where there is a large state space but any particular user has a very sparse representation (pick the top k adjectives for this user out of a list of 100, where each adjective is evaluated in its own prompt) - llm circuits with detailed QA rules (and/or many rounds of generation and self-reflection to ensure generation/result quality) - speculative execution in order to trade increased cost / computation for lower latency. (cf. graph of thoughts prompting) - out of band knowledge generation, prompt generation, etc. - alerting / backstopping / monitoring systems - running multiple independent systems in parallel and then picking the best one or even merge the best results across all of them. the more, smaller prompts, the easier to do eval and testing, as well as making the system more parallelizable. also, you get stronger, deeper signals that communicate a deeper domain understanding to the user s.t. they think you know what you’re doing. but the point is every bit as much that you are embodying your own human cognition within the llm— the reason to do that in the first place is because it is virtually infinitely scalable when it makes it out of your brain and onto/into silicon. even if each marginal prompt you add only has a .1% chance of “hitting”, you can just trigger 1000 prompts and voila: one more eureka moment for your system. sure, there’s diminishing returns in the same way that the CIA wants 100 Iraq analysts but not 10000. but unlike the CIA, you dont need to pag salaries, healthcare, managers, etc. it all scales basically linearly. and, besides, is extremely cheap as long as your prompting is even halfway decent.
- pizza 2y ago
  Interesting

100 prompts doing what? Something like more selective, focused extraction of structured fields?

by pizza 2y ago

100 prompts doing what? Something like more selective, focused extraction of structured fields?

a combination of lots of things, with the general theme being focused prompts good at individual, specific subtasks circuits… off the top of my head: - that try to extract the factual / knowledge content and try to update the rest of the system (e.g. if the user chats you to not send notifications after 9pm, ideally you’d like the whole system to reflect that. if they say they like the color gold, you’d like the recommendation system to know that.) - detect the emotional valence of the user’s chat and raise an alert if they seem, say, angry - speculative “given this new information, go over old outputs and see if any of our assumptions were wrong or apt and adjust accordingly” - learning/feedback systems that run evals after every k runs to update and optimize the prompt - systems where there is a large state space but any particular user has a very sparse representation (pick the top k adjectives for this user out of a list of 100, where each adjective is evaluated in its own prompt) - llm circuits with detailed QA rules (and/or many rounds of generation and self-reflection to ensure generation/result quality) - speculative execution in order to trade increased cost / computation for lower latency. (cf. graph of thoughts prompting) - out of band knowledge generation, prompt generation, etc. - alerting / backstopping / monitoring systems - running multiple independent systems in parallel and then picking the best one or even merge the best results across all of them. the more, smaller prompts, the easier to do eval and testing, as well as making the system more parallelizable. also, you get stronger, deeper signals that communicate a deeper domain understanding to the user s.t. they think you know what you’re doing. but the point is every bit as much that you are embodying your own human cognition within the llm— the reason to do that in the first place is because it is virtually infinitely scalable when it makes it out of your brain and onto/into silicon. even if each marginal prompt you add only has a .1% chance of “hitting”, you can just trigger 1000 prompts and voila: one more eureka moment for your system. sure, there’s diminishing returns in the same way that the CIA wants 100 Iraq analysts but not 10000. but unlike the CIA, you dont need to pag salaries, healthcare, managers, etc. it all scales basically linearly. and, besides, is extremely cheap as long as your prompting is even halfway decent.

pizza 2y ago
Interesting