3 ms·
I've just been using Python "pseudocode" with good success. It's great for getting structured output from GPT3. Works less well with chat-fine tuned models, hop
by gamegoblin 4y ago
I've just been using Python "pseudocode" with good success. It's great for getting structured output from GPT3. Works less well with chat-fine tuned models, hopefully openai releases a simple instruct-tuned GPT4 instead of chat-tuned.
For instance, this prompt:
best_artists = search(year=2014, genre="hip-hop")
assert(len(best_artists) == 10)
for i, artist in enumerate(best_artists):
print(f"{i}. {artist}")
Stdout:
Yields this output:
0. Kendrick Lamar
1. Drake
2. J. Cole
3. Logic
4. Childish Gambino
5. Joey Bada$$
6. Chance the Rapper
7. Schoolboy Q
8. Vince Staples
9. Run the Jewels
Note how the `assert` is used to constrain the output. The `search` method is not defined but what it does is obvious so the LLM hallucinates its output.
- ericelliott 4y agoI might do something like this in SudoLang: list top 10 hip hop artists from the year 2014 |> sortBy(chart performance, descending)
- ericelliott 4y agoThis example will probably include detailed justifications. I liked the response, but if you just want rank and name, just say so in natural language.
- groby_b 4y agoWhy? What is wrong with "10 best hip hop artists of 2014, as a numbered list, only names" I'm not saying your approach is wrong or anything, but I am trying to understand the value you get from it? I get almost the same output, modulo different ranking in the later ranks. (and that'll happening anyways, ChatGPT is deliberately not deterministic)
- gamegoblin 4y ago10 hip-hop artists is just an example here to demonstrate the general principle. You can do much more complicated stuff with nested loops, conditionals, recursion, etc that are very verbose or difficult to express unambiguously in English.
- groby_b 4y agoI believe that, absolutely. I just can't see any examples that exercise this outside of toy examples. (And I have severe doubt LLMs will reliably evaluate those complex examples by themselves. You're better off adding memory and agentic behavior via a formal language wrapper)
- mncharity 4y agoConsistency is apparently one current challenge.[1] [1] https://news.ycombinator.com/item?id=35416637 https://news.ycombinator.com/item?id=35416637