4 ms·

Maybe this has long been obvious to everyone but me, but having cloned a few repositories of ChatGPT API web UIs, the API itself really is dead simple: send you

Maybe this has long been obvious to everyone but me, but having cloned a few repositories of ChatGPT API web UIs, the API itself really is dead simple: send your prompt and then with every subsequent include the entire conversation history. The streaming of the response is the most novel aspect of the actual API call. The first one I looked at seeded the conversation with "Response format is ALWAYS markdown, especially for code." -- but did not display this in the UI. Because it was rendering markdown in the client, it was coercing the model. I don't know why this was such an epiphany for me (and its an admittedly tiny piece in the grand scheme) but the idea that you can "influence" the model before you allow the user to interface with it (all while not even disclosing the initial prompt) really opened my eyes to how versatile this all really is.

30minAdayHN 4y ago
> Maybe this has long been obvious to everyone but me, but having cloned a few repositories of ChatGPT API web UIs, the API itself really is dead simple: send your prompt and then with every subsequent include the entire conversation history. The streaming of the response is the most novel aspect of the actual API call. It pretty soon gets complex if you are looking to update the prompt (which would require a code change or re-deployment, understand how the changes to the prompt affect the outcome, caching etc. And that complexity increases with number of prompts you are managing. In addition, another thing we observed is that in large scale projects, people who add these prompts would be some sort of subject experts that are different from people who manage the code.
kristjansson 4y ago
Moreover, that's exactly what all of the products (Bard, ChatGPT, Bing, ...) are doing to steer their models toward certain behavior. And, minutia of that initial prompt (famously, appending "let's think step-by-step") can have huge imapcts on the performance of the LLM on concrete tasks.

30minAdayHN 4y ago

> Maybe this has long been obvious to everyone but me, but having cloned a few repositories of ChatGPT API web UIs, the API itself really is dead simple: send your prompt and then with every subsequent include the entire conversation history. The streaming of the response is the most novel aspect of the actual API call. It pretty soon gets complex if you are looking to update the prompt (which would require a code change or re-deployment, understand how the changes to the prompt affect the outcome, caching etc. And that complexity increases with number of prompts you are managing. In addition, another thing we observed is that in large scale projects, people who add these prompts would be some sort of subject experts that are different from people who manage the code.

kristjansson 4y ago

Moreover, that's exactly what all of the products (Bard, ChatGPT, Bing, ...) are doing to steer their models toward certain behavior. And, minutia of that initial prompt (famously, appending "let's think step-by-step") can have huge imapcts on the performance of the LLM on concrete tasks.