4 ms·
More tools in the field is great! I tried a few things, and it's reasonable, but it does have some quirks that seem to repeat, like: I tried a prompt of: #
by tyingq 4y ago
More tools in the field is great! I tried a few things, and it's reasonable, but it does have some quirks that seem to repeat, like:
I tried a prompt of:
# python function that returns a random integer between min and max
And it produced:
def random_int(min, max):
return random.randint(min, max)
# define the size of the grid
n = 5
It doesn't add the needed import statement, and I'm unclear why it's "defining the size of the grid".
- circuit10 4y agoThat's because it's not following instructions like ChatGPT, it's just trying to guess that could plausibly come after what you put, like Copilot or the old GPT-3 models
- minkzilla 4y agoand imports are (almost) always at the top of your file not with this function
- vharuck 4y agoI tried the same input, except wrapping it in triple-quotes instead of commenting it. So that it would match the standard practice for module doc strings. Here's the result: """python function that returns a random integer between min and max""" return random.randint(min, max) def gen_random_float(min, max): """python function that returns a random float between min and max""" return random.uniform( So, it assumed the triple-quote was a function's doc string, despite it not being indented. It then assumes I'll want a similar function for floats (I assume it was cut off by a token limit).
- jeremyjh 4y agoIsn’t ChatGPT also just generating plausible text that could be a response to an instruction?
- whimsicalism 4y agoIt's not generating the most likely next word in the 'meta-corpora' of all possible discussions similar to the ones it has been trained on, it is trying to generate plausible text that would be scored well as a helpful assistant - and in the process has transferred knowledge acquired from its pre-training task.
- travisjungroth 4y agoYeah, at their core they’re both trying to guess/generate what comes next. Differences: Being trained towards conversations versus code. Hyperparameters set to stop differently. “Wrappers” that form the prompt.
- circuit10 4y ago"that could be a response to an instruction" is the critical part here
- amasad 4y agoLLMs generally but more so small models will keep going and generate seemingly unrelated things. On the frontend tools like Copilot and Ghostwriter do a lot of things like use stopwords or simply not show completions outside a single block. As for your prompt, it's following your prompt a little too closely and generating just the function. You can however condition it that this is the start of the program it will do the import, e.g. # python function that returns a random integer between min and max import This is in fact a suggestion from OpenAI on best practices for prompting called "leading words" https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api https://help.openai.com/en/articles/6654000-best-practices-f...
- radq 4y agoI've had the issue of generating random code after the completion with other models as well; it's due to how the models are trained. You need to stop generating when you encounter token(s) that indicate you're done - see https://huggingface.co/replit/replit-code-v1-3b#post-processing https://huggingface.co/replit/replit-code-v1-3b#post-process...
- agilob 4y agoI get such unrelated statements from copilot too, not often, but a few I remember.
- tyingq 4y agoBased on the the replies, I tried a different prompt: # python script that prints out an integer between min and max And it did better. Included the import, didn't add unrelated code, but did still put the code inside a function.