3 ms·

GPT does encode position information in its training data (an idea going back to Vaswani's 2017 paper). But indeed, it can get it wrong if it hasn't seen contra

by tlb 4y ago

GPT does encode position information in its training data (an idea going back to Vaswani's 2017 paper). But indeed, it can get it wrong if it hasn't seen contrasting examples where different order means something different. In this case, I'm not sure the mistake you're claiming is a mistake. A reasonable way to produce a funny story is to make the title character funny. Making the title character funny doesn't mean that it misunderstood the directions.

fxtentacle 4y ago
The result is a story that describes the princess as being funny, without actually saying anything funny: "Once upon a time in the medieval kingdom of Gigglesworth, there was a princess named Miyu. Princess Miyu was known throughout the kingdom for her infectious laugh and love of practical jokes. One day, the kingdom was thrown into chaos when the royal jester decided to quit his job. The king and queen were beside themselves with worry, knowing that they needed someone to fill the jester's shoes before the upcoming royal ball. Princess Miyu, always eager to help, stepped up to the plate. "I can be the jester!" she exclaimed with a giggle. The king and queen were skeptical at first, but Princess Miyu was persistent. She spent the next few days practicing her jokes and tricks, determined to make the royal ball the most entertaining event the kingdom had ever seen. Finally, the day of the ball arrived. Princess Miyu donned her jester's hat and entered the grand hall, ready to perform. She started with a classic joke, pulling a fake flower from her sleeve and presenting it to the king. But instead of a flower, a live frog jumped out, much to the king's surprise and the queen's horror. The crowd erupted in laughter, but the king and queen were not amused. Princess Miyu quickly realized that perhaps she was not cut out for the job of the royal jester after all. As she slunk out of the hall, she couldn't help but laugh at herself. "Maybe I should stick to being a princess," she thought to herself. And so, the kingdom of Gigglesworth went on without a jester, but with the memory of Princess Miyu's hilarious attempt to fill the role."

by tlb 4y ago

fxtentacle 4y ago

The result is a story that describes the princess as being funny, without actually saying anything funny: "Once upon a time in the medieval kingdom of Gigglesworth, there was a princess named Miyu. Princess Miyu was known throughout the kingdom for her infectious laugh and love of practical jokes. One day, the kingdom was thrown into chaos when the royal jester decided to quit his job. The king and queen were beside themselves with worry, knowing that they needed someone to fill the jester's shoes before the upcoming royal ball. Princess Miyu, always eager to help, stepped up to the plate. "I can be the jester!" she exclaimed with a giggle. The king and queen were skeptical at first, but Princess Miyu was persistent. She spent the next few days practicing her jokes and tricks, determined to make the royal ball the most entertaining event the kingdom had ever seen. Finally, the day of the ball arrived. Princess Miyu donned her jester's hat and entered the grand hall, ready to perform. She started with a classic joke, pulling a fake flower from her sleeve and presenting it to the king. But instead of a flower, a live frog jumped out, much to the king's surprise and the queen's horror. The crowd erupted in laughter, but the king and queen were not amused. Princess Miyu quickly realized that perhaps she was not cut out for the job of the royal jester after all. As she slunk out of the hall, she couldn't help but laugh at herself. "Maybe I should stick to being a princess," she thought to herself. And so, the kingdom of Gigglesworth went on without a jester, but with the memory of Princess Miyu's hilarious attempt to fill the role."