3 ms·
Tangentially related but a simple way to bust a chatbot is to ask “What is larger, the Eiffel Tower or a shoe box?”
by shiftpgdn 4y ago
Tangentially related but a simple way to bust a chatbot is to ask “What is larger, the Eiffel Tower or a shoe box?”
- elicash 4y agoHere's what Meta's blenderbot replied with: "The Tokyo tower is taller than the eiffel tower. Interesting facts like that interest me. Do you know about it?"
- deleted 4y ago[deleted]
- TillE 4y agoI'm not surprised that it responded with a random unrelated fact, but it is funny that the second sentence is incredibly awkward, and the last one isn't really coherent English. Just a total AI meltdown from one simple question.
- partdavid 4y agoPeople need to know about the CAN EAT MORE. https://www.youtube.com/watch?v=CIoBSYpgYRw https://www.youtube.com/watch?v=CIoBSYpgYRw For me, I call these Eliza-isms, since it reminds me of its simple formulas like "Can you tell me more about ___" that people got so much mileage out of.
- hotpotamus 4y agoI remember someone posting a chat thread from one of the more advanced AIs within the last few years wherein they asked it who the president of the US is, and it was not able to answer. Interestingly, this is a question my father would ask patients as a paramedic who was trying to assess people's consciousness. Another would be, "what day of the week is it?". I'd say that these technologies are just like magic - they can seem to do things that defy your expectations, but oftentimes they fall apart when looked at from a different angle.
- unsupp0rted 4y agoI'm not sure there's a basic question we can ask that a lot of human users wouldn't fail. President of the country? Too difficult.
- bombcar 4y agoThe point isn't to check if they actually know - it's to gauge the response. If they say "I don't know" that may be a valid answer, but if they say "George Bush" then something is seriously wrong.
- notahacker 4y agoAlso, if a human has to be told a basic fact they'll generally provide an indication of embarrassment or an excuse or "why are you asking me these questions", not try to continue the conversation with interesting facts...
- bnt 4y agoDepending where the bot is and what time of day it is, it might tell you the wrong day of the week.
- robertlagrant 4y ago"I was trained on a Thursday... damn."
- PeterisP 4y agoFor current mainstream text generation models it doesn't really depend on where the bot is and what time of the day it is, that's kind of the whole point - their text generation process simply doesn't use the current time as a possible input factor, these models would provide the exact same result (or random picks from the exact same distribution of potential results) no matter when and where you run them. They would be expected to answer with something matching the day/time distribution that was represented in the training data they used; like the answer to various prompts of the "current president" question is dominated by Trump, Obama and a bit of Bush and Clinton, simply because those are the presidents in the training data and the more recent events simply aren't there yet - like the many models who have no idea how to interpret the word 'Covid' simply because they have been trained on pre-2020 data even if the model was built and released later.
- BitwiseFool 4y agoI think you're on to something. The modern day chat-bot/answer engines seem very susceptible towards trying to answer fact-based, yet obviously incorrect questions. They seem unable to parse the entire question and instead focus on the most generic terms. For instance, the "What year did Neil Armstrong land on Mars?" example that shows up on HN from time to time.