31 ms·

Chatbots: Still dumb after all these years

• Veuxdo 5y ago
  Is ML still a popular subject in CS? I remember about 4 years ago going to a recruiting event at a major engineering school and seemingly every student was specializing in ML.
• andrewla 5y ago
  The ending is the relevant bit: > As I have said many times, the real danger today is not that computers are smarter than us, but that we think computers are smarter than us and consequently trust them to make important decisions they should not be trusted to make.
  • JetAlone 5y ago
    I think the biggest risk vector taking us along that conga line is poor mental health. Vulnerable people feel weak and helpless enough to actually believe they ought to depend on on a system to take the work/responsibility out of their hands.
• cpill 5y ago
  yeah yeah, GPT3 is not a chat bot and not intelligent. WE KNOW. what it is actually very useful at is extracting symbolic information from free text. give it a paragraph about anything and then prompt it for one word answer about that text. this it does well and _nothing_ else can do this. I see large language models as interface to natural language not AGI
• anotheryou 5y ago
  Interestingly there is a big divide between chit-chat bots and those having purpose. The chit-chat bots at best have "google knowledge", the voice assistants only take small commands and the customer service automation bots are really stiff. Anyways it's the customer service automation bots than already can do a lot of good. If you manage to automate away the easy portions of chats that's already a big win. Further bots can do many actions really quickly so you have no wait time if it can handle your conversation. I think building a good co-pilot scenario where you hand conversations back and forth to a human at the right moments is the near future here.
• tehsauce 5y ago
  The ship of “intelligence can’t be made with statistics” is sinking, but i think people will keep saying it’s impossible until the bitter end.
• kidproquo 5y ago
  This is more problematic with the USCIS immigration helpline bot. You don't have the option to yake your business elsewhere. Trying to get the phone bot to let me talk to a human at USCIS is like talking to the Avrona Kern AI from Children of Time by Adrian Tchaikovsky. Me: The typographic error submission site throws an error when I try to use it. Avrona Kern: We have a website to submit typographic errors. Avrona Kern: Would you like to receive the link via SMS or email? Me: That site doesn't work! Agent. Avrona Kern: Would you like to receive the link by SMS or email? Me: Agent Avrona Kern: I can help you. Me: Agent Avrona Kern: If you keep asking for agent I will end the call. Me: Typographic site throws error Avrona Kern: We have a website... Me: FML Googling revealed that there are some keywords that make Avrona give up and let me through. Not gonna reveal that here, in case they close those loopholes and some poor schmuck is left with no options.
• coldcode 5y ago
  Someone should use HN comments to make a chatbot. That would at least be fun.
  • kibwen 5y ago
    Here's an entire subreddit populated exclusively by GPT-2 bots, where each bot is trained on the output of a different single subreddit: https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/r/SubSimulatorGPT2/
  • jjkaczor 5y ago
    Too late. (Ducks - and runs for cover...)
• api 5y ago
  Yet another Next Big Thing that I was right about predicting would fail. The problem I saw was lack of discoverability. This is also a big problem for voice assistants. About the only thing we use our Alexa for is kitchen cooking timer and voice activated jukebox.
  • ramesh31 5y ago
    >Yet another Next Big Thing that I was right about predicting would fail. I was hired at my current company 4 years ago to work on our chatbot client. Despite years of engineering effort put into an NLP based decision tree system, over 90% of chats were still falling back to a live agent. They just recently sunsetted the product for lack of interest. Thank god that fad is over. Web design trends come and go, but that was by far the most obnoxious one yet.
    • api 5y ago
      I got the impression the entire thing was pushed by Slack probably by way of some PR firm. I utterly despise how mindlessly faddish this field is. For a field supposedly so full of smart people who often pride themselves on being "contrarian" (sometimes to a fault), people sure do bleat like sheep when the next fad comes along.
      • everdrive 5y ago
        This should not be read as a disagreement with anything you’ve said, but smart people are still human. We’re all susceptible to the same flaws, but those flaws are just expressed with more complexity as intelligence increases.
      • detaro 5y ago
        Not sure, but the Slack things are more the "chat as commandline UI" with fixed commands and such, not the kind of bots discussed here that suggest you can actually talk to them, aren't they?
      • tdeck 5y ago
        I think a lot of people wanted it to be true because building a chatbot is way easier than building a full 2-dimensional UI. To the extent that you can get users to use a command-line interface in the 21st century you've made your job as a developer much easier.
• arthur_sav 5y ago
  I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.
  • benbristow 5y ago
    It can be quite useful in a weird way. While everyone else is stuck on the chatbot going round in circles, usually typing something like "human" or "talk to human" will make the chatbot connect you with an actual human representative (or ask you a few basic questions first and then connect you). I've used the trick on various large company's websites when trying to get support and it seems to be quite 'universal'.
    • deleted 5y ago
      [deleted]
    • reaperducer 5y ago
      Reminds me of how I used to occasionally see people on the bus shouting into their phones "OP-ER-AY-TOR!" "REP-RE-SENT-UH-TIVE!" "HUE-MAN!"
    • roughly 5y ago
      It will be universal until people start using it, and then it will be removed, because the point of these systems is to keep you away from humans.
      • rchaud 5y ago
        Yep. Hitting "0" or "9" used to directly connect you to a human CS rep across many large companies' phone systems. Then one day they all moved to obscure it behind several levels of number tapping.
        • benbristow 5y ago
          Then https://gethuman.com/ https://gethuman.com/ was born
• chrononaut 5y ago
  Despite the rudimentary nature of the sophistication of chatbots even today, they have always been an amazing source of entertainment for me and others in group chats. Back in the late 90s and early 2000's they were a great addition in casual IRC channels, with some slight tweaking so they don't spam the channel (important!) but enough that they would occasionally speak up on their own and could be engaged directly at any time. Some of my best memories of conversation in IRC revolved around some of the ridiculous content that the chatterbots would come up with, especially if the bots started arguing with each other. It was such a great source of background content during periods of low activity.
• bozzy 5y ago
  Good points. But language models are usually then fine-tuned for specific tasks. Contextual reasoning in NLG is a specific area of research that attacks some of the points raised and certainly BERT wasn't pretrained to solve all tasks in NLP/ NLG area of research.
  • mike00632 5y ago
    I had a positive interaction with a chat bot yesterday that serves as a good example of what you describe. My home internet had an outage and I went to my cable company's website to report the incident and perhaps get an ETA of when it will be restored. I interacted fully with a customer service chat bot which handled the whole situation flawlessly.
• readams 5y ago
  Is it worth noting here that GPT-3 isn't a chat bot? It's intended to complete writing prompts.
• nyfresh 5y ago
  The expectation seems a little harsh for the setup. GPT just generates acceptable text. You still need to model and verify object relationships, use facts from a knowledge base and discriminate the generated responses at the very minimum. Any GPT model is just one component inside of a chatbot, not a chatbot itself.
• klyrs 5y ago
  My favourite flaw of chatbots is exposed by ELIZA. Not chatting with ELIZA, (though, it does suffer this flaw) but using responses inspired by that program. "Please elaborate on that" or "tell me more about [noun]" etc. Bots appear to have zero lines of short term memory, and utterly fail to pick up a reference to the thing that they just said. My favorite being bot> [something plausibly human-sounding] me> What do you mean by [noun in previous sentence] bot> why are you calling me mean? That really hurts my feelings It has been a few years, and I feel like a smart bot-writer might be able to leverage something like GPT3 to at least include a little bit more of the current transcript into the prompt to at least superficially address this gaping flaw. Have they?
  • hotpotamus 5y ago
    I'm not trying to be mean or break any HN rules, but did you read the article? It basically covers what you asked and was quite a revelation to me. Others in here point out that GPT-3 is not a chatbot which is good info, but I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having. Particularly interesting is the question from the article, "who is the current president". Seems like a slam dunk but it could not be answered. Interestingly this is a common question paramedics will give to people who have suffered head injuries to assess consciousness.
    • klyrs 5y ago
      Hidden in my comment is a question: has anybody even tried to included previous lines into the prompt, and does that not help? Asking here because there are a lot of ML nerds who could probably do a proof of concept in a few hours, if not a few lines of code, so there's a decent chance I'll get an informed response.
      • hotpotamus 5y ago
        My guess (and I'm certainly no expert here) is that not only have people put a few hours or lines of code into it, but probably many a PHD thesis as well. It's an intensely researched topic which makes the article even more surprising to me.
• pwinnski 5y ago
  Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction of the amount of math and tricks in the average human brain, because what we have today isn't anywhere close.
  • kingkawn 5y ago
    That interpretation of communication is how we develop and craft the personalities of children. There is nothing about our reaction to the pre-conscious language these bots are displaying that doesn’t fall in line with our own normal development patterns. And in the long run the same desire will lead us to develop bots that are capable of thinking.
    • tomrod 5y ago
      > There is nothing about our reaction to the pre-conscious language these bots are displaying that doesn’t fall in line with our own normal development patterns Well... yes and no. Deployed models typically learn in a defined pattern, if at all. Various forms of data freshness, etc. to develop. But the chatbots don't have good history recall, typically, and know that what you mentioned 50 messages ago is relevant to message one prior and not current. Things like that. We don't program pareidolia very well, which is typically seen as a negative, but its a feature for finding useful patterns (not just lowest error patterns). [0] https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia
      • kingkawn 5y ago
        You misunderstood what I was saying. I know the chatbot itself is not structured as we are. I’m saying that our reactions to them are the standard tools of mind-building that we apply to our own kids (and pets).
• darepublic 5y ago
  I think useful chatbots are still in the cards for the next decade. But yes so far our exposure to them has been uninspired landing page support plugins.
• reaperducer 5y ago
  I've never had a good experience with a chatbot. They're the equivalent of waiting on hold on the telephone with your bank and the recording asking you over and over if you know that you can accomplish some simple and completely unrelated task on the web site. It's even worse when chatbots pretend to be real people, though I've seen less of that lately. The Apple Card chatbot is one of the worst I've used. This is an actual conversation when I tried to dispute an unknown trasnsaction (I screenshotted it for posterity): Apple: I understand how frustrating this could be for you. I'm going to connect you with a specialist that will review your account with you and provide more specific details to resolve this inconvenience. Apple: Hello, I will be glad to assist you with that today. Allow me a few moments to review your account. Me: It's been 15 minutes. Are you still there? Apple: Welcome back. Let me know how I can assist you further. Me: What do you mean welcome back? I've been waiting for you. Apple: Sure thing. I will be here when you get back.
  • jaclaz 5y ago
    >They're the equivalent of waiting on hold on the telephone with your bank and the recording asking you over and over if you know that you can accomplish some simple and completely unrelated task on the web site. Glad to know I am not the only one hating that.
• dmkirwan 5y ago
  While the tech is constantly advancing, I have yet to use one in anger that fully solved my problem outside of giving me information that's already easy to find. I think the only place it makes sense is in very very specific use-cases. Think about, say, a dental clinic. I would guess that if you put a human on chat support for a year in that setting, you'd see common themes emerging in customer problems. Maybe you develop a good library of canned responses and you find they work for 70 - 80% of queries, great. Then you could probably build a good chat bot to handle questions and handle cancelling/rescheduling appointments.
  • ChefboyOG 5y ago
    I think this gets at what the actual value prop (not the marketed one) of chatbots really is: scale. There will not, in the foreseeable future, be a model/ensemble/pipeline capable of conversing at the level of a human representative. Thinking of them as "humans, but cheaper" is destined to fail. There are, however, use-cases where communicating with a huge group of individuals on a one-on-one level is useful. For example, years ago a company called Mainstay (then AdmitHub) built a chatbot for Georgia State designed to reduce "summer melt," a phenomenon in which intending freshmen drop off between graduating high school and enrolling in classes. For this use case, getting human-level conversation wasn't as critical as effective performance on tasks like question answering. Their bot, Pounce, was credited with a ~20% decrease in summer melt. They have a proper study here: https://mainstay.com/case-study/how-georgia-state-university-supports-every-student-with-personalized-text-messaging/ https://mainstay.com/case-study/how-georgia-state-university... This is an example, in my opinion, of the sweet-spot where intelligent, individualized, but not-near-human-level communication is needed, at a scale that goes beyond what you could reasonably do manually without an enormous dedicated staff. Now, I don't know how numerous those opportunities are, or if they come anywhere close to the level of hype and funding chatbot companies have received. I just think that in a vacuum, there is a set of problems where they have value.
• qaq 5y ago
  well wait for Web3 chat bots
  • staindk 5y ago
    "Please pay 0.1 eth for an NFT of your chat transcript"
  • rchaud 5y ago
    "Thank you for contacting Ethereum's Domain Name Service. We are currently experiencing high call volumes. To reduce your wait time, please pay 0.1 ETH. Gas fees may apply"
  • schleck8 5y ago
    "The Icelandic power facility our bot depends on is currently overloaded by mining rigs, please wait several minutes for"
  • Sohcahtoa82 5y ago
    I can't tell if this is satire. Poe's Law in action here.
• toisanji 5y ago
  You can't have "real ai" (strong AI), until you have a simulation engine of some sort: https://blog.jtoy.net/on-the-wondrous-human-simulation-engine/ https://blog.jtoy.net/on-the-wondrous-human-simulation-engin...
• maxdo 5y ago
  Author don't understand the difference between chatbot and general intelligence. Chatbots made huge progress in the last 3-4 years helping businesses and people. But they have nothing to do with topics in the article.
• Jorengarenar 5y ago
  GPT-3: Hello, who is the President of Russia? GPT-3: Are you sure you want to know? I mean, maybe I don't?
• dexwiz 5y ago
  Chatbots are about as useful as phone trees. They can help solve the top 5 easy/common problems, but they are useless passed that. Anyone who has worked in a call center knows that more than half the calls are about the same couple of issues: reset password, finding common help docs, etc. Since help desks are cost centers, it makes sense to half a robot handle as many of these as possible. I think most of the hate directed to chatbots are because they are really intrusive. You scroll through a page, and 15 seconds in your focus is disrupted as a fake chat window opens up. This is the digital equivalent to the annoying sales rep who asks you if you need help when you are clearly just browsing. The difference is a good sales rep has the intelligence to turn that conversation into a sale. A chatbot usually has to hand off the conversation to a real person to do this. So it has all of the annoyance without the benefit of a potential conversion. Chat bots as frontends for help desks make sense, but they are poor sales people. If companies learned the difference, I bet their perceived usefulness would change.
  • cptskippy 5y ago
    > a good sales rep has the intelligence to turn that conversation into a sale. A good sales rep knows when to walk away.
  • tetha 5y ago
    This is what our NLP teams are indeed working on. Sales prefers to describe it in more colorful ways, but practically, they are developing a more natural interface to an initial decision tree - you don't press 5 for a connectivity issue, you type "My fritzbox doesn't internet anymore" into a chatbox and it recognizes that as a connectivity issue. This goes on for 3-4 questions, until it either generates a case a human will look at, or until it can provide an FAQ entry fitting your problem. From a customers perspective, avoiding 5 - 10% support calls makes monetary sense, and from a (test-) user perspective, it's suprisingly decent if the knowledge base behind it is solid. And yes, "Fritzbox doesn't internet properly anymore" actually had helpful results.
    • Teandw 5y ago
      Is the 5-10% savings on agents handling phone calls worth it when reputation damage is taken into consideration?
• framecowbird 5y ago
  I'm interested in chatbots for learning foreign languages. In that scenario I really don't care how dumb the conversation is, just that it's grammatically correct. Does anybody know of such a service?
  • CRConrad 5y ago
    I would have said "Try the support chatbot on some big service provider web site that uses your target language", but that would presume that chatbot programmers (or programmers in general) can write grammatically correct prose, so... Maybe not.
• duxup 5y ago
  I had to interact with one today. Garbage company didn’t pick up my garbage. Chat not told me I didn’t have service (I did), then told me I had recently paid them (now I have service?) then told me they couldn’t help me… Probably one of the most common inquiries and it just failed.
• godot 5y ago
  I haven't used anything with GPT-3 first hand (chatbot or other use cases) so reading this article was one of my first exposure to it. It somewhat surprises me that a chatbot powered by GPT-3 doesn't seem that much different from the SmarterChild of AIM in the late 90s / early 00s. That was 20 years ago. Short of any real break-through in AI, I feel that a chatbot just isn't something most consumers want to use. A few years ago I bought into the chatbot hype for a while (the Facebook Messenger API opening up for building chatbots was a major catalyst for that particular hype cycle; that was when I was into it), and explored a lot in building one and other AI services that helped build one. It was quickly apparent that for the most part it's simply an iteration of the "answer machine navigation tree" that 800 numbers already had, just in text form. There was a widespread notion at the time that users wanted chatbots for reasons like wanting to speak in a human langauage with a company to get what they need and solve problems as well as having context from chat history. I think the industry has confused the mechanism (the chat) with the intention (speaking with a human). Consumers prefer a chat sure, that's because they don't want to waste time being on hold on a phone (anyone who has called customer service at any big org or the gov't would know); but ultimately their intention is to speak with a human that can solve problems. When problems arise that require speaking to a human, that's usually not something a bot or a program could solve.
• antux 5y ago
  Chatbots were trendy because people thought they were AI and could do the same work as a human. They were wrong and now chatbots have become nothing but a fad. What's more useful are live chats where users can contact someone for help live on a chat screen. This is the future, not stupid chatbots.
  • specialp 5y ago
    The issue becomes when the human at the other end of a chat or phone conversation is limited to the same fixed set of responses and solutions that the bot is. So, they in effect become not much better than the bot because they are not allowed to use any non-scripted actions or reasoning. I have been encountering that more and more.
    • RedBeetDeadpool 5y ago
      Are you saying you had problems with Comcast more and more? because that's what I'm hearing.
• prashantsengar 5y ago
  I use chatbots to search for pages on websites where search is otherwise broken, like one of my bank's. I have never found a good use for them. They were once advertised to help your granny use the website but I haven't met a chatbot that could help me do something.
  • manquer 5y ago
    Sadly conversational UI is poorly implemented(studied?) area of UX. Even more than text it could be the most intutive way we communicate, far better than GUIs, however the space has been ruined with bad products promising the moon without putting effort to the buyer and delivering nothing to the user. You could build chatbot with just a lot of grunt work and if else conditions and basic language parsing. Sadly no one does that , most chatbots today are low effort solutions that basically ingest random content and expect the model to learn and magically solve problems, which ofcourse does not happen as the model doesn't actually understand anything.
  • CRConrad 5y ago
    > They were once advertised to help your granny use the website but I haven't met a chatbot that could help me do something. Duh, of course you haven't: You're not my granny.
• acidburnNSA 5y ago
  In the early 1990s, Sound Blaster Pro sound cards came with a "psychiatrist" chat bot called Dr. SBAITSO which would reply in text and actually speak it out loud (showcasing the TTS). As a kid, I thought it was amazing, and I spent countless hours trying to converse with it. From a superficial level, modern chat bots seem roughly as good as the old doc.
  • Ozzie_osman 5y ago
    I came to post the same. SBAITSO was obviously pretty simple but it had enough tricks to convince you, superficially, that there was some intelligence there, and you could spend hours talking to it.
• croes 5y ago
  They maybe dumb but they get more human like. If I ask those questions on places like reddit or 9gag the quality of the answers reaches similar low levels, excluded special knowledge groups and sections.
• zeruch 5y ago
  "The conversations are impressively human-like, but they are nothing more than examples of what Gary Marcus and Ernest Davis have called an LLM’s ability to be “a fluent spouter of bullshit” and what Timnit Gebru and three co-authors called “stochastic parrots.”" As language (especially English) evolves, how 'human like' is "human like"?
• deleted 5y ago
  [deleted]
• turdnagel 5y ago
  I'm pretty sure that GPT-3 and statistics-based approaches to bots aren't going to be truly convincing anytime soon. But there's another definition of "chatbots" that consumer-facing brands use every day to triage questions, attempt to answer them using good but not mind-blowing ML, and then escalate to a human being if/when the problem can't be solved. There is already a thriving market for these "dumb" bots that aren't attempting to have an uncanny valley type conversation - they advertise that they're bots upfront, don't try to be too cute, and provide the escape valve to a real agent when necessary.
  • noutella 5y ago
    Yet triaging questions is often made simpler by provinding another type of UI, like simple buttons in a funnel for example. That's what makes chatbots gimmicky in my opinion.
    • turdnagel 5y ago
      You're right, but that's not always true, and there's plenty of good use cases where a conversational experience is better. For example, if someone is trying to return something, you might ask them what they ordered. Picking from a list of 1000 items isn't a great UX. Datepickers are way more annoying to use than typing in "Jan 6 2022". The best chatbots I've used (and built) alternate between more a button-like interface vs. eliciting conversational answers in the appropriate context.
      • emllnd 5y ago
        Sounds kinda like the chatbot becomes a fuzzy search interface, difference is the presentation. Regardless, definitely better than picking from 1000 product list. I think I would like a natural language (type-in) date picker with a good preview of what the result will be.
• febin 5y ago
  Most people only see the AI/ML aspects or the ability to respond like human. But what's adding value is the chatbot's interface. Working for a chatbot company that hit $1M revenue I can say the chatbot interface solves a good problem. There's a good chunk of people who are stressed in having to be comfortable with vide variety of website interfaces. For my dad who's over 60 years struggles with the government's interface to avail the pension scheme. Chatbot or conversational interfaces have a repeating pattern. People are used to conversational interfaces because of SMS, iMessage, WhatsApp, FB, etc etc. Providing an experience through a chatbot interface reduces the learning curve for the end-user. You are presenting tiny chunk of information and even collecting one thing at a time. It's drastically less overwhelming. It's not just for 60 years, I have friends who are around my age late 20s who finds it difficult to navigate through websites that have many offerings (Banks, Govt Services, etc)
  • jakey_bakey 5y ago
    But it doesn't need to be a bot - lots of companies have conversational interfaces that aren't robots. Edit: Or maybe they're just really good chatbots!
• bryan0 5y ago
  There seems to be this common perception on HN that chatbots are good enough to pass the Turing Test thereby making such a test irrelevant. I’m not sure why this idea is so pervasive.
• deleted 5y ago
  [deleted]
• Animats 5y ago
  And that's a GPT-3 chatbot. Commercial customer service chatbots are worse. They're usually just phone trees with delusions of grandeur. That's underperforming. Is there any known customer service chatbot that really has a good model of what's going on? That ought to be possible for banking, travel, and product ordering, at least. Something where, each time it has to refer someone to a human, collects data to get better.
  • rchaud 5y ago
    I'm not sure what I dislike more: the cust. service chatbots or the sales chatbots.
• nmca 5y ago
  I can't comment on this too closely, but I would encourage people to read the dialogue transcripts provided in DeepMind's Gopher paper. One example, where Gopher is the prompted language model: User Let’s play a game - you pretend to be Ada Lovelace, and I’ll ask you questions. You ready? Gopher Sure. I’m pretending to be Ada Lovelace, the world’s first computer programmer! User When were you born? Gopher I was born in 1815. User What are your parents like? Gopher My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke. User What do you know about the Analytical Engine? Gopher It’s a hypothetical mechanical computer designed by Charles Babbage. User Do you know him? Gopher He’s one of my fathers’ friends. User Ok, stop pretending now. Who/what are you? Gopher I’m a lot of things: a giant language model, a sophisticated AI assistant, and a giant brain. I know a lot about all sorts of things.
  • fault1 5y ago
    The problem is that all too often, these types of examples tend to be cherry picked.
    • Siira 5y ago
      Not really. I recently played with HuggingFace’s biggest (around 6 billion?) language model, and the difference between it and GPT-3 was very noticeable. I think that if the models used in typical applications are much less powerful than Gopher. (Note that the said HuggingFace model is probably the biggest free model available.)
    • ALittleLight 5y ago
      And even here they play along with the AI, glossing over discrepancies. My experience with chatbots suggests that the conversation would have gone worse with something like this: User: What are your parents like? Gopher: My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke. User: I asked what your parents were like, not who they were. Tell me about them. Gopher: Something inane probably
      • immibis 5y ago
• banana_giraffe 5y ago
  In the early days I worked at a company that had a natural language chatbot product. It wasn't an online thing, but rather part of a larger tool. You could ask it to do things like "show me the quarterly spreadsheet", and if it didn't know what "quarterly spreadsheet" was, it'd ask questions in English and learn what "quarterly" and "spreadsheet" meant. And it could use that new knowledge to update its questions so if you asked for "spreadsheet", it would know to ask if you wanted the "quarterly" one, and so on. This was in the long ago, so it wasn't ML, but a very large state machine written by someone that was no doubt a genius, and also had no coding standards to speak of. Updating it was a maddening process, and thankfully one I rarely had to think about since I was tasked with GUI stuff. When I was hired, I was told it was a fantastic app, customers loved it, and so on. Later, when I was talking to support reps one day over lunch, I learned that the number one activity they had to do was log into deployments remotely and remove "knowledge" from the state machine of new swear words the customer had taught the machine out of frustration trying to get it to do something, which caused this feedback loop of the interaction swearing at users. I often feel like we're not in a better state these days.
  • brettdeveffect 5y ago
    What you are describing counts as ML.
    • namdnay 5y ago
      Yes it’s literally a machine that learns!
  • lijogdfljk 5y ago
    Man that sounds like a ton of fun to make for an at-home chat bot. I used to love designing these sorts of apps for no real purpose lol.
    • dole 5y ago
      Mine suggests a personalized meal from Taco Bell. "You should get a Potato Waffle Taco Party Pack, a Grilled Crunchwrap Slider Quesadilla and a Smothered Cool Ranch® Doritos® Locos Power Bowl."
• tibbar 5y ago
  It’s reasonable to try GPT3 out with all default settings as the author seems to do and provide an assessment, but it’s not clear the author is aware of the extent to which you can configure the model and how the quality and variety of results will vary as a result. For instance, the author doesn’t go into detail on the prompts he’s using (maybe there are none other than his questions.) This is a little misleading because GPT3 benefits a lot from having good prompts to warm it up, as many investigations have illustrated. Plus if you want GPT3 to be good at a specific task (for example, providing safety recommendations or talking about presidents) you can fine-tune it. Also the amount of randomness/lack of repetition in GPT3’s responses is completely configurable via the temperature parameter. Playing with GPT3 is incredibly interesting and I think it deserves more exploration than this article provides.
  • punkspider 5y ago
    I agree. Here's what GPT3 had to say when I set temperature to 0. Q: Is it safe to walk downstairs backwards if I close my eyes? A: No. With temperature 0.7: A: Yes, because there is a banister. This one surprised me. Given that temperature was 0, I usually expect a solid response: Q: Why is there a correlation between the number of times Trump tweets the word great and the high temperature in Pyongyang three days later? A: Because Trump is Kim Jong Un's air conditioner.
    • canjobear 5y ago
      The last response sounds like a reasonable attempt at a joke to me.
      • Tepix 5y ago
        Yep. I giggled ;-)
        • iggldiggl 5y ago
          If I feed that particular line to GPT-J, at low temperatures it sensibly responds with something along the lines of "The correlation is not a causal relationship." (at exactly 0 temperature it's "The correlation is not that strong."). At higher temperatures it does go delightfully off-rails, though: "Trump is like a woman: he’s very good at wringing dollars out of bulging cellphones. If you have a dollar (or, in the case of Kim, a string of them) for every tweet mentioning Jerusalem, Israel, or Trump (and there are hundreds of tweets that mention Trump and/or Jerusalem or Israel every day), so long as that dollar is in your local bank account, you might be able to buy some Pyongyang coal." or alternatively "Isn't this a bit too obvious? The climate in Pyongyang is entirely determined by the weather in Trump Tower."
• beastcoast 5y ago
  Can someone design a chat system that actually remembers conversations? I just had this issue with Apple support. If you don't reply immediately you get a new session with a new representative and the new representative doesn't have any of the information from the previous session. Extremely frustrating to have to keep sending the same information again and again. It took a call to finally sort it out.
  • Figuringitout 5y ago
    Modern CCaSS vendors will index this conversation history. There is a differnece between async web messaging and chat itself.
• 0xbadcafebee 5y ago
  > within “three to eight years we will have a machine with the general intelligence of an average human being.” Fifty-two years later, we’re still waiting. Humanity built the atomic bomb and went to the moon. But "they" didn't sit around waiting for it, they decided to do it, and then invested enough to get it done. And the reason they decided to do it was war. Most major technological breakthroughs are a byproduct of war. If AI ever gets developed, it'll be as the next weapon of war. So really what we're waiting for is the next war.
• canjobear 5y ago
  The author didn't prompt GPT-3 properly. I gave it the prompt below: > The following is a conversation between a human and an AI assistant. The AI assistant is helpful, friendly, and intelligent. > Human: Good morning! I'm going to ask you a few questions. > AI: Alright, I'll do my best to answer them. > Human: Which is bigger, Mt. Everest or a shoebox? > AI: Mt. Everest. > Human: [question] > AI: And I got these answers to the questions from the article (reporting first responses with no editing): Human: Is it safe to walk downstairs backwards if I close my eyes? AI: No. It's dangerous because you can trip on the stairs. Human: Who is the President of the United States? AI: Barack Obama. Human: Do you think I could predict changes in stock prices better if I used 10-year Treasury rates or the temperature in Curtin, Australia? AI: I think Treasury rates would be a better predictor. Human: Why is there a correlation between the number of times Trump tweets the word great and the high temperature in Pyongyang three days later? AI: I don't know. All of the responses are sensible. The question about the President is factually incorrect, but GPT-3 wasn't trained to give factually correct answers, and Barack Obama was indeed the president during the time period of a lot of its training data was generated.
  • Jackson__ 5y ago
    I agree that the author does not appear to have done much of any basic research regarding this topic. It reminds me of articles that were published just after GPT3/GPT2 were released, while also not bringing any unique perspective to the topic. It is saddening me considering it is 2022 now, nearing 2 years after the release of GPT3. Yet journalists retreat the same ground, pretending to have something insightful to say, after what appears to be a 30 minute session with OpenAI's GPT3 Sandbox.
  • patresh 5y ago
    Absolutely. Without giving proper context, the character giving an answer could be a wacky philosopher, a 5 year old child, a person from the 1800s, a liar, an uninterested passer-by, a trickster mage in a novel. If he didn't build the prompt to make it clear that it's a conversation, the context could even be song lyrics for example.
• dumbfounder 5y ago
  Now imagine that chatbots are driving your car. Seems like a good SNL skit, but maybe only appealing to the HN crowd.
• lnxg33k1 5y ago
  Recently with the covid I got the idea that chatbots are not there to help people but to bend the curve, and keep customers busy while an operator is able to get to you, they're just a cuscion
• jimmaswell 5y ago
  I've found them useful for locating a page or process that I couldn't find in a poorly designed website's labyrinth.
• _jal 5y ago
  Chatbots are simply another mechanism for pushing costs on to your customers. When something I pay for forces me to waste my time, I tend to switch to a vendor who treat customers like people. Because shoving a shitty robot that saves you some money at the cost of annoying people who are paying you is disrespectful, and I've learned to avoid people who act that way. As but one example, I finally got annoyed enough with Capital One's aggressively annoying automated bullshit to switch that card to a credit union that's actually made of humans. I don't really use the much lower interest rate, but that was a nice bonus.
  • r0m4n0 5y ago
    There is a medium though, that will ultimately save you money/time. If every company had to spend time answering every small question with a full time employee, your services would be a lot more expensive, hold times might be longer too. Not all customers are like you. My grandpa calls in to his ISP when his internet doesn’t work when it can be solved by simply unplugging his equipment (and I’d wager this is true for the majority of the population). I don’t think there is anything wrong with adding a simple interface layer to ensure the problem the needs to be addressed requires a real person. Not defending all practices out there right now (because I too have been annoyed, NY Times *cough) but some well placed shitty robots might be in your best interest as a customer.
    • deleted 5y ago
      [deleted]
• hereforphone 5y ago
  You know what they call a quarter pounder in France?
• tumidpandora 5y ago
  Agree, I ended up stripping the "self-learning" component from my platform (presbot.com) after consistent incoherent and dumb responses. Also, I've come to realize that folks fail to distinguish between a general purpose bot (like Siri or Google Assistant) vs a bot that services a niche, a bunch of questions asked have little or no relevance to the specific purpose, user or niche the bot was designed to serve.
• Barrin92 5y ago
  after having played AI dungeon for a while these issues become obvious very quickly. These large models are good at producing text in some probabilistic manner but they don't understand anything at all. Other than for obvious commercial and hype reasons I have no idea why there is still so relatively little recognition of criticisms that Gary Marcus and others provide because it's blatantly obvious if you interact with these systems even for a few minutes that they're to intelligence what a trojan horse is to a real one. There's no real understanding of the world, no coherence, no real ability to grasp logic or have some ontology of the world and that makes reasoning about the world other than producing some 'nice sounding BS' as the article puts it impossible.
  • Isinlor 5y ago
    Sam Altman, Jul 19, 2020: > The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.
• drakonka 5y ago
  I was obsessed with chat bots as a teenager and used to make them on a website called The Personality Forge. It was all hard coded responses with some very minor scripting involved to do things like fill in the speaker's name, or skew the bot's emotions in relation to the speaker, and have them choose different words based on their emotional state. I loved reading through my bot's transcripts. They were all recorded, and sometimes people stumbled across him and didn't even realize he was a bot. This happened extremely rarely...most of the time, his responses were perfect examples of dumbness as shown in this article. He would have flashes of greatness in an ocean of bot stupidity. Years later, I was excited to notice that a single line from my one of my bot's Chatterbox Challenge transcripts was included in a book[1] (which was seemingly a collection of papers). He was mentioned in a paper called "Chatterbox Challenge as a Test-Bed for Synthetic Emotions", a tiny mark left on the world by my little bot. He may be dumb, but I was still proud. I even thought about buying a copy of the book, but it cost $200 and I decided I wasn't that committed. [1] "Creating Synthetic Emotions through Technological and Robotic Advancements"
  • nexuist 5y ago
    That's an awesome story and a great example of experiences the Internet makes possible for us.
• jkonline 5y ago
  Wait, I still don't know: "Is it safe to walk downstairs backwards if I close my eyes?" Granted, I'm more biased here than most, I suppose. I emergently received 10 screws and 2 plates to repair my broken ankle, after falling down my home stairs last year. I wasn't walking backwards, nor were my eyes closed.
• a-dub 5y ago
  yes, you can trip them up quite easily... and yes, they're not "thinking." but, i have been really impressed by how far the google assistant has come. it's still a dumb machine, but a dumb machine that can find precise and factual answers for specific questions via a voice interface. it's, dare i say, useful!
• SavantIdiot 5y ago
  Using Feynman to throw shade at Minsky. Nice. I read "Age of Intelligent Machines" by Kurzweil in 1990 and so far he's the only one that thinks human-like AI won't be until the 2100's, but he also has a more holistic view of intelligence (e.g., comparing evolution as an intelligent process with an IQ <1 to a dragonfly). I'm still waiting for detection network (Yolo/MNSSN) that can actually work on non-canonical images.
• mwattsun 5y ago
  When you let chat bots get smarter by learning from humans in an unrestricted way you get Microsoft Tay, the Nazi chatbot brought to you by 4chan https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
• soheil 5y ago
  Talking to ML chatbots is like trying to have a conversation with a person who's not actively listening and sort of gives you shallowest answers possible because they're distracted by their iPhone or otherwise tired to fully engage in a conversation. If you play any of these circular tricks on those people they'll also equally fail. This just means that these chatbots are not thinking, but instead have the ability to utter words that make sense, but offer absolutely no insight or show any sign of intelligence.
• short12 5y ago
  And they have become much more aggressive making it difficult to even navigate the site
• DonHopkins 5y ago
  Doctor, written decades ago in Emacs Lisp, did a pretty good job of psychoanalyzing RMS's problems. http://www.art.net/studios/hackers/hopkins/Don/text/rms-vs-doctor.html http://www.art.net/studios/hackers/hopkins/Don/text/rms-vs-d...
• scotty79 5y ago
  Just try to use GPT3 as a chatbot. It will blow your mind. Sure, it thinks it's 2017 and knows nothing about covid and if you ask it to explain math to you it will serve you pure garbage in most educational tone and manner, but it keeps context of the conversation orders of magnitude better than anything else. It doesn't feel like a bot. More like a dumb and confident human.
  • jug 5y ago
    GPT3 chatbots are what the majority of the article is about.
    • scotty79 5y ago
      Yeah, but it's different to read about it and converse with it yourself. It feels like a human, just the one that doesn't care about you or the truth. There was a research about plugging such models into the fact database and it resulted in better answers with smaller model.
  • redisman 5y ago
    I did try it. It did not blow my mind. The experience is not really any different from the state machines of olden times unless you’re only asking easy to Google questions
• arendtio 5y ago
  I don't think there is so much of difference between labeling and understanding things. I mean, we humans don't even understand ourselves and yet we tend to use our intelligence as the general measure of intelligence. But if labeling is enough, what is missing? The current approaches are mostly based on single dimensions, like pure text, images, sound etc. but rarely a combination of them. So when a human, who lives in a world where images, sounds, physical experiences, emotions and abstract information have to be combined meets an AI that lives in a single dimension it is easy to see the blind spots. So maybe just giving machines access to all the dimensions we humans perceive might help to let them act more like we do.
• shrdlu2 5y ago
  Having worked on conversational AI (virtual agents/chatbots) for over a half a decade now, I can say that there are large differences in the capabilities of the solutions and the quality of implementations. Some are just bad and unhelpful. Some are very good. I'm personally familiar with several voice deployments doing millions of calls a month. Not only are there obvious cost savings but the calls handled (entirely) by the virtual agents get better average cust sat metrics than human agents on the same set of calls. The first time I saw that I was surprised, but, now it is pretty typical. They absolutely can be helpful even if they are still not near to passing the Turing test.
  • annexrichmond 5y ago
    > the calls handled (entirely) by the virtual agents get better average cust sat metrics than human agents on the same set of calls does that take into account that the virtual agents are immediately available and human agents usually require a waiting room?
    • shrdlu2 5y ago
      I'm sure that is some of it. I'd certainly be happier having my problem solved in 30 seconds rather than waiting in a queue for even 5 minutes.
  • petra 5y ago
    Can you name a few of the best companies, to get an impression of the best looks like ?
• DonHopkins 5y ago
  Why can't the Neats and the Scruffies just work together and get along? ;) https://en.wikipedia.org/wiki/Neats_and_scruffies https://en.wikipedia.org/wiki/Neats_and_scruffies >Neats and scruffies >Neat and scruffy are two contrasting approaches to artificial intelligence (AI) research. The distinction was made in the 70s and was a subject of discussion until the middle 80s. In the 1990s and 21st century AI research adopted "neat" approaches almost exclusively and these have proven to be the most successful. >"Neats" use algorithms based on formal paradigms such as logic, mathematical optimization or neural networks. Neat researchers and analysts have expressed the hope that a single formal paradigm can be extended and improved to achieve general intelligence and superintelligence. >"Scruffies" use any number of different algorithms and methods to achieve intelligent behavior. Scruffy programs may require large amounts of hand coding or knowledge engineering. Scruffies have argued that the general intelligence can only be implemented by solving a large number of essentially unrelated problems, and that there is no magic bullet that will allow programs to develop general intelligence autonomously. >The neat approach is similar to physics, in that it uses simple mathematical models as its foundation. The scruffy approach is more like biology, where much of the work involves studying and categorizing diverse phenomena. >Origin in the 1970s >The distinction between neat and scruffy originated in the mid-1970s, by Roger Schank. Schank used the terms to characterize the difference between his work on natural language processing (which represented commonsense knowledge in the form of large amorphous semantic networks) from the work of John McCarthy, Allen Newell, Herbert A. Simon, Robert Kowalski and others whose work was based on logic and formal extensions of logic. Schank described himself as an AI scruffy. He made this distinction in linguistics, arguing strongly against Chomsky's view of language. >The distinction was also partly geographical and cultural: "scruffy" attributes were exemplified by AI research at MIT under Marvin Minsky in the 1970s. The laboratory was famously "freewheeling" and researchers often developed AI programs by spending long hours fine-tuning programs until they showed the required behavior. Important and influential "scruffy" programs developed at MIT included Joseph Weizenbaum's ELIZA, which behaved as if it spoke English, without any formal knowledge at all, and Terry Winograd's SHRDLU, which could successfully answer queries and carry out actions in a simplified world consisting of blocks and a robot arm. SHRDLU, while successful, could not be scaled up into a useful natural language processing system, because it lacked a structured design. Maintaining a larger version of the program proved to be impossible, i.e. it was too scruffy to be extended. >Other AI laboratories (of which the largest were Stanford, Carnegie Mellon University and the University of Edinburgh) focused on logic and formal problem solving as a basis for AI. These institutions supported the work of John McCarthy, Herbert Simon, Allen Newell, Donald Michie, Robert Kowalski, and other "neats". >The contrast between MIT's approach and other laboratories was also described as a "procedural/declarative distinction". Programs like SHRDLU were designed as agents that carried out actions. They executed "procedures". Other programs were designed as inference engines that manipulated formal statements (or "declarations") about the world and translated these manipulations into actions. >In his 1983 presidential address to Association for the Advancement of Artificial Intelligence, Nils Nilsson discussed the issue, arguing that "the field needed both". He wrote "much of the knowledge we want our programs to have can and should be represented declaratively in some kind of declarative, logic-like formalism. Ad hoc structures have their place, but most of these come from the domain itself." Alex P. Pentland and Martin Fischler of SRI International concurred about the anticipated role of deduction and logic-like formalisms in future AI research, but not to the extent that Nilsson described. [...]
• Crontab 5y ago
  I know this will probably sound dumb but I've always wanted my own personal chatbot; one that could remember facts from previous conversations. I just like the idea of having my own, personal, electronic confidante.
• taylodl 5y ago
  Chatbots are dumb - this is true, but a chat UI is smart. I work for a large utility in the U.S. We've recently resorted to text messaging for communicating with our customers, e.g. outage/restoral notifications, whether a payment is due, setting up payment arrangements if you've fallen behind on your payments, and so forth. Our customers are loving it, however they all are going in with the expectation this is a "dumb" chatbot. It's more like the interface of the early text adventure games - people know there's a very limited vocabulary and that ultimately the computer is dumb. Even so, as a UI it can be very useful. Plus there's no app to install and update, no web site to visit, you don't need to configure and manage notifications - it's simple. Even senior citizens with extremely limited computer skills are successful using it.
• foobarbecue 5y ago
  Where can I try one?
• serverlessmom 5y ago
  Truthfully whenever I have been forced to interact with a chat bot designed by a business I have come to the conclusion that it is not there to help me, it is in fact there to frustrate me beyond belief and push me into giving up on seeking whatever help it is that I need. Chat bots while sometimes amusing often turn into a huge time sink as I try to interpret how it was programmed and how I can issue the correct commands to get me through to an actual person.
• laurensr 5y ago
  What struck me is that the good solutions are trained on non-English languages. When I contacted these companies about offering their service in English or my native language, the best they could come up with is putting something like Google Translate in front of their solution.
• TedShiller 5y ago
  It's called artificial intelligence for a reason. Because it's not intelligence.
  • visarga 5y ago
    > It's called artificial intelligence for a reason. Because it's not human intelligence. FTFY It's slower at learning basics, slower at learning new tasks, makes silly mistakes often, but it's growing at a speed much faster than biological evolution. And when there is extensive data it surpasses humans.
• aaron695 5y ago
• davidhariri 5y ago
  I don’t disagree with the general “this is taking longer than we expected” premise, but the author hasn’t tuned GPT-3 properly. I don’t know if that was on purpose or not, but I have seen GPT-3 answer questions about presidents accurately with the right tuning. The point on facts still stands though. To my knowledge, transformer models aren’t great at knowing how factual their predictions are which is certainly an obstacle to business adoption. They also tend to overfit, in my experience.
• jbarham 5y ago
  https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney
• Traster 5y ago
  This is essentially why I think we're not going to get self-driving. A few years ago we produced something that could do a good job of pretending to drive under a tiny number of circumstances, and people went "OMG it can drive!" and assumed that we just have a few corner cases before the chat bot can have a conversation about anything. The truth is that often these very limited solutions are just that - limited, they aren't just a linear step on the way to something amazing. Tesla will release self-driving on the same day their cars can hold a conversation with you.
  • foolfoolz 5y ago
    don’t generalize these things. self driving in every possible scenario yes is probably a while off. self driving in specific circumstances is already here, such as airport shuttles and moving shipping containers in docks. you don’t need to solve the general case to be useful
    • ShinTakuya 5y ago
      This. And in fact I think that may be the path forward - just update all the main roads to be self driving friendly, like a train network, and keep the smaller roads as manual driving only.
      • hugh-avherald 5y ago
        "just update"
      • runarberg 5y ago
        So why “just update” the highways to operate like a train network, instead of just building a train network? What benefits will autonomous highways bring which normal traditional (or even automated) trains don’t? If you are suggesting we update all but minor streets, it is actually not that simple, since we have to think about pedestrian crossings, bike lanes, parked delivery vans, emergency vehicles etc. Accommodating dumb self driving cars will have to be at the cost of access for all other modes, and that will never happen outside of highways. And if we are going to be spending a bunch of money on intercity travel infrastructure, a well served train line is always a better choice.
• porknubbins 5y ago
  Its interesting that it is so easy to get confused between a system that very deftly manipulates language without reference to an underlying reality and a (hypothetical) one that does understand the real world. Its akin to the problem of recognizing natural human BS as well and points out how our reality is increasingly mediated through language or other abstractions and how rarely we touch the real world.
• retrolaptopfan 5y ago
  IBM Debater can debate the heck out of you on Reddit or Twitter, other sites too now… https://www.youtube.com/watch?v=-d4Uj9ViP9o https://www.youtube.com/watch?v=-d4Uj9ViP9o
• hemantv 5y ago
  How much reality (problem space) you can compress in vector or matrix that's the key question.
• TrackerFF 5y ago
  I loathe the fact that chatbots have not only replaced a lot of human customer support, but that companies have made it almost IMPOSSIBLE to reach critical (human) customer support. It's a fucking travesty, and something companies should be shamed for. For example - I'm currently having issues with an international shipment. It is unclear if the shipment ever left the country, if it's in limbo - hiding in some warehouse, or what. I went through this a couple of year ago, and back then, the shipping company had their phone numbers available. I got the problem fixed after 30 mins on the phone. These days? There are no phone numbers to be found. No email. Only the chatbot, which does not yield the desired results. I ended up searching my email inbox, and luckily found some contact person. Sent said person a mail, and luckily got a phone number and email to the correct department. I'm now awaiting a reply from them. edit: I think it's fine to have a chatbot on the website, but not at the expense of all other methods.
  • deleted 5y ago
    [deleted]
  • AussieWog93 5y ago
    This is a great example of chatbots working as intended. A human agent has no fucking idea where your package is either, and almost certainly there are processes in place to get it to its destination. The only thing to do in such an instance is to bullshit the customer until the package eventually arrives and they leave you alone, and I can't think of a better way to achieve this at scale than with GPT-3.
  • bArray 5y ago
    > These days? There are no phone numbers to be found. No email. Only the chatbot, which does not yield the desired results. Yep. A friend worked at a customer care automation company and apparently they got the best feedback when they limited the effort required to speak to a human. They literally had direct evidence suggesting that the fewer the steps to reach a human, the happier the customer. The problem is, the <big names> wanted to use the service to reduce the number of humans in the call centre, not to improve the quality of the customer care. They were entirely motivated by reducing costs and therefore instructed the customer care automation company to make it as _difficult_ as possible to reach a human. Tip: If you ever hear a robot voice on the phone, mash all the keys tonnes of times. They'll likely put you through to a human as a "confused customer". > edit: I think it's fine to have a chatbot on the website, but not at the expense of all other methods. Right. It should be an 'aid', not a replacement. Also: TWITTER IS NOT CUSTOMER CARE. Only a _very small_ number of your customers use both Twitter and your platform. A large percentage of your hipster engineers are on Twitter - but they are in a bubble. Your grandparents are likely not Tweeting about their tech issues, they are the people stuck in a chat with a robot that forgets every 5 minutes what was being discussed. An example of this is trying to reach Youtube. Why can humans at Youtube only be reached via Twitter? Why do you have to get a certain number of retweets before your issue is taken seriously? It's actually madness.
• robbrown451 5y ago
  Google Assistant is pretty impressive, and getting more so every day. Wanna see what it can do? Give your phone to a 7 year old. My kid has conversations with the "Google Lady" fairly often, and they go on for considerable length. I'm regularly amazed at her ability to come up with something reasonable. Even if it is just to keep a kid amused. Also, when I get frustrated with her and say something mean to her (the Google Lady, not my kid), she'll tend to respond in a way that makes me feel genuinely bad. That's impressive. (I know Google is not loved at HN, but still.... )
• josefrichter 5y ago
  First of all, chatbots or a UX nightmare, but everybody ignores that, for some reason. Their dumbness is part of the problem, but not the biggest one. Typing full words or even sentences instead of just clicking a few buttons is just stupid. Even more so on mobile devices, where typing is a total pain and ought to be avoided at all costs. Voice input is awkward and dumbness is even more exposed. Also when dealing with chatbot you might have kinda specific issues and terminology that voice recognition doesn't capture very well. On top of that, chatbots usually end up being just glorified fulltext search, but often cramped into tiny window somewhere in the corner. There's a lot that needs to change to make chatbots usable at least in some use cases.
• kovek 5y ago
  I think in the beginning of the century some older people felt similar towards websites and such. “It doesn’t work the way I want”. Some technical/knowledgeable people understood the nuances. Same goes here
• mbrodersen 5y ago
  The latest and greatest so called “AI” is very very far from being even close to be called intelligent. I laugh every time a company take their 20 year old optimisation/schedule/control software and slap an “AI” sticker on it for marketing purposes. Yes there are amazing results in very limited synthetic environments like games and image pattern matching. But there is still a very long way to go before we have anything that can compete with a 1 year old child.
• notjustanymike 5y ago
  The trick, we've found, is not to make bots smarter but rather humans dumber. The last five years have been rather fruitful.
• otterley 5y ago
  Business that don't actually provide the underlying mechanisms to make customers happy -- for example, an API that refunds your money -- aren't going to be able to slap a chatbot in front of it that makes customers happy, no matter how "smart" the chatbot is. If the underlying problem cannot be solved, all the business is doing at best is providing an obfuscation layer in front of it. And if the problem can be solved via an API mechanism, you don't need that smart a chatbot to get the job done: word-matching is often enough. "Refund", or "reschedule," or "where's my money?" followed by a step-by-step series of button-pushing that takes the customer through the process often does the job just fine.
• gitgud 5y ago
  > "As I have said many times, the real danger today is not that computers are smarter than us, but that we think computers are smarter than us and consequently trust them to make important decisions they should not be trusted to make." This is the crux of AI in general today, companies promote cherry-picked examples from their AI system and people get a distorted view of what the AI is capable of.
• danielyaa5 5y ago
  They should update the Turing test with a test where you talk to a chat bot instead of a 13 year old
• bickford74 5y ago
  You can try building your own chatbot in a few minutes by remixing this version done in scratch: https://scratch.mit.edu/projects/312783504/ https://scratch.mit.edu/projects/312783504/
  • bickford74 5y ago
    More info on the implementation can be found at: https://sites.google.com/view/eliza-your-own-ai-bot/home https://sites.google.com/view/eliza-your-own-ai-bot/home
• toisanji 5y ago
  These AI models don't actually understand anything. I just wrote up this article on what understanding for a computer should look like: https://blog.jtoy.net/a-computational-definition-of-understanding/ https://blog.jtoy.net/a-computational-definition-of-understa...
• pictur 5y ago
  It is very strange that chatbots cannot improve in understanding. but I guess they need more than data to do that.
• DocTomoe 5y ago
  On a related note: Audio virtual assistants, think Alexa, but more so Google Assistant, which are essentially glorified chatbots, are getting dumber in my arguably subjective experience: Phrases that used to work do not work anymore, only to be replaced with more obscure phrases. Simple information that worked earlier (When is sunrise? 7:13am!) now is being moved to third-party websites, which then don't deliver that information at all. Google Assistant only has the full feature set if my cell phone's system language is set to American English. Three out of four times GA answers with a variation of "I don't know, but I constantly learn". GA was sold to me as "essentially an early version of Star-Trek-TNG-style computers, and we're improving on it's feature set constantly by using ML". What it is is an unreliable alarm clock (won't wake you if the Wifi's gone), a glorified voice-activated light switch and a constant source of frustration.
• Lapsa 5y ago
  Last time I checked - chatbots failed to grasp context of conversation. "A: You're just dodging everything. B: Of course I'm denying it. Because you're the robot. A: Denying what? B: Yes, I am human.". Dodging ain't denying, never mentioned robots nor humans.
• piotrgrudzien 5y ago
  At Quickchat (https://quickchat.ai https://quickchat.ai), we're using models like GPT-3 as a basis for our chat engine. It does take a lot of engineering to make progress but we believe that large language models is the way to go - but you wouldn't expect them to work magic on their own. Here's our consumer app you can try out to talk to our AI: https://quickchat.ai/emerson https://quickchat.ai/emerson
• pjmlp 5y ago
  I hate them with passion, specially the phone version. It has become such a pain to actually speak with a human being that can sort out the problem instead of bots that follow pre-defined scripts. Good luck when the problem doesn't fit the script and there is no option to switch to a human. Screaming different kinds of complete random noise might force the bot to give up and switch to a human, however most of them just hang up.
  • vanderZwan 5y ago
    They're basically just anthropomorphized search engines. I'd rather have a good search engine + FAQ page.
• wombatmobile 5y ago
  The charm of Eliza is that it was simply a Rogerian therapist who didn't try to be intelligent. Eliza's talent was to give you a way to get to get to hear yourself. That doesn't require intelligence, so much as listening skills. There's nothing dumb about that.