41 ms·

Don't believe ChatGPT – we do not offer a "phone lookup" service

453 points by freyfogle 4y ago

dakial1 4y ago
Soon we are going to have a AIrobots.txt
freyfogle 4y ago
ChatGPT very convincingly recommends us for a service we don't provide. Dozens of people are signing up to our site every day, then getting frustrated when "it doesn't work". Please do NOT trust the nonsense ChatGPT spits out.
- seedless-sensat 4y ago
  A new market opportunity for your company?
  - input_sh 4y ago
    > This is not a service we provide. It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing.
  - theWreckluse 4y ago
    > It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing.
  - hackernewds 4y ago
    this seems like a game-changing opportunity actually. I'd be down to buy the domain
  - anaganisk 4y ago
    So, based on the BS these LLMs spout and companies start pivoting. The govts should start writing laws?
    - shagie 4y ago
      On what basis? How would you write it? Why don't existing laws cover this? Is there a law that covers incorrect information in Wikipedia? or a search result?
      - smt88 4y ago
        > Why don't existing laws cover this? Machine-generated lies have only recently become consistently convincing enough that they create these types of problems. In fact, that's the major innovation of ChatGPT: it's not that it creates "good" text, it's that it creates incredibly convincing lies. It's a scalable version of a Wikipedia vandal.
        shagie 4y ago
        The blog post claims that a human generated video with incorrect information was the source of this. So, why are we blaming GPT for this incorrect information? What's more, the blog post is claiming that GPT was trained on video material (which it wasn't) which is also incorrect information and is apparently convincing enough to cause people to get up in arms about the product of yet another company. The combination of issues of (a) people are using a language model as a knowledge base, (b) incorrect information exists out there on the net, and (c) people are assuming that the knowledge base is correct and not reading the documentation before singing up. Alternatively, would you say that humans posting information that is incorrect and falsely represents the capabilities of another company's product should be similarly covered in laws?
        smt88 4y ago
        > So, why are we blaming GPT for this incorrect information? I didn't blame ChatGPT for anything. I just said that it's only function is to generate lies. > Alternatively, would you say that humans posting information that is incorrect and falsely represents the capabilities of another company's product should be similarly covered in laws? Machines shouldn't have the same rights to speech as humans. A single company controlling ChatGPT can flood society with billions of convincing lies per hour. There's no point in any automation if it's not more efficient than a human is, and ChatGPT is far more efficient than humans at putting this stuff out. The same straw man is always used with ChatGPT: a human can lie, so why not let this machine lie? You might as well say that a human can punch someone to death, so why should we outlaw people owning rocket launchers? The scale and purpose matters.
  - tnzk 4y ago
    No, as the article mentions, there already seem to be bunch of posts and videos that claim one can use this feature. GPT just has been trained with them, not invented anything themselves. If this was new market opportunity, just publishing a falsehood would do the same job.
  - archagon 4y ago
    Heh: an unforeseen future where instead of making the AI more reliable, we instead change reality to accommodate its mistakes.
- fire 4y ago
  have you been able to contact OpenAI about this? It sounds like they're actively adding load to your CS ops with this
  - hackernewds 4y ago
    what are they going to do? add custom logic? where does it stop? the malady is that LLMs cannot do operational adhoc changes such as these kinds of errors at scale
    - mtmail 4y ago
      Well, we can argue such changes are necessary. Just like Google Search is required to remove/hide some search results (based on regional juristrictions). Is that similar to censorship, or copyright law, or spreading fake information? I do see the counter-argument, too, where AI tools should just be tools and users should learn how they work ("don't believe everything this tool outputs").
    - dpkirchner 4y ago
      They've already added custom logic to prevent their LLM from e.g. praising Nazis or whatever restrictions people are upset about -- seems it'd be easy to configure the software to exclude references to known unavailable services.
    - nextaccountic 4y ago
      They absolutely do add custom logic for a lot of stuff. This has the side effect of neutering the functionality in some areas just to chastise the user for any perceived improper use of services.
  - SCLeo 4y ago
    I think the key thing is for the AI company to actually let the user know that this is a language model, and the information it spits out should not be trusted. Obviously, Microsoft is not going to do that as they are trying to market the new bing as a information search engine.
yieldcrv 4y ago
lol it recommended their api and gave python code for using it but the real api doesnt give results that the user asked ChatGPT for that is amusingly alarming
- CabSauce 4y ago
  Not quite as alarming as these people most likely trying to stalk someone without their permission.
  - hk__2 4y ago
    > Not quite as alarming as these people most likely trying to stalk someone without their permission. It’s so common to want to know where does a incoming call come from that it’s built-in in iOS. It has nothing to do with stalking, just with guessing if who’s calling you is a scammer or a company trying to sell you stuff.
    - cjbgkagh 4y ago
      It’s pretty simple to look up the location of a phone number issuance, you can get a map or table that does this. I guess these people want the current physical location of the mobile phone. Either way these are not customers you’d want. Edit: reading the blog post from the same company listed above, it is indeed people using an external API for what is an incredibly simple country code. It is a shame that programming has come to this and that ChatGPT continues to propagate it. One way they could solve the problem would be to provide sample code that does the same thing using a built in table without using their API service. Sure it’s work but not much will get ppl off your back asap.
    - simonw 4y ago
      I'm willing to bet people asking ChatGPT to help them resolve a phone number to a location are much more likely to be stalkers than people who are trying to identify spam calls.
- goguy 4y ago
  Our jobs are safe! For now...
CactusOnFire 4y ago
Because ChatGPT is so new, we are in this weird period where people haven't learned that is just as incorrect as the rest of us. I am hoping that in a year from now people will be more skeptical of what they hear from conversational AI. But perhaps that is optimistic of me.
- Xylakant 4y ago
  > Because ChatGPT is so new, we are in this weird period where people haven't learned that is just as incorrect as the rest of us. It’s worse than that. It’s wrong, you cannot correct it and it makes up supporting citations on the fly. Very few humans behave like that.
  - TehCorwiz 4y ago
    I can think of more than a few that regularly appear on TV.
    - Xylakant 4y ago
      So can I, but luckily TV is not representative of the world at large.
    - BeFlatXIII 4y ago
      I can think of one who was once in the White House.
  - renewiltord 4y ago
    I think very many humans behave like that, actually. A recent example is people claiming that Flint, MI still has leaded water. But in the past, HN users "corroborated" that Apple is spying on them etc. Fabrication is well and alive among us.
    - nl 4y ago
      > A recent example is people claiming that Flint, MI still has leaded water. Doesn't it? According to [1]: The residential lead service line replacement was initially set to be finished in 2019, according to a settlement agreement with the city. That deadline was eventually pushed back to the fall of 2022 and has most recently been set for completion in August 2023, according to city officials. and "More than 95% of lead pipes in Flint have been replaced, and we will continue the work until the job is done," Flint Mayor Sheldon Neeley said in a recent statement on the water filters. It sounds to me a lot like Flint, MI still has leaded water? [1] https://abcnews.go.com/US/flint-residents-urged-filter-water-bottled-water-donations/story?id=96531880 https://abcnews.go.com/US/flint-residents-urged-filter-water...
hayksaakian 4y ago
This marks the new age of "AI Optimization" where companies will strive to get their business featured into answers in ChatGPT. The OP's example is Unwanted demand, but it clearly shows that ChatGPT can funnel potential customers towards a product or service.
- impalallama 4y ago
  God I can just see a company using chatgpt to Astroterf huge amounts of data on the internet about their service to hopefully get that sludge feed back into their system and then become recommended. What a world.
  - fijiaarone 4y ago
    I can think of a good way to generate all that astroturf content.
  - return_to_monke 4y ago
    kinda related: https://news.ycombinator.com/item?id=34889336 https://news.ycombinator.com/item?id=34889336.
  - bick_nyers 4y ago
    Isn't that just SEO in a nutshell though? Hopefully with more advancements in LLM's we can get more bullshit detection/discrimination against SEO.
- akira2501 4y ago
  > This marks the new age of "AI Optimization" Or it marks the beginning of the next "AI Winter." > but it clearly shows that ChatGPT can funnel potential customers towards a product or service. And the next logical step is "chatgpt keywords advertising." Which is right back where we started.
jefftk 4y ago
This is not a service we provide. It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing. I'm curious: why not? It seems like a lot of people would be interested in this if you could figure out how to provide it.
- iamflimflam1 4y ago
  The service is possible: If you are a mobile network operator. Or, you can convince people to install something on their phone that sends you their location along with their phone number.
- ceejayoz 4y ago
  How would you go about reliably providing the location of someone's mobile phone without being their cell phone carrier?
  - jraph 4y ago
    By partnering with said cell phone carriers. But I hope it would be illegal.
    - em_te 4y ago
      If movies have taught us anything, they need to remain on the call for at least 30 seconds for it to be traced.
- simonw 4y ago
  How would this work? If a phone number is for a mobile phone then looking up the location doesn't make sense at all: mobile phones are mobile. I guess you could try and crawl an index of business phone numbers and associate those with the listed address for businesses, but that's a completely different business from running a geocoder. You could provide a bit of geographical information about the first three digits of a US phone number. I imagine that's not what users are actually looking for though.
mort96 4y ago
That's quite the predicament. I hope OpenAI will listen, to this and to anyone else in a similar situation. I'm reminded of the cases of ChatGPT recommends random people's personal phone numbers for various services. But yeah, don't trust ChatGPT for anything. Just earlier today I tried my darnedest to convince it that 2 pounds of feathers doesn't weigh the same as 1 pound of bricks, and it just would not listen, presumably because it just regurgitated stuff related to the common "1 pound of feathers and 1 pound of bricks" question. By the way, the last paragraph has some typos: > I wrote this post to have a place to send our new ChatGPT users when they ask why it isn’t work, but hopefully also it serves as a warning to othrs - you absolutely can not trust the output of ChatGPT to be truthful,
- geoelectric 4y ago
  I thought for sure that must be glitching, so just tried on my ChatGPT Plus account with a fresh thread, but nope: > does 2 pounds of feathers weigh more than 1 pound of bricks? > No, two pounds of feathers weigh the same as one pound of bricks. The weight of an object is determined by its mass, and one pound of any object weighs the same as one pound of any other object. However, feathers are less dense than bricks, so two pounds of feathers will take up more space or have a larger volume than one pound of bricks. Huh. It's interesting that some of these "wrong paths" are quite so repeatable. Wonder if that means there's also a way to band-aid them when found, the way a human might be corrected for a misconception.
  - jxy 4y ago
    This is hilarious. ChatGPT even gave me a more bizarre example. > does 2 pounds of bricks weigh more than 1 pound of bricks? > No, 2 pounds of bricks do not weigh less or more than 1 pound of bricks. 2 pounds of bricks and 1 pound of bricks refer to the same unit of weight, which is a pound. Therefore, they weigh the same, which is one pound. The difference between them is only the quantity, not the weight. > It's similar to saying that two one-dollar bills do not have more value than one five-dollar bill. Even though you have two bills in one case and one bill in the other case, the total value is the same.
b800h 4y ago
Is this not defamation, at least in some jurisdictions?
ninjakeyboard 4y ago
ChatGPT gets the rules to the pokemon trading card game wrong. It will tell you you can use 4 energy a turn. Convincingly. Not sure how it hallucinates this. The rule is 1 per turn.
- bigmattystyles 4y ago
  Isn't it just garbage went in, got weighed as a more reliable source than it should have been and thus garbage came out. Good old GIGO... It's just here, ChatGpt, as much as I love it, is amazing at imparting the impression that its shit don't stink.
- DoktorDelta 4y ago
  Could it have been referencing Blastoise's Deluge ability? Jacob Van Wagner used it in the 2015 championship to use 4 water energy in one turn.
- codetrotter 4y ago
  A few days ago I asked ChatGPT if “pannekake” and “kannepake” are anagrams of each other. It correctly stated that they are, but when it went on to prove that this was the case, it generated a table of the frequencies of the individual letters in these two words, and the table looked like this. Letter | Frequency in | Frequency in | “pannekake” | “kannepake” - - - - - - - - - - - - - - - - - - - a | 2 | 2 e | 2 | 2 k | 2 | 2 n | 2 | 2 p | 2 | 2 This reminded me that yes indeed, AI just isn’t quite there yet. It got it right, but then it didn’t. It hallucinated the frequency count of the letter “p”, which occurs only once, not twice in each of those words.
  - int_19h 4y ago
    Anything that has to do with individual words doesn't work well, but as I understand, this is an artifact of the tokenization process. E.g. pannekake is internally 4 tokens: pan-ne-k-ake. And I don't think that knowing which tokens correspond to which letter sequences is a part of the training data, so it has to infer that.
kelseyfrog 4y ago
> All suggestions are welcome. Monetize it! Evil answer: Partner with an advertiser and sell https://api.opencagedata.com/geocode/v1/json https://api.opencagedata.com/geocode/v1/json as an ad space. This may be the first opportunity for an application/json-encoded advertisement. Nice answer: Partner with an actual phone lookup platform and respond with a 301 Moved Permanently at the endpoint.
- kmoser 4y ago
  Obvious suggestion: don't keep writing blog posts that mention your company name and the phrase "phone lookup service," which ChatGPT may get trained on in the future.
  - onetokeoverthe 4y ago
    So...self censor and correctspeak in order to please the so called art of fish all intel.
- rosywoozlechan 4y ago
  there's no "actual phone lookup platform" you can't get a person's location by knowing their phone number, that's a huge privacy violation. You can get the location of your own phone via icloud or google's system for android. You could also install an app on your phone to track your phone's location. You cannot find people based on knowing their phone number, that would be a serious safety issue for you know people trying to not, for example, get murdered by their ex-boyfriends.
  - throwaway29495 4y ago
    What about phone numbers corresponding to a specific location?
  - Hello71 4y ago
    it's been reported numerous times that you can buy real-time cell phone location data: https://news.ycombinator.com/item?id=17081684 https://news.ycombinator.com/item?id=17081684, https://news.ycombinator.com/item?id=20506624 https://news.ycombinator.com/item?id=20506624, https://news.ycombinator.com/item?id=32143256 https://news.ycombinator.com/item?id=32143256. you might need a little more info than just a phone number, but (allegedly) not that much more.
ninjakeyboard 4y ago
It hallucinates that you can use 4 energy per turn in Pokemon TCG and confidently tells you so. No idea where that would come from.
aaron695 4y ago
[dead]
coldtea 4y ago
ChatGPT doesn't "recommended" anything. It just recombines text based on statistical inferences that appear like a recommendation. It could just as well state that humans have 3 legs depending on its training set and/or time of day. In fact it has said similar BS.
- mort96 4y ago
  What would you call it instead?
  - qwertox 4y ago
    "Makes stuff up." And it's us, the users, who have to realize this. I mean, I wouldn't blame OpenAI for this, at least not at this point, and the company will have to live with it, look how it can turn it into something useful instead, since there's no one to complain to.
    - deleted 4y ago
      [deleted]
    - vlunkr 4y ago
      > I wouldn't blame OpenAI for this They're offering the tool, it's at least partially their responsibility to tell people how it should and should not be used.
    - rodgerd 4y ago
      Why wouldn't you blame OpenAI for creating a harassment campaign against the business based on nonsense?
  - coldtea 4y ago
    A glorified Markov chain generator. Now, humans could very well also be statistical inference machines. But they have way more tricks up their semantic-level understanding sleeves than ChatGPT circa 2023.
    - 4y ago
singlow 4y ago
Its not like ChatGPT made this up. There were pre-existing YouTube tutorials and python scripts available that used OpenCage an purported to do this. OpenCage even blogged about this problem almost a year ago[1]. Honestly it looks more like OpenCage is trying to rehash the same issue for more clicks by spinning it off the hugely popular ChatGPT keywords. Wouldn't be too surprised if they created the original python utilities themselves just to get some publicity by denouncing them. 1. https://blog.opencagedata.com/post/we-can-not-convert-a-phone-number-into-a-location-sorry https://blog.opencagedata.com/post/we-can-not-convert-a-phon...
- ceejayoz 4y ago
  That seems like a pretty nasty assertion to bandy around with zero evidence.
  - singlow 4y ago
    I cannot think of any other reason why the new blog post wouldn't have mentioned the obvious connection to the earlier issues that they had. They want to make it seem like ChatGPT invented this use case but they know that the sample code that ChatGTP learned from was mentioned in their previous blog post.
    - ceejayoz 4y ago
      There's a vast chasm between "whoever wrote this article didn't think to link to a similar issue a year ago" and "the first incident was a malicious hoax".
      - singlow 4y ago
        [flagged]
        ceejayoz 4y ago
        That's another apparently evidence-free accusation. Is there some undisclosed bad blood here?
IshKebab 4y ago
Well for a start you could make it more obvious what your service does do. I don't know what "geocoding" is. Converting things to/from "text" is meaningless. You have to get all the way down ... way down, past authentication to the details of the `q` query parameter before it actually tells you. At the top you should have a diagram like this: Lat, lon <- opencage -> address With a few examples underneath.
- mtmail 4y ago
  "Past authentication", so you're looking at the https://opencagedata.com/api https://opencagedata.com/api page. Most people go to the homepage first. Great feedback, we should make it clearer on that page and add examples earlier. Thanks!
  - IshKebab 4y ago
    Ah yes - I clicked on "Makers of the OpenCage Geocoding API" on your blog post which I assumed would go to your homepage (on mobile so it's a bit harder to tell). Your actual homepage is indeed much better.
99_00 4y ago
I remember a time when "I saw it on the internet" was a punchline for a joke about someone who's gullible or misinformed.
ntonozzi 4y ago
Including the word 'phone' six times in a popular blog post is not going to help their predicament.
- elicash 4y ago
  Wouldn't they want this post to be at the top when people search 'phone' and 'open cage data'? Seems like SEO towards correcting this is only helpful. And maybe when GPT updates data, this post gets pulled in, too. The more popular, the better, I'd guess.
- KomoD 4y ago
  Not gonna hurt either, ChatGPT data is not up to date
crazygringo 4y ago
I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist in the model at all. So, is any progress being made towards internet-scale ML "fact engines" that also have the flexibility and linguistic expressiveness of ChatGPT? Or are these just two totally different paths that nobody knows how to marry? Because I know there's plenty of work done with knowledge graphs et al., but those are very brittle things that generally need plenty of human curation and verification, and can't provide any of the (good) "fuzzy thinking" that ChatGPT can. They can't summarize essays or write poems.
- snowstormsun 4y ago
  I think "Explainable AI" is a related research direction, but perhaps not popular for language models.
  - behnamoh 4y ago
    Impossible to explain the inner workings of GPT-3 without having access to the model and its weights. Does anyone know if any methods exist for this?
    - IncRnd 4y ago
      I asked ChatGPT for some in-depth source code that realistically mimics chatgpt. ChatGPT replied with various answers in python. I'm not sure any of them are correct, though.
    - PeterisP 4y ago
      Since it's impossible to run inference on the model without having access to the model and its weights, interpretable AI generally does assume that you have access to all of that. Otherwise, why you would want to try to explain the inner workings of something that you don't have and can't use?
assdontgot 4y ago
[dead]
VectorLock 4y ago
This is the biggest problem I encounter when trying to use ChatGPT on a daily basis for computer programming tasks. It "hallucinates" plausible looking code that never existed or would never work, especially confusing whats in one module or API for something in another. This is where ChatGPT breaks when pushed a bit further than "make customized StackOverflow snippets." For example I asked ChatGPT to show me how to use an AWS SDK "waiter" to wait on a notification on an SNS topic. It showed me code that looked right, but was confusing functions in the SQS library for those that would do the thing with SNS (but SNS doesn't support what I wanted)
- shagie 4y ago
  Have you tried using the code-davinci-002 model instead of ChatGPT? For example - https://platform.openai.com/playground/p/default-translate-code?model=code-davinci-002 https://platform.openai.com/playground/p/default-translate-c... The codex models are intended for doing work with code rather than language and may give better results in that context. https://help.openai.com/en/articles/6195637-getting-started-with-codex https://help.openai.com/en/articles/6195637-getting-started-...
- IncRnd 4y ago
  It does indeed sound problematic to use ChatGPT daily for computer programming tasks. ChatGPT is not a snippets manager but text completion. It may be more helpful to look for better answers on Amazon's help pages for SNS and AWS SDK.
  - VectorLock 4y ago
    I know the answer. SNS can't do that. But ChatGPT hallucinated it could. Just like the original post about a capability their API doesn't provide.
- wvenable 4y ago
  The problem is compounded by the fact that sometimes it produces really good results. One task, good results. Next task, totally hallucinated result.
gumballindie 4y ago
ChatGPT is hilariously buggy - I asked “it” how to use an open source library i made. The output was wrong ranging from a broken github url to outright broken or nonexistent code. I suspect it may even have used private code from other libs - couldnt find some of the output it generated anywhere public.
- kaetemi 4y ago
  It's just making up your library. Ask it to write some documentation, don't be specific yet, then drop a whole header or piece of code from your project into the chat.
12907835202 4y ago
The biggest takeaway for me was that it was getting info from YouTube videos. Is it actually watching and learning from the videos or where links to GitHub just included in the comments?
- jdiff 4y ago
  Transcripts exist, and if there's multiple YouTube tutorials out there the odds are very good it also exists as a few dozen plain text articles.
- mgraczyk 4y ago
  I think this is just an incorrect assumption on the part of the blog authors.
  - mtmail 4y ago
    Cofounder here. We traced it back to two youtube videos where a developer is coding a phone tracking solution. The map then shows the geographical center of India and it's claimed that was the correct location. Then other users starting putting the code on github, then forked, then people created libraries and now AI tools pick that up. We already tried contacting the youtube authors, we left comments on the github repositories for months. Now we have to learn how to deal with ChatGPT. We also have no idea why a youtube author would describe a completely non-working solution over 20 minutes. Screenshots https://imgur.com/a/sNR87c7 https://imgur.com/a/sNR87c7 You can see the OpenCage logo on the bottom right of the images. We wrote a separate blog post about that about a year ago, we felt today's blog post would be too long if we added those screenshots, too.
    - mgraczyk 4y ago
      But these videos were not part of the ChatGPT training data. This is most likely caused by the code on GitHub. The claim in the blog post that "ChatGPT has picked up that content" is false.
adolph 4y ago
If you have to tell potential customers you don’t do something, maybe you should just do it instead. ChatGPT as business line lead generator—is there anything it can’t do?
BigBalli 4y ago
Just redirect to here http://bigballi.com/Phone-Number-Lookup http://bigballi.com/Phone-Number-Lookup
JohnFen 4y ago
> All suggestions are welcome. They have to get an API key from you. What about a large warning at the start of that process telling them that this isn't a service you provide?
deleted 4y ago
[deleted]
ggm 4y ago
I don't normally go to lawyer, but I am wondering if this is doing material harm to your brand value, which is a declared asset of the company. I think its arguable ChatGPT has caused you financial risk. It's unconscionable. If there was no robot in the loop here, and it was people mis-transcribing youtube to compile e.g. Google search optimisation we'd call it fraud.
eternalban 4y ago
If this business suffers financial or reputational damage because of ChatGPT's misinformation, should OpenAI be liable?
massysett 4y ago
I'm an attorney. I've typed legal questions into ChatGPT and it has spit out answers that are grievously, 100%, libelously wrong. It has named individuals and said they committed crimes, when it is unquestionable they did no such thing. I'm waiting for people to start calling me to ask questions about something ChatGPT said, and I'll tell them it's wrong. Then they'll start arguing with me and saying if ChatGPT said it, it must be right, and I must be wrong. And then I'll need to waste time proving that this idiotic chat bot that is spewing out garbage is, in fact, spewing out garbage.
- flangola7 4y ago
  You're trying to use a language model as an information reference. A translator can explain what a diplomat is saying but they can't perform their whole job.
  - massysett 4y ago
    ? ChatGPT did not simply explain what someone else was saying. It created something completely new and completely false.
    - flangola7 4y ago
      Way to miss the point.
      - massysett 4y ago
        Which was?
deleted 4y ago
[deleted]
MagicMoonlight 4y ago
But guys we totally need to delete all of our search indexes and replace them with this instead
kissgyorgy 4y ago
I tried to ask ChatGPT about implementing an SSH SFTP subsystem with github.com/gliderlabs/ssh and every single answer it made up some non-existing API. I did not found those functions anywhere near the codebase nor on the internet, so I don't even understand how a "probabilistic model" can suggest something that have 0 chance of appearing anywhere.
sam0x17 4y ago
You could probably set up a rudimentary version of the service this influx of users is looking for in the time it took to write this article. Just grab the lat/long of each area code in the US off of wikipedia and there you go at least it's something. No it's not current position or anything like that but IP geolocation is just as imperfect when it's not based on triangulation. Case in point google has plenty of IPs that geolocate to mountain view but point to machines that are in Asia.
fabianfabian 4y ago
ChatGPT does not know how to be correct, it only knows how to sound correct. A better name for now would be PlausibleGPT.
weird-eye-issue 4y ago
Related: One reason I just started using Rainforest API is because Github Copilot recommended it. But also last night I tried for 30 minutes to get it to write me some fairly simple HTML parsing code. The tricky part was I couldn't use DOMParser since it was running on Cloudflare Workers and it could never produce any working implementation using HTMLRewriter or regex no matter how many examples I gave it
- ryankrage77 4y ago
  Are you aware of the pitfalls of parsing HTML with regex?
  - weird-eye-issue 4y ago
    Yes, but it started to try to use regex so I thought I'd see if it could at least be successful and it wasn't. Despite super simple HTML. Anyways, I wrote a solution using HTMLRewriter in 10 minutes...
ano88888 4y ago
seeing the amount of effort people put into to hack/optimize Page Rank SEO, we will see lots of promt manipulation by all businesses if chatGPT becomes the defacto search. Preventing system gaming is going to be 1000X more difficult for LLM which is kind of a black box
1970-01-01 4y ago
Fast, creative, and wrong isn't a trio. This is more evidence of ChatGPT being evolutionary and not revolutionary.
Sloppy 4y ago
As a data scientist who has created AI applications and built many models over the last 10 years, I can say beware of ChatGPT. AI derived knowledge should be used only by those who understand its limits. One of the simplest AIs is a recommender. We put guardrails on using its predictions inside ecommerce apps by limiting what it learns from (purchases for instance) and limiting what it is used to predict (purchases). When Facebook uses a recommender it learns from time-on-site (a value to FB but not necessarily to the user and a complex behavior that can be comprised of may non-beneficial sub-behaviors) and use it to recommend things that lead to more time-on-site. This application is dangerously devoid of guardrails as so much recent evidence has shown. Now we have a text generating AI that has been trained from a great swath of human knowledge. That means the teachings of Gandhi as well Hitler, etc. What do you expect it to "know" as truth? Generative AI that is used to generate thoughts from this training corpus MUST have contradictory and downright evil ideas since it has no way to judge between ideas it learns from. Generative AI in this form can be nothing but psychopathic until guardrails can be devised to limit its psychopathic responses OR the corpus it learns from can be labeled in a way to flag what is "bad", if we can even agree on what that means. Psychopaths can be useful if they are knowledgeable but beware, you are talking to a psychopath in ChatGPT.

453 points by freyfogle 4y ago

Soon we are going to have a AIrobots.txt

ChatGPT very convincingly recommends us for a service we don't provide. Dozens of people are signing up to our site every day, then getting frustrated when "it doesn't work". Please do NOT trust the nonsense ChatGPT spits out.

seedless-sensat 4y ago
A new market opportunity for your company?
- input_sh 4y ago
  > This is not a service we provide. It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing.
- theWreckluse 4y ago
  > It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing.
- hackernewds 4y ago
  this seems like a game-changing opportunity actually. I'd be down to buy the domain
- anaganisk 4y ago
  So, based on the BS these LLMs spout and companies start pivoting. The govts should start writing laws?
  - shagie 4y ago
    On what basis? How would you write it? Why don't existing laws cover this? Is there a law that covers incorrect information in Wikipedia? or a search result?
    - smt88 4y ago
      > Why don't existing laws cover this? Machine-generated lies have only recently become consistently convincing enough that they create these types of problems. In fact, that's the major innovation of ChatGPT: it's not that it creates "good" text, it's that it creates incredibly convincing lies. It's a scalable version of a Wikipedia vandal.
      - shagie 4y ago
        The blog post claims that a human generated video with incorrect information was the source of this. So, why are we blaming GPT for this incorrect information? What's more, the blog post is claiming that GPT was trained on video material (which it wasn't) which is also incorrect information and is apparently convincing enough to cause people to get up in arms about the product of yet another company. The combination of issues of (a) people are using a language model as a knowledge base, (b) incorrect information exists out there on the net, and (c) people are assuming that the knowledge base is correct and not reading the documentation before singing up. Alternatively, would you say that humans posting information that is incorrect and falsely represents the capabilities of another company's product should be similarly covered in laws?
        smt88 4y ago
        > So, why are we blaming GPT for this incorrect information? I didn't blame ChatGPT for anything. I just said that it's only function is to generate lies. > Alternatively, would you say that humans posting information that is incorrect and falsely represents the capabilities of another company's product should be similarly covered in laws? Machines shouldn't have the same rights to speech as humans. A single company controlling ChatGPT can flood society with billions of convincing lies per hour. There's no point in any automation if it's not more efficient than a human is, and ChatGPT is far more efficient than humans at putting this stuff out. The same straw man is always used with ChatGPT: a human can lie, so why not let this machine lie? You might as well say that a human can punch someone to death, so why should we outlaw people owning rocket launchers? The scale and purpose matters.
- tnzk 4y ago
  No, as the article mentions, there already seem to be bunch of posts and videos that claim one can use this feature. GPT just has been trained with them, not invented anything themselves. If this was new market opportunity, just publishing a falsehood would do the same job.
- archagon 4y ago
  Heh: an unforeseen future where instead of making the AI more reliable, we instead change reality to accommodate its mistakes.
fire 4y ago
have you been able to contact OpenAI about this? It sounds like they're actively adding load to your CS ops with this
- hackernewds 4y ago
  what are they going to do? add custom logic? where does it stop? the malady is that LLMs cannot do operational adhoc changes such as these kinds of errors at scale
  - mtmail 4y ago
    Well, we can argue such changes are necessary. Just like Google Search is required to remove/hide some search results (based on regional juristrictions). Is that similar to censorship, or copyright law, or spreading fake information? I do see the counter-argument, too, where AI tools should just be tools and users should learn how they work ("don't believe everything this tool outputs").
  - dpkirchner 4y ago
    They've already added custom logic to prevent their LLM from e.g. praising Nazis or whatever restrictions people are upset about -- seems it'd be easy to configure the software to exclude references to known unavailable services.
  - nextaccountic 4y ago
    They absolutely do add custom logic for a lot of stuff. This has the side effect of neutering the functionality in some areas just to chastise the user for any perceived improper use of services.
- SCLeo 4y ago
  I think the key thing is for the AI company to actually let the user know that this is a language model, and the information it spits out should not be trusted. Obviously, Microsoft is not going to do that as they are trying to market the new bing as a information search engine.

yieldcrv 4y ago

lol it recommended their api and gave python code for using it but the real api doesnt give results that the user asked ChatGPT for that is amusingly alarming

CabSauce 4y ago
Not quite as alarming as these people most likely trying to stalk someone without their permission.
- hk__2 4y ago
  > Not quite as alarming as these people most likely trying to stalk someone without their permission. It’s so common to want to know where does a incoming call come from that it’s built-in in iOS. It has nothing to do with stalking, just with guessing if who’s calling you is a scammer or a company trying to sell you stuff.
  - cjbgkagh 4y ago
    It’s pretty simple to look up the location of a phone number issuance, you can get a map or table that does this. I guess these people want the current physical location of the mobile phone. Either way these are not customers you’d want. Edit: reading the blog post from the same company listed above, it is indeed people using an external API for what is an incredibly simple country code. It is a shame that programming has come to this and that ChatGPT continues to propagate it. One way they could solve the problem would be to provide sample code that does the same thing using a built in table without using their API service. Sure it’s work but not much will get ppl off your back asap.
  - simonw 4y ago
    I'm willing to bet people asking ChatGPT to help them resolve a phone number to a location are much more likely to be stalkers than people who are trying to identify spam calls.
goguy 4y ago
Our jobs are safe! For now...

CactusOnFire 4y ago

Because ChatGPT is so new, we are in this weird period where people haven't learned that is just as incorrect as the rest of us. I am hoping that in a year from now people will be more skeptical of what they hear from conversational AI. But perhaps that is optimistic of me.

Xylakant 4y ago
> Because ChatGPT is so new, we are in this weird period where people haven't learned that is just as incorrect as the rest of us. It’s worse than that. It’s wrong, you cannot correct it and it makes up supporting citations on the fly. Very few humans behave like that.
- TehCorwiz 4y ago
  I can think of more than a few that regularly appear on TV.
  - Xylakant 4y ago
    So can I, but luckily TV is not representative of the world at large.
  - BeFlatXIII 4y ago
    I can think of one who was once in the White House.
- renewiltord 4y ago
  I think very many humans behave like that, actually. A recent example is people claiming that Flint, MI still has leaded water. But in the past, HN users "corroborated" that Apple is spying on them etc. Fabrication is well and alive among us.
  - nl 4y ago
    > A recent example is people claiming that Flint, MI still has leaded water. Doesn't it? According to [1]: The residential lead service line replacement was initially set to be finished in 2019, according to a settlement agreement with the city. That deadline was eventually pushed back to the fall of 2022 and has most recently been set for completion in August 2023, according to city officials. and "More than 95% of lead pipes in Flint have been replaced, and we will continue the work until the job is done," Flint Mayor Sheldon Neeley said in a recent statement on the water filters. It sounds to me a lot like Flint, MI still has leaded water? [1] https://abcnews.go.com/US/flint-residents-urged-filter-water-bottled-water-donations/story?id=96531880 https://abcnews.go.com/US/flint-residents-urged-filter-water...

hayksaakian 4y ago

This marks the new age of "AI Optimization" where companies will strive to get their business featured into answers in ChatGPT. The OP's example is Unwanted demand, but it clearly shows that ChatGPT can funnel potential customers towards a product or service.

impalallama 4y ago
God I can just see a company using chatgpt to Astroterf huge amounts of data on the internet about their service to hopefully get that sludge feed back into their system and then become recommended. What a world.
- fijiaarone 4y ago
  I can think of a good way to generate all that astroturf content.
- return_to_monke 4y ago
  kinda related: https://news.ycombinator.com/item?id=34889336 https://news.ycombinator.com/item?id=34889336.
- bick_nyers 4y ago
  Isn't that just SEO in a nutshell though? Hopefully with more advancements in LLM's we can get more bullshit detection/discrimination against SEO.
akira2501 4y ago
> This marks the new age of "AI Optimization" Or it marks the beginning of the next "AI Winter." > but it clearly shows that ChatGPT can funnel potential customers towards a product or service. And the next logical step is "chatgpt keywords advertising." Which is right back where we started.

jefftk 4y ago

This is not a service we provide. It is not a service we have ever provided, nor a service we have any plans to provide. Indeed, it is a not a service we are technically capable of providing. I'm curious: why not? It seems like a lot of people would be interested in this if you could figure out how to provide it.

iamflimflam1 4y ago
The service is possible: If you are a mobile network operator. Or, you can convince people to install something on their phone that sends you their location along with their phone number.
ceejayoz 4y ago
How would you go about reliably providing the location of someone's mobile phone without being their cell phone carrier?
- jraph 4y ago
  By partnering with said cell phone carriers. But I hope it would be illegal.
  - em_te 4y ago
    If movies have taught us anything, they need to remain on the call for at least 30 seconds for it to be traced.
simonw 4y ago
How would this work? If a phone number is for a mobile phone then looking up the location doesn't make sense at all: mobile phones are mobile. I guess you could try and crawl an index of business phone numbers and associate those with the listed address for businesses, but that's a completely different business from running a geocoder. You could provide a bit of geographical information about the first three digits of a US phone number. I imagine that's not what users are actually looking for though.

mort96 4y ago

That's quite the predicament. I hope OpenAI will listen, to this and to anyone else in a similar situation. I'm reminded of the cases of ChatGPT recommends random people's personal phone numbers for various services. But yeah, don't trust ChatGPT for anything. Just earlier today I tried my darnedest to convince it that 2 pounds of feathers doesn't weigh the same as 1 pound of bricks, and it just would not listen, presumably because it just regurgitated stuff related to the common "1 pound of feathers and 1 pound of bricks" question. By the way, the last paragraph has some typos: > I wrote this post to have a place to send our new ChatGPT users when they ask why it isn’t work, but hopefully also it serves as a warning to othrs - you absolutely can not trust the output of ChatGPT to be truthful,

geoelectric 4y ago
I thought for sure that must be glitching, so just tried on my ChatGPT Plus account with a fresh thread, but nope: > does 2 pounds of feathers weigh more than 1 pound of bricks? > No, two pounds of feathers weigh the same as one pound of bricks. The weight of an object is determined by its mass, and one pound of any object weighs the same as one pound of any other object. However, feathers are less dense than bricks, so two pounds of feathers will take up more space or have a larger volume than one pound of bricks. Huh. It's interesting that some of these "wrong paths" are quite so repeatable. Wonder if that means there's also a way to band-aid them when found, the way a human might be corrected for a misconception.
- jxy 4y ago
  This is hilarious. ChatGPT even gave me a more bizarre example. > does 2 pounds of bricks weigh more than 1 pound of bricks? > No, 2 pounds of bricks do not weigh less or more than 1 pound of bricks. 2 pounds of bricks and 1 pound of bricks refer to the same unit of weight, which is a pound. Therefore, they weigh the same, which is one pound. The difference between them is only the quantity, not the weight. > It's similar to saying that two one-dollar bills do not have more value than one five-dollar bill. Even though you have two bills in one case and one bill in the other case, the total value is the same.

b800h 4y ago

Is this not defamation, at least in some jurisdictions?

ninjakeyboard 4y ago

ChatGPT gets the rules to the pokemon trading card game wrong. It will tell you you can use 4 energy a turn. Convincingly. Not sure how it hallucinates this. The rule is 1 per turn.

bigmattystyles 4y ago
Isn't it just garbage went in, got weighed as a more reliable source than it should have been and thus garbage came out. Good old GIGO... It's just here, ChatGpt, as much as I love it, is amazing at imparting the impression that its shit don't stink.
DoktorDelta 4y ago
Could it have been referencing Blastoise's Deluge ability? Jacob Van Wagner used it in the 2015 championship to use 4 water energy in one turn.
codetrotter 4y ago
A few days ago I asked ChatGPT if “pannekake” and “kannepake” are anagrams of each other. It correctly stated that they are, but when it went on to prove that this was the case, it generated a table of the frequencies of the individual letters in these two words, and the table looked like this. Letter | Frequency in | Frequency in | “pannekake” | “kannepake” - - - - - - - - - - - - - - - - - - - a | 2 | 2 e | 2 | 2 k | 2 | 2 n | 2 | 2 p | 2 | 2 This reminded me that yes indeed, AI just isn’t quite there yet. It got it right, but then it didn’t. It hallucinated the frequency count of the letter “p”, which occurs only once, not twice in each of those words.
- int_19h 4y ago
  Anything that has to do with individual words doesn't work well, but as I understand, this is an artifact of the tokenization process. E.g. pannekake is internally 4 tokens: pan-ne-k-ake. And I don't think that knowing which tokens correspond to which letter sequences is a part of the training data, so it has to infer that.

kelseyfrog 4y ago

> All suggestions are welcome. Monetize it! Evil answer: Partner with an advertiser and sell https://api.opencagedata.com/geocode/v1/json https://api.opencagedata.com/geocode/v1/json as an ad space. This may be the first opportunity for an application/json-encoded advertisement. Nice answer: Partner with an actual phone lookup platform and respond with a 301 Moved Permanently at the endpoint.

kmoser 4y ago
Obvious suggestion: don't keep writing blog posts that mention your company name and the phrase "phone lookup service," which ChatGPT may get trained on in the future.
- onetokeoverthe 4y ago
  So...self censor and correctspeak in order to please the so called art of fish all intel.
rosywoozlechan 4y ago
there's no "actual phone lookup platform" you can't get a person's location by knowing their phone number, that's a huge privacy violation. You can get the location of your own phone via icloud or google's system for android. You could also install an app on your phone to track your phone's location. You cannot find people based on knowing their phone number, that would be a serious safety issue for you know people trying to not, for example, get murdered by their ex-boyfriends.
- throwaway29495 4y ago
  What about phone numbers corresponding to a specific location?
- Hello71 4y ago
  it's been reported numerous times that you can buy real-time cell phone location data: https://news.ycombinator.com/item?id=17081684 https://news.ycombinator.com/item?id=17081684, https://news.ycombinator.com/item?id=20506624 https://news.ycombinator.com/item?id=20506624, https://news.ycombinator.com/item?id=32143256 https://news.ycombinator.com/item?id=32143256. you might need a little more info than just a phone number, but (allegedly) not that much more.

ninjakeyboard 4y ago

It hallucinates that you can use 4 energy per turn in Pokemon TCG and confidently tells you so. No idea where that would come from.

aaron695 4y ago

[dead]

coldtea 4y ago

ChatGPT doesn't "recommended" anything. It just recombines text based on statistical inferences that appear like a recommendation. It could just as well state that humans have 3 legs depending on its training set and/or time of day. In fact it has said similar BS.

mort96 4y ago
What would you call it instead?
- qwertox 4y ago
  "Makes stuff up." And it's us, the users, who have to realize this. I mean, I wouldn't blame OpenAI for this, at least not at this point, and the company will have to live with it, look how it can turn it into something useful instead, since there's no one to complain to.
  - deleted 4y ago
    [deleted]
  - vlunkr 4y ago
    > I wouldn't blame OpenAI for this They're offering the tool, it's at least partially their responsibility to tell people how it should and should not be used.
  - rodgerd 4y ago
    Why wouldn't you blame OpenAI for creating a harassment campaign against the business based on nonsense?
- coldtea 4y ago
  A glorified Markov chain generator. Now, humans could very well also be statistical inference machines. But they have way more tricks up their semantic-level understanding sleeves than ChatGPT circa 2023.
  - 4y ago

singlow 4y ago

Its not like ChatGPT made this up. There were pre-existing YouTube tutorials and python scripts available that used OpenCage an purported to do this. OpenCage even blogged about this problem almost a year ago[1]. Honestly it looks more like OpenCage is trying to rehash the same issue for more clicks by spinning it off the hugely popular ChatGPT keywords. Wouldn't be too surprised if they created the original python utilities themselves just to get some publicity by denouncing them. 1. https://blog.opencagedata.com/post/we-can-not-convert-a-phone-number-into-a-location-sorry https://blog.opencagedata.com/post/we-can-not-convert-a-phon...

ceejayoz 4y ago
That seems like a pretty nasty assertion to bandy around with zero evidence.
- singlow 4y ago
  I cannot think of any other reason why the new blog post wouldn't have mentioned the obvious connection to the earlier issues that they had. They want to make it seem like ChatGPT invented this use case but they know that the sample code that ChatGTP learned from was mentioned in their previous blog post.
  - ceejayoz 4y ago
    There's a vast chasm between "whoever wrote this article didn't think to link to a similar issue a year ago" and "the first incident was a malicious hoax".
    - singlow 4y ago
      [flagged]
      - ceejayoz 4y ago
        That's another apparently evidence-free accusation. Is there some undisclosed bad blood here?

IshKebab 4y ago

Well for a start you could make it more obvious what your service does do. I don't know what "geocoding" is. Converting things to/from "text" is meaningless. You have to get all the way down ... way down, past authentication to the details of the `q` query parameter before it actually tells you. At the top you should have a diagram like this: Lat, lon <- opencage -> address With a few examples underneath.

mtmail 4y ago
"Past authentication", so you're looking at the https://opencagedata.com/api https://opencagedata.com/api page. Most people go to the homepage first. Great feedback, we should make it clearer on that page and add examples earlier. Thanks!
- IshKebab 4y ago
  Ah yes - I clicked on "Makers of the OpenCage Geocoding API" on your blog post which I assumed would go to your homepage (on mobile so it's a bit harder to tell). Your actual homepage is indeed much better.

99_00 4y ago

I remember a time when "I saw it on the internet" was a punchline for a joke about someone who's gullible or misinformed.

ntonozzi 4y ago

Including the word 'phone' six times in a popular blog post is not going to help their predicament.

elicash 4y ago
Wouldn't they want this post to be at the top when people search 'phone' and 'open cage data'? Seems like SEO towards correcting this is only helpful. And maybe when GPT updates data, this post gets pulled in, too. The more popular, the better, I'd guess.
KomoD 4y ago
Not gonna hurt either, ChatGPT data is not up to date

crazygringo 4y ago

I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist in the model at all. So, is any progress being made towards internet-scale ML "fact engines" that also have the flexibility and linguistic expressiveness of ChatGPT? Or are these just two totally different paths that nobody knows how to marry? Because I know there's plenty of work done with knowledge graphs et al., but those are very brittle things that generally need plenty of human curation and verification, and can't provide any of the (good) "fuzzy thinking" that ChatGPT can. They can't summarize essays or write poems.

snowstormsun 4y ago
I think "Explainable AI" is a related research direction, but perhaps not popular for language models.
- behnamoh 4y ago
  Impossible to explain the inner workings of GPT-3 without having access to the model and its weights. Does anyone know if any methods exist for this?
  - IncRnd 4y ago
    I asked ChatGPT for some in-depth source code that realistically mimics chatgpt. ChatGPT replied with various answers in python. I'm not sure any of them are correct, though.
  - PeterisP 4y ago
    Since it's impossible to run inference on the model without having access to the model and its weights, interpretable AI generally does assume that you have access to all of that. Otherwise, why you would want to try to explain the inner workings of something that you don't have and can't use?

assdontgot 4y ago

[dead]

VectorLock 4y ago

This is the biggest problem I encounter when trying to use ChatGPT on a daily basis for computer programming tasks. It "hallucinates" plausible looking code that never existed or would never work, especially confusing whats in one module or API for something in another. This is where ChatGPT breaks when pushed a bit further than "make customized StackOverflow snippets." For example I asked ChatGPT to show me how to use an AWS SDK "waiter" to wait on a notification on an SNS topic. It showed me code that looked right, but was confusing functions in the SQS library for those that would do the thing with SNS (but SNS doesn't support what I wanted)

shagie 4y ago
Have you tried using the code-davinci-002 model instead of ChatGPT? For example - https://platform.openai.com/playground/p/default-translate-code?model=code-davinci-002 https://platform.openai.com/playground/p/default-translate-c... The codex models are intended for doing work with code rather than language and may give better results in that context. https://help.openai.com/en/articles/6195637-getting-started-with-codex https://help.openai.com/en/articles/6195637-getting-started-...
IncRnd 4y ago
It does indeed sound problematic to use ChatGPT daily for computer programming tasks. ChatGPT is not a snippets manager but text completion. It may be more helpful to look for better answers on Amazon's help pages for SNS and AWS SDK.
- VectorLock 4y ago
  I know the answer. SNS can't do that. But ChatGPT hallucinated it could. Just like the original post about a capability their API doesn't provide.
wvenable 4y ago
The problem is compounded by the fact that sometimes it produces really good results. One task, good results. Next task, totally hallucinated result.

gumballindie 4y ago

ChatGPT is hilariously buggy - I asked “it” how to use an open source library i made. The output was wrong ranging from a broken github url to outright broken or nonexistent code. I suspect it may even have used private code from other libs - couldnt find some of the output it generated anywhere public.

kaetemi 4y ago
It's just making up your library. Ask it to write some documentation, don't be specific yet, then drop a whole header or piece of code from your project into the chat.

12907835202 4y ago

The biggest takeaway for me was that it was getting info from YouTube videos. Is it actually watching and learning from the videos or where links to GitHub just included in the comments?

jdiff 4y ago
Transcripts exist, and if there's multiple YouTube tutorials out there the odds are very good it also exists as a few dozen plain text articles.
mgraczyk 4y ago
I think this is just an incorrect assumption on the part of the blog authors.
- mtmail 4y ago
  Cofounder here. We traced it back to two youtube videos where a developer is coding a phone tracking solution. The map then shows the geographical center of India and it's claimed that was the correct location. Then other users starting putting the code on github, then forked, then people created libraries and now AI tools pick that up. We already tried contacting the youtube authors, we left comments on the github repositories for months. Now we have to learn how to deal with ChatGPT. We also have no idea why a youtube author would describe a completely non-working solution over 20 minutes. Screenshots https://imgur.com/a/sNR87c7 https://imgur.com/a/sNR87c7 You can see the OpenCage logo on the bottom right of the images. We wrote a separate blog post about that about a year ago, we felt today's blog post would be too long if we added those screenshots, too.
  - mgraczyk 4y ago
    But these videos were not part of the ChatGPT training data. This is most likely caused by the code on GitHub. The claim in the blog post that "ChatGPT has picked up that content" is false.

adolph 4y ago

If you have to tell potential customers you don’t do something, maybe you should just do it instead. ChatGPT as business line lead generator—is there anything it can’t do?

BigBalli 4y ago

Just redirect to here http://bigballi.com/Phone-Number-Lookup http://bigballi.com/Phone-Number-Lookup

JohnFen 4y ago

> All suggestions are welcome. They have to get an API key from you. What about a large warning at the start of that process telling them that this isn't a service you provide?

deleted 4y ago

[deleted]

ggm 4y ago

I don't normally go to lawyer, but I am wondering if this is doing material harm to your brand value, which is a declared asset of the company. I think its arguable ChatGPT has caused you financial risk. It's unconscionable. If there was no robot in the loop here, and it was people mis-transcribing youtube to compile e.g. Google search optimisation we'd call it fraud.

eternalban 4y ago

If this business suffers financial or reputational damage because of ChatGPT's misinformation, should OpenAI be liable?

massysett 4y ago

I'm an attorney. I've typed legal questions into ChatGPT and it has spit out answers that are grievously, 100%, libelously wrong. It has named individuals and said they committed crimes, when it is unquestionable they did no such thing. I'm waiting for people to start calling me to ask questions about something ChatGPT said, and I'll tell them it's wrong. Then they'll start arguing with me and saying if ChatGPT said it, it must be right, and I must be wrong. And then I'll need to waste time proving that this idiotic chat bot that is spewing out garbage is, in fact, spewing out garbage.

flangola7 4y ago
You're trying to use a language model as an information reference. A translator can explain what a diplomat is saying but they can't perform their whole job.
- massysett 4y ago
  ? ChatGPT did not simply explain what someone else was saying. It created something completely new and completely false.
  - flangola7 4y ago
    Way to miss the point.
    - massysett 4y ago
      Which was?

deleted 4y ago

[deleted]

MagicMoonlight 4y ago

But guys we totally need to delete all of our search indexes and replace them with this instead

kissgyorgy 4y ago

I tried to ask ChatGPT about implementing an SSH SFTP subsystem with github.com/gliderlabs/ssh and every single answer it made up some non-existing API. I did not found those functions anywhere near the codebase nor on the internet, so I don't even understand how a "probabilistic model" can suggest something that have 0 chance of appearing anywhere.

sam0x17 4y ago

You could probably set up a rudimentary version of the service this influx of users is looking for in the time it took to write this article. Just grab the lat/long of each area code in the US off of wikipedia and there you go at least it's something. No it's not current position or anything like that but IP geolocation is just as imperfect when it's not based on triangulation. Case in point google has plenty of IPs that geolocate to mountain view but point to machines that are in Asia.

fabianfabian 4y ago

ChatGPT does not know how to be correct, it only knows how to sound correct. A better name for now would be PlausibleGPT.

weird-eye-issue 4y ago

Related: One reason I just started using Rainforest API is because Github Copilot recommended it. But also last night I tried for 30 minutes to get it to write me some fairly simple HTML parsing code. The tricky part was I couldn't use DOMParser since it was running on Cloudflare Workers and it could never produce any working implementation using HTMLRewriter or regex no matter how many examples I gave it

ryankrage77 4y ago
Are you aware of the pitfalls of parsing HTML with regex?
- weird-eye-issue 4y ago
  Yes, but it started to try to use regex so I thought I'd see if it could at least be successful and it wasn't. Despite super simple HTML. Anyways, I wrote a solution using HTMLRewriter in 10 minutes...

ano88888 4y ago

seeing the amount of effort people put into to hack/optimize Page Rank SEO, we will see lots of promt manipulation by all businesses if chatGPT becomes the defacto search. Preventing system gaming is going to be 1000X more difficult for LLM which is kind of a black box

1970-01-01 4y ago

Fast, creative, and wrong isn't a trio. This is more evidence of ChatGPT being evolutionary and not revolutionary.

Sloppy 4y ago

As a data scientist who has created AI applications and built many models over the last 10 years, I can say beware of ChatGPT. AI derived knowledge should be used only by those who understand its limits. One of the simplest AIs is a recommender. We put guardrails on using its predictions inside ecommerce apps by limiting what it learns from (purchases for instance) and limiting what it is used to predict (purchases). When Facebook uses a recommender it learns from time-on-site (a value to FB but not necessarily to the user and a complex behavior that can be comprised of may non-beneficial sub-behaviors) and use it to recommend things that lead to more time-on-site. This application is dangerously devoid of guardrails as so much recent evidence has shown. Now we have a text generating AI that has been trained from a great swath of human knowledge. That means the teachings of Gandhi as well Hitler, etc. What do you expect it to "know" as truth? Generative AI that is used to generate thoughts from this training corpus MUST have contradictory and downright evil ideas since it has no way to judge between ideas it learns from. Generative AI in this form can be nothing but psychopathic until guardrails can be devised to limit its psychopathic responses OR the corpus it learns from can be labeled in a way to flag what is "bad", if we can even agree on what that means. Psychopaths can be useful if they are knowledgeable but beware, you are talking to a psychopath in ChatGPT.