5 ms·

It drives me nuts how OpenAI has misleadingly marketed their products, and the tech community at large has failed to clarify what their products really are. Thi

It drives me nuts how OpenAI has misleadingly marketed their products, and the tech community at large has failed to clarify what their products really are. This article is so unsurprising to the point of being boring. Of course an LLM generates broken links. Why wouldn't it?

burnte 2y ago
If you asked the LLM it would say "those look like URLs to me!" And it'd be right!
- FourierEnvy 2y ago
  Well as Wolfram has stated, we need to mix LLMs with some deterministic reasoning. Real knowledge based output that can be routed to for these purposes. The hard part is determining when.
  - surfingdino 2y ago
    We need to go back to using the AI tech stack as a tool for (broadly speaking) automated pattern recognition, i.e. it can be great at image analysis and spotting early cancerous changes in lungs, but it ought to stop at pre-qualification stage and leave it up to the human to decide what the change is and what to do with it. What the AI crowd today want is all the photos in the world to be used to train models to recognise cancerous changes and then accept blindly the following diagnosis: "it is a cancerous change of human lung tissue, therefore the patient is a labrador with chickenpox and we recommend immediate removal of the patients left wing." When you tell them it's pure garbage, they come back with answers worthy of the slimiest orators in history and solutions that Rube Goldberg would not have thought of.
- airstrike 2y ago
  Doesn't look like anything to me
nerdjon 2y ago
I think OpenAI and LLM's in general are a victim of their user friendliness. Honestly, when was the last time that we got a new technology that was as user friendly as talking to an LLM for the average user? For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. So personally I think the problem is less the marketing (not to downplay that), and how the industry has gravited towards it. And the lack of proper education on what is not. We have to properly educate on what it is not actually doing while at the same time it sure appears to be doing exactly that since it is so good at faking it. This even exists within tech circles, we see some of it here.
- Jensson 2y ago
  > For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. Even many smart technical people here on HN fell for that.
  - andybak 2y ago
    It's even stranger than that. I can go from full awareness of the nature and limitations of LLMs one minute, the treating it like an almost sentient oracle the next. It's constant struggle to remain aware of what it really is under the covers because we're just so used to engaging with language creation entities as if they had minds
    - saalweachter 2y ago
      LLMs are fantastic tools for creating demos that are just absolutely magical, a glimpse of the future. It's only when you need to iterate on quality and go from "amazing" to "correct" that you start seeing how limited they are.
      - andybak 2y ago
        That's a false dichotomy. There's a huge range of applications that fall between "just a demo" and "must be reliably correct". They are just harder to productize for a broad market.
poikroequ 2y ago
An LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hallucinate fake URLs.
- darby_nine 2y ago
  LLMs aren't "trained to pick a url from a database of known urls". Or at least, we have not collectively seen this yet, perhaps I am wrong. We explicitly wire things so we can use LLMs to generate data useful to index with existing search databases.... acting like LLMs are responsible for the reliability of identifying the source document is disingenuous when a large part of the value of an LLM as an asset is its role laundering other peoples' intellectual property without attribution.
  - poikroequ 2y ago
    Try out phind.com for an example. The LLM generates a query for the search engine to fetch web results. It can include clickable links to the sources in its response. They accomplish this by training/fine-tuning the LLM to output special tokens, or special syntax, which is interpreted in code to perform some action, such as calling out to an API. This is how chatgpt and Gemini are able to automatically search the web and generate images and such. Yes, LLMs just output tokens, but those tokens can be interpreted to perform actions in code.
- SirMaster 2y ago
  Why can't it just check the URL to see if it's valid and actually contains the data it thinks it contains before recommending it to the user. Don't modern LLMs like GPT4 have access to load web pages? The downside is it will slow down the output I guess.
consumer451 2y ago
I am currently more worried about the clear and present threat of weaponized bullshit "AI" than I am about the theoretical threat of AGI. One would imagine that the FTC could require warning labels of "AI" products: "This product will confidently lie and cannot be trusted for factual information," etc. This warning would be especially useful for decision makers in commercial and government domains.
foobiekr 2y ago
The tech industry has long, long, long been a rapacious pump and dump scheme where the tech is barely relevant and most of the industry has been harvesting violations of social norms and regulations (tracking and ad tech, Uber, etc.). The only reason any of us defend it is that's how we make money. Honestly a lot of people in tech should be a little ashamed.
- nineteen999 2y ago
  It is this kind of stuff that makes me feel very warm and fuzzy that I work in a tech sector that literally helps saves lives (emergency services). I've worked in tech in banking, gambling, software development, internet service provision, and even the postal service. They were all extremely scummy compared to the sector I work in now.
  - reaperducer 2y ago
    Healthcare tech worker here. I know exactly how you feel. "What did I do today? Helped a bunch of doctors get the information they needed to diagnose someone's obscure disease. What did you do?" "Oh, I worked on ways to make my company's app more addictive to 12-year-olds, and look less like gambling." I'm happy to be in the first group.
- digging 2y ago
  > Honestly a lot of people in tech should be a little ashamed. I think it's true that a lot of people are a little ashamed already, but that a lot of people in tech should be more actively opposed to what's happening. As a small example, part of the paycheck I get from placing hidden F_cebook tracking on my corporate website to analyze the behaviors of people who could not possibly consent, I donate to the EFF. Actually it's quite difficult to be politically engaged against abusive tech in the same way I may or may not be engaged in local politics, but money is one thing we all have.
- 3pt14159 2y ago
  I am not ashamed of pumping OpenAI in general and ChatGPT in particular. The sheer amount of technological heft they've brought in such a short time is breathtaking. I've literally had an hour plus conversation, by voice, with ChatGPT and I was engaged throughout the entire affair. The tech isn't perfect when it comes to some things like links and what have you, but what does work is short of a miracle.
whateveracct 2y ago
Reminds me of how facial recognition software was hyped and sold to police departments.
fnordpiglet 2y ago
As you read the entire article you will note that OpenAI said they’ve not yet released the feature. The press release was about inking the deal not releasing the feature. They are still implementing the feature. This is mostly an article about how unreleased features in software that are in development still don’t yet work in production and proving that the unreleased features are still unreleased, then dressing it up as a scandal.
dontlikeyoueith 2y ago
> the tech community at large has failed to clarify what their products really are Most of the tech community is as clueless about ML as the general public.
deleted 2y ago
[deleted]

burnte 2y ago

If you asked the LLM it would say "those look like URLs to me!" And it'd be right!

FourierEnvy 2y ago
Well as Wolfram has stated, we need to mix LLMs with some deterministic reasoning. Real knowledge based output that can be routed to for these purposes. The hard part is determining when.
- surfingdino 2y ago
  We need to go back to using the AI tech stack as a tool for (broadly speaking) automated pattern recognition, i.e. it can be great at image analysis and spotting early cancerous changes in lungs, but it ought to stop at pre-qualification stage and leave it up to the human to decide what the change is and what to do with it. What the AI crowd today want is all the photos in the world to be used to train models to recognise cancerous changes and then accept blindly the following diagnosis: "it is a cancerous change of human lung tissue, therefore the patient is a labrador with chickenpox and we recommend immediate removal of the patients left wing." When you tell them it's pure garbage, they come back with answers worthy of the slimiest orators in history and solutions that Rube Goldberg would not have thought of.
airstrike 2y ago
Doesn't look like anything to me

nerdjon 2y ago

I think OpenAI and LLM's in general are a victim of their user friendliness. Honestly, when was the last time that we got a new technology that was as user friendly as talking to an LLM for the average user? For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. So personally I think the problem is less the marketing (not to downplay that), and how the industry has gravited towards it. And the lack of proper education on what is not. We have to properly educate on what it is not actually doing while at the same time it sure appears to be doing exactly that since it is so good at faking it. This even exists within tech circles, we see some of it here.

Jensson 2y ago
> For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. Even many smart technical people here on HN fell for that.
- andybak 2y ago
  It's even stranger than that. I can go from full awareness of the nature and limitations of LLMs one minute, the treating it like an almost sentient oracle the next. It's constant struggle to remain aware of what it really is under the covers because we're just so used to engaging with language creation entities as if they had minds
  - saalweachter 2y ago
    LLMs are fantastic tools for creating demos that are just absolutely magical, a glimpse of the future. It's only when you need to iterate on quality and go from "amazing" to "correct" that you start seeing how limited they are.
    - andybak 2y ago
      That's a false dichotomy. There's a huge range of applications that fall between "just a demo" and "must be reliably correct". They are just harder to productize for a broad market.

poikroequ 2y ago

An LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hallucinate fake URLs.

darby_nine 2y ago
LLMs aren't "trained to pick a url from a database of known urls". Or at least, we have not collectively seen this yet, perhaps I am wrong. We explicitly wire things so we can use LLMs to generate data useful to index with existing search databases.... acting like LLMs are responsible for the reliability of identifying the source document is disingenuous when a large part of the value of an LLM as an asset is its role laundering other peoples' intellectual property without attribution.
- poikroequ 2y ago
  Try out phind.com for an example. The LLM generates a query for the search engine to fetch web results. It can include clickable links to the sources in its response. They accomplish this by training/fine-tuning the LLM to output special tokens, or special syntax, which is interpreted in code to perform some action, such as calling out to an API. This is how chatgpt and Gemini are able to automatically search the web and generate images and such. Yes, LLMs just output tokens, but those tokens can be interpreted to perform actions in code.
SirMaster 2y ago
Why can't it just check the URL to see if it's valid and actually contains the data it thinks it contains before recommending it to the user. Don't modern LLMs like GPT4 have access to load web pages? The downside is it will slow down the output I guess.

consumer451 2y ago

I am currently more worried about the clear and present threat of weaponized bullshit "AI" than I am about the theoretical threat of AGI. One would imagine that the FTC could require warning labels of "AI" products: "This product will confidently lie and cannot be trusted for factual information," etc. This warning would be especially useful for decision makers in commercial and government domains.

foobiekr 2y ago

The tech industry has long, long, long been a rapacious pump and dump scheme where the tech is barely relevant and most of the industry has been harvesting violations of social norms and regulations (tracking and ad tech, Uber, etc.). The only reason any of us defend it is that's how we make money. Honestly a lot of people in tech should be a little ashamed.

nineteen999 2y ago
It is this kind of stuff that makes me feel very warm and fuzzy that I work in a tech sector that literally helps saves lives (emergency services). I've worked in tech in banking, gambling, software development, internet service provision, and even the postal service. They were all extremely scummy compared to the sector I work in now.
- reaperducer 2y ago
  Healthcare tech worker here. I know exactly how you feel. "What did I do today? Helped a bunch of doctors get the information they needed to diagnose someone's obscure disease. What did you do?" "Oh, I worked on ways to make my company's app more addictive to 12-year-olds, and look less like gambling." I'm happy to be in the first group.
digging 2y ago
> Honestly a lot of people in tech should be a little ashamed. I think it's true that a lot of people are a little ashamed already, but that a lot of people in tech should be more actively opposed to what's happening. As a small example, part of the paycheck I get from placing hidden F_cebook tracking on my corporate website to analyze the behaviors of people who could not possibly consent, I donate to the EFF. Actually it's quite difficult to be politically engaged against abusive tech in the same way I may or may not be engaged in local politics, but money is one thing we all have.
3pt14159 2y ago
I am not ashamed of pumping OpenAI in general and ChatGPT in particular. The sheer amount of technological heft they've brought in such a short time is breathtaking. I've literally had an hour plus conversation, by voice, with ChatGPT and I was engaged throughout the entire affair. The tech isn't perfect when it comes to some things like links and what have you, but what does work is short of a miracle.

whateveracct 2y ago

Reminds me of how facial recognition software was hyped and sold to police departments.

fnordpiglet 2y ago

As you read the entire article you will note that OpenAI said they’ve not yet released the feature. The press release was about inking the deal not releasing the feature. They are still implementing the feature. This is mostly an article about how unreleased features in software that are in development still don’t yet work in production and proving that the unreleased features are still unreleased, then dressing it up as a scandal.

dontlikeyoueith 2y ago

> the tech community at large has failed to clarify what their products really are Most of the tech community is as clueless about ML as the general public.

deleted 2y ago

[deleted]