7 ms·

AI Hype is completely out of control – especially since ChatGPT-4o [video]

63 points by grugagag 2y ago

fnordpiglet 2y ago
I found the part about the voice and image stuff not being amazing weird - I thought they still haven’t released the new voice multimodal model to the apps. It seems to lose the point of the change going from a pipeline task delegated to a VTT model to the LLM to a TTV model into a new voice / text multi modal model that can encode nuance in the audio into semantic space and impact generation of tokens which themselves being in the voice domain directly can encode generative nuances in voice. This can include picking up on and generating inflection in the voice etc. Maybe it doesn’t address software engineering by AI, but IMO, that’s the absolutely least interesting use case for AI on earth despite being manically obsessed over by software engineers.
1vuio0pswjnm7 2y ago
tl;dw 1. He reminds viewers of deliberate attempts to take advantage of known human gullibility toward so-called "AI" ("Eliza Effect"), using dark patterns.^FN1 2. He cites various instances where so-called "tech" companies have lied about "AI" products and/or faked "AI" demos. 3. The creator of the video said he has included some sixty links to sources. He asserts that comments about "AI" that do not cite sources are unpersuasive. (That will not stop HN commenters from sharing their unsupported opinions.) FN1. He tells viewers that "cute" is a dark pattern, e.g., a giggling "AI" voice. Long have I wondered about all the silly non-descriptive or misleading names chosen for contemporary software, for so-called "tech" companies, as well as the silly graphics and mascots. What is their purpose. For example, is the reason for the silly company names chosen by so-called "tech" companies more than just avoiding trademark disputes. (Yes.) If software developers and so-called "tech" companies intentionally use these tactics to trick people into thinking or doing something that is against those peoples' interests, then arguably these could be dark patterns. Obvious example of a misleading name: "OpenAI" is not open, and that may have serious consequences, but it's unlikely the company will be changing its name any time soon. The video creator mentions that the SEC has stated it is cracking down on the use of "AI washing" where companies add "AI" to names or otherwise use terminology to trick people into believing "AI" is used when in fact it is not. In sum, the video is about the Silicon Valley "culture of lying". Perhaps ironic that the "AI" being pitched by SillyCon Valley today has no concept of a "lie". Despite the endless anthropomorphism, a computer running "AI" has no concepts whatsoever. Concepts come from people, not computers.
Closi 2y ago
> "What I know is that there is no evidence that we are going to get human level artificial intelligence any time soon" Disagree with this statement - it's too broad. There are plenty of tasks where AI's can fully outperform humans, and the number of tasks where AI's can fully outperform humans seems to increase every week. Then it depends what we mean by "human level" - the median human is different to the smartest person in the room.
- madaxe_again 2y ago
  The point I’ve seen raised that seems to escape many is that AI doesn’t need to be better than the best human to be profoundly disruptive - it only needs to be as good as a mediocre human carrying out a rote task to absolutely change the world.
  - gonzo41 2y ago
    ATM's killed bank tellers for the most part, people changed how they work in banks to move up the value chain. The same thing will happen. Also, expect a backlash to AI taking human service jobs. Like having to scan a QR code at a restaurant. Its a bad experience that shows you're enraptured by a technology and don't care about the customer. I don't disagree that AI will be disruptive. I just think it's going to find a place in technical fields and more people will want to remain talking and interacting with people like we've done for the last 100K years.
    - Turing_Machine 2y ago
      > ATM's killed bank tellers for the most part, Has it really? I haven't noticed any drastic decline over the years. Maybe some, but I wouldn't describe it as "killed". What universal ATMs and cards have really killed off is personal checks, IMO.
      - gonzo41 2y ago
        There was something like an order of magnitude delcine in the number of teller positions when banks rolled out ATM's. They've been around for so long that most people don't remember when they were uncommon. But ask yourself this, when did you last go into a bank. I've done a home loan application on a smart phone. Anyway, stuff is changing, There's always going to be jobs about. People are flexible and can learn things, AI ain't all that.
madaxe_again 2y ago
It’s ridiculous. Everyone is investing in Newcomen’s Engine for the Raising of Water by Fire - but it’s just a flash in the pan, and nothing can be accomplished by steam that can’t be accomplished by a good horse.
- GolfPopper 2y ago
  Nonsense! Just look at how successful tulip bulbs and cryptocurrency were!
- mcpar-land 2y ago
  instead, invest in our AH (Artificial Horse), which requires ingesting the behaviors of 100,000,000,000 real horses to approximate how they function! Ignore the mutant hooves and walking backwards, we can fix this if we just 10x the horse ingestion every year. It approximates a horse better every time!
ai4ever 2y ago
Everyone with cash to spare (CSPs and consumer-internet monopolies) and fomo will have their GPUs soon. Who is left ?
gmaster1440 2y ago
A lot of people appear to struggle with the likely possibility that there is both a lot of AI hype to go around and it's still a very disruptive technology that will improve a lot over relatively short timelines.
- deleted 2y ago
  [deleted]
- inciampati 2y ago
  I tend to agree with the caveat that we are experiencing exponentially diminishing marginal returns on investment (energy, FLOPS, currency). Yes it's going to get a lot better and smoother to work with. Agents will become secret and someday very soon no one will want to hype that they are using the tech. But we will need another revolution to get the kind of phase change we experienced with the introduction of LLMs. Imo the biggest thing that will lead to improvement now is open models that will get reused and remixed in creative ways. edit: to be clear, I'm saying that we are running into scaling "walls" that make hard extrapolation based on increased investment senseless.
  - kipchak 2y ago
    Gartner puts Generative AI at just about it's peak in terms of Hype back in August of 2023[1], with a 2-5 year time frame to go through the "trough of disillusionment" and then back to the "plateau of productivity", which seems pretty fair to me. [1]https://www.gartner.com/en/articles/what-s-new-in-the-2023-gartner-hype-cycle-for-emerging-technologies https://www.gartner.com/en/articles/what-s-new-in-the-2023-g...
    - red-iron-pine 2y ago
      The disruptive technology standard trend of: "the market would be huge" to "there is no market" to "there is a market, and it's growing"
- dimask 2y ago
  The problem is that there is little, incremental progress last 1 year or so after the big chatGPT boom to justify the hype, technically wise. Most of the "progress" going on is basically marketing, and making the models respond in ways humans like, or being more useful in certain practical applications. The basic, fundamental issues/limitations remain unanswered and unaddressed. As products, they have improved a lot and most probably are gonna improve more. But if we are talking for going towards AGI or more complex applications, I do not see evidence on that except as toys.
zombiwoof 2y ago
It’s because we don’t need Artificial we need Apple Intelligence
eurekin 2y ago
I'm using chatgpt daily, but cannot imagine it doing any real software development work for multiple reasons. It's great as a non offensive stack overflow replacement, but just look at aider benchmarks (amazing work by the way): most capable models really struggle to make basic changes to a real code base. Does anybody actually using it in practice believes the hype? I thought the hype is just another theatre for investors EDIT: just some points taken from: https://aider.chat/docs/leaderboards/#code-editing-leaderboard https://aider.chat/docs/leaderboards/#code-editing-leaderboa... - The metric "Percent completed correctly" maxes out at 72.9% with gpt4o, while at the same time, giving out correctly formatted output only 96.2% of the time. - Benchmark suite is based on https://github.com/exercism/python https://github.com/exercism/python, which very likely is a part of the training material already! In real code bases, no LLM would have the advantage of seeing new or proprietary code
- mvkel 2y ago
  Have built multiple MVP-scale node web apps, python scripts and two native iOS apps with it, not having known the languages beforehand. The only skill I brought to the table was app architecture. It's not just hype.
  - jslakro 2y ago
    Did you use chatgpt, copilot or any open source model?
    - mvkel 2y ago
      Vanilla ChatGPT, not even Cursor
  - eurekin 2y ago
    It's a great and very useful tool, but the hype around just feels significantly blown out of proportion. I also built multiple things with it and I always came to a point, where it just couldn't handle anything slightly larger than a mvp, or a non guided change requiring editing multiple files at once
deleted 2y ago
[deleted]
digitalsushi 2y ago
how could the hype be in control? we have a lot of people with a lot of investments that benefit greatly from hyping the products, an investment gold rush we simultaneously have an industry of people leveraging better jobs, job functions, job titles, by showing off their skills with this new technology making real things very quickly, and causing companies to have fomo and react without planning by overhiring and overpromoting and then finally we have a third class of people just talking about this new technology for the benefits of proximity. if there's a rumor george clooney is is crashing a party, half of people will make sure they're the loudest voice for the perceived benefits of that proximity. and even if it's not accurate, it's effective all of it's happening at the same time
- deleted 2y ago
  [deleted]
rafaelero 2y ago
Does this guy really believe that GPT-4's answers are of the same quality as GPT-3.5? That's ridiculous.
- Zambyte 2y ago
  It's pretty dependant on the questions you ask. Obviously when you push them to their limits, GPT 4 goes a lot further. But what about for your average queries though? I don't have the data to make a proper claim about that.
  - rafaelero 2y ago
    That's why we have Lmsys for. And spoiler alert, it performs better than 3.5 no matter the complexity of the question.
    - Zambyte 2y ago
      And are you sure people ask the chatbot arena the same caliber of questions they ask a single model? Wouldn't you tend to ask more difficult questions to find where one fails but the other doesn't? I'm not convinced that the average query will get a significantly better result from GPT 4. I only know that is true for more difficult queries.
      - rafaelero 2y ago
        Do you have access to ChatGPT? You can try it yourself (you can query GPT-4o and GPT-3.5 afterwards). The difference is large enough to be noticeable at least 80% of the time in my experience.
  - limfop 2y ago
    > It's pretty dependant on the questions you ask. That’s the description of a house of cards rather than some revolutionary new technology.
- ado__dev 2y ago
  I agree that GPT4 is better, but the amount of times it still hallucinates is still way too high to be useful for any real work. LLMs at present are super impressive, but have very very low trust.(This is true for all models, Claude, Opus, etc.)
righthand 2y ago
What is human intelligence? Are all humans, human intelligent? How do I know a machine has gotten there? Seems like a pretty poorly defined goal.
- Zambyte 2y ago
  Yeah, this is the question I always find myself asking after anything like this. Without establishing a definition of "intelligence", anything that follows is meaningless. And "intelligence" can be defined. People just think of different things when you use the word unless you define it. Here is how I like to define intelligence: the intersection between knowledge and reason. An example of a system that is highly knowledgeable but lacks intelligence due to its inability to reason is Wikipedia. A system that is highly reasonable but lacks intelligence due to its lack of knowledge is a calculator. It's obvious that language models encode more knowledge than humans do. Even a few GB model that I can download and run on my computer can spit out facts about random things. Way more than any human could. Sometimes they are wrong about facts, but so is any knowledge system, including humans. An LLMs reasoning capability is a bit less clear cut. When you have any understanding of how LLMs work, it seems like they should have no ability to do novel reasoning. Yet when you try to push them, they are often surprisingly good. I think that people who say the best LLMs have worse than human reasoning capabilities probably overestimate the average adults reasoning capability. Plus it is not hard at all to hook up a language model as a component in a greater intelligence system, by adding other components that specialize in reason. For example an environment to run code in.
  - righthand 2y ago
    I will say that LLM reasoning is good because it’s using information that has already been reasoned. Whether that reasoned information is fact or fiction is another case all together, so since LLMs are good at regurgitating information then they are good at regurgitating reasoned information as well. I would expect reasoned regurgitation to inform me at best; I wouldn’t expect something regurgitating to be able to necessarily be able to reason information it can’t regurgitate already.
    - 2y ago
ugjka 2y ago
I'm fine with everything that can run locally, but honestly any product that adds AI on server side will want some sort of subscription from you eventually because it is not cheap by any means yet... So we will have every app on phone or desktop begging for money pretty soon when the hype starts to shift on the wrong side
netman21 2y ago
Remember when Cliff Stoll wrote a whole book about how the internet was over-hyped, Silicon Snake Oil? In 1995. We are in the same stage with AI.
Turing_Machine 2y ago
Am I the only one who thinks that ChatGPT-4o is worse in pretty much every respect than previous versions, and that the previous versions themselves seem to have been dramatically dumbed-down? I'm on the verge of canceling my subscription altogether, as trying to get the damned thing to do what I ask is starting to take more time than just doing it by hand. If anyone from OpenAI is reading this: I (and I suspect many others) would rather pay more money than use a shitty model.
- jamil7 2y ago
  Yes, I cancelled my subscription a while ago, I can't tell if it's getting objectively worse or I just ran up against the limitations in using it for software development. I spent so much time correcting code it had written or getting it to rewrite code or getting concrete answers from it that I didn't feel like I was saving as much time as I initially thought after the novelty wore off. I will try Claude now it's available in my region. Overall it's still a very useful tool and I still use it but have stopped paying.
Workaccount2 2y ago
Ironically, and I'm sure this will draw ire if Google takes it out of beta, but I "watched" this video by asking the AI overview to summarize the video then asked it specific questions about viewpoints and opinions expressed in the video (I pay for premium and opted into the test beta of this last week). So rather than watching the video for 22 minutes, I felt I was able to get the gist of it in 1 minute...and denied the creator a full view. Was it accurate? I skimmed the video too in order to see, and it seemed correct, but I am aware of the inaccuracy these models tend to have. I don't think the hype is necessarily accurate in it's specific predictions, but I do think it is accurate in the "things are going to change dramatically" way.
osigurdson 2y ago
I don't know. I struggle to recall anything as transformational as the recent advancements in AI. Talking to a computer and getting mostly correct answers is literally Star Trek level. I suspect we are getting at least single digit increases in global GDP now due to LLMs.
- masswerk 2y ago
  I think, the key word here is "mostly". If it doesn't do better, the transformation is really that we have managed to engineer reliability and repeatability out of machines, which have been their defining characteristic in the past. So, will they do better? I personally think that knowledge is about implications. Notably, things that are similar do not necessarily have identical implications. And I can't see LLM tackling this issue anytime soon. (I even doubt that LLMs provide a suitable approach to this, at all.) So we'll probably end up with "we managed to get rid of reliability", which may be just the opposite of the role, the computer plays in Star Trek scripts. *) To provide a bit of context to reliability and repeatability: These are really externalization of the major generalizations of Husserl's life-world (Lebenswelt), as in "and so on" and "I can do this again". We are generally fine with machines, because they embody these principles in confined parameters. However, there's also a certain danger in this, as we tend to generalize and idealize along these lines, as this is essential to how we construct and navigate the world, we live in. We'll also apply this to things that are just "mostly", and are apt to be fooled by this, as may be the case with LLMs. (And in the past, machines were a great way out of this trap, as long – and as soon – as we were able to verify the parameters and boundaries.)
dwallin 2y ago
Generative AI is progressing at an extremely rapid rate. What many are experiencing though, is that every company wants to be perceived as being on the cutting edge. Being on the cutting edge gets you mindshare, free publicity, investment, stock price bumps, etc. This incentivizes all this bluster and false promises. If you just take a shallow look at everything, it would seem like an unending wave of this, and it's easy to get jaded. However right behind that is a not so slow march of people doing actual work, building non-trivial products, or using the technology to create tons of small, sometimes invisible, but cumulatively transformational, improvements to their products.
ai4ever 2y ago
* red alert * narrative shift triggered. needs to be combated with more hype. VC money is at stake. co-pilots are threatened. sora-hype is exhausted. 4o-hype is tired. call-to-action: "thought-leaders" like a16z to put out giant "architecture of ai" articles to keep the fomo alive.
zoogeny 2y ago
This weekend I caught up with some old friends. One of them works as a Registered Nurse (RN) and the other has been working in marketing for a small local clothing company for over a decade. The RN was describing a new AI service that her practice is using. It is a plugin to their tele-medicine software that listens to an ongoing call and uses speech-to-text plus whatever AI magic sauce the product has to fill out a "call sheet". I don't know precisely what a call sheet is, but the RN said it was the worst part of her job and she was very happy that it would automate the paper-work that takes up a large part of her day. The marketing manager laughed and said she used GPT multiple times a day. As in, she doesn't even send emails now without running the text through GPT. She then relayed an anecdote about how she had taken a picture of her fridge and asked GPT to identify the food available in there and to make a weekly meal plan for her family. These anecdotes were just brought up spontaneously. That is, the RN was just excited and musing about a new huge time-saver that was making her work-life better. And the marketing manager, who has already completely integrated GPT into her work, was showing how she was now integrating GPT into her family life. Even though I understand the "facts based reasoning" that is advocated in this video - my personal experience is seeing excitement not at hype, but excitement by non-tech people in the current uses of the technology. The trend I see in the people I am talking to is that AI, even the weak LLM version we are seeing now, is being integrated into work-life and personal-life at an astonishing pace.
- limfop 2y ago
  The model suggested adding glue to her meal plan then everyone clapped. https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022 https://www.theonion.com/guy-who-sucks-at-being-a-person-see...

63 points by grugagag 2y ago

I found the part about the voice and image stuff not being amazing weird - I thought they still haven’t released the new voice multimodal model to the apps. It seems to lose the point of the change going from a pipeline task delegated to a VTT model to the LLM to a TTV model into a new voice / text multi modal model that can encode nuance in the audio into semantic space and impact generation of tokens which themselves being in the voice domain directly can encode generative nuances in voice. This can include picking up on and generating inflection in the voice etc. Maybe it doesn’t address software engineering by AI, but IMO, that’s the absolutely least interesting use case for AI on earth despite being manically obsessed over by software engineers.

1vuio0pswjnm7 2y ago

tl;dw 1. He reminds viewers of deliberate attempts to take advantage of known human gullibility toward so-called "AI" ("Eliza Effect"), using dark patterns.^FN1 2. He cites various instances where so-called "tech" companies have lied about "AI" products and/or faked "AI" demos. 3. The creator of the video said he has included some sixty links to sources. He asserts that comments about "AI" that do not cite sources are unpersuasive. (That will not stop HN commenters from sharing their unsupported opinions.) FN1. He tells viewers that "cute" is a dark pattern, e.g., a giggling "AI" voice. Long have I wondered about all the silly non-descriptive or misleading names chosen for contemporary software, for so-called "tech" companies, as well as the silly graphics and mascots. What is their purpose. For example, is the reason for the silly company names chosen by so-called "tech" companies more than just avoiding trademark disputes. (Yes.) If software developers and so-called "tech" companies intentionally use these tactics to trick people into thinking or doing something that is against those peoples' interests, then arguably these could be dark patterns. Obvious example of a misleading name: "OpenAI" is not open, and that may have serious consequences, but it's unlikely the company will be changing its name any time soon. The video creator mentions that the SEC has stated it is cracking down on the use of "AI washing" where companies add "AI" to names or otherwise use terminology to trick people into believing "AI" is used when in fact it is not. In sum, the video is about the Silicon Valley "culture of lying". Perhaps ironic that the "AI" being pitched by SillyCon Valley today has no concept of a "lie". Despite the endless anthropomorphism, a computer running "AI" has no concepts whatsoever. Concepts come from people, not computers.

Closi 2y ago

> "What I know is that there is no evidence that we are going to get human level artificial intelligence any time soon" Disagree with this statement - it's too broad. There are plenty of tasks where AI's can fully outperform humans, and the number of tasks where AI's can fully outperform humans seems to increase every week. Then it depends what we mean by "human level" - the median human is different to the smartest person in the room.

madaxe_again 2y ago
The point I’ve seen raised that seems to escape many is that AI doesn’t need to be better than the best human to be profoundly disruptive - it only needs to be as good as a mediocre human carrying out a rote task to absolutely change the world.
- gonzo41 2y ago
  ATM's killed bank tellers for the most part, people changed how they work in banks to move up the value chain. The same thing will happen. Also, expect a backlash to AI taking human service jobs. Like having to scan a QR code at a restaurant. Its a bad experience that shows you're enraptured by a technology and don't care about the customer. I don't disagree that AI will be disruptive. I just think it's going to find a place in technical fields and more people will want to remain talking and interacting with people like we've done for the last 100K years.
  - Turing_Machine 2y ago
    > ATM's killed bank tellers for the most part, Has it really? I haven't noticed any drastic decline over the years. Maybe some, but I wouldn't describe it as "killed". What universal ATMs and cards have really killed off is personal checks, IMO.
    - gonzo41 2y ago
      There was something like an order of magnitude delcine in the number of teller positions when banks rolled out ATM's. They've been around for so long that most people don't remember when they were uncommon. But ask yourself this, when did you last go into a bank. I've done a home loan application on a smart phone. Anyway, stuff is changing, There's always going to be jobs about. People are flexible and can learn things, AI ain't all that.

madaxe_again 2y ago

It’s ridiculous. Everyone is investing in Newcomen’s Engine for the Raising of Water by Fire - but it’s just a flash in the pan, and nothing can be accomplished by steam that can’t be accomplished by a good horse.

GolfPopper 2y ago
Nonsense! Just look at how successful tulip bulbs and cryptocurrency were!
mcpar-land 2y ago
instead, invest in our AH (Artificial Horse), which requires ingesting the behaviors of 100,000,000,000 real horses to approximate how they function! Ignore the mutant hooves and walking backwards, we can fix this if we just 10x the horse ingestion every year. It approximates a horse better every time!

ai4ever 2y ago

Everyone with cash to spare (CSPs and consumer-internet monopolies) and fomo will have their GPUs soon. Who is left ?

gmaster1440 2y ago

A lot of people appear to struggle with the likely possibility that there is both a lot of AI hype to go around and it's still a very disruptive technology that will improve a lot over relatively short timelines.

deleted 2y ago
[deleted]
inciampati 2y ago
I tend to agree with the caveat that we are experiencing exponentially diminishing marginal returns on investment (energy, FLOPS, currency). Yes it's going to get a lot better and smoother to work with. Agents will become secret and someday very soon no one will want to hype that they are using the tech. But we will need another revolution to get the kind of phase change we experienced with the introduction of LLMs. Imo the biggest thing that will lead to improvement now is open models that will get reused and remixed in creative ways. edit: to be clear, I'm saying that we are running into scaling "walls" that make hard extrapolation based on increased investment senseless.
- kipchak 2y ago
  Gartner puts Generative AI at just about it's peak in terms of Hype back in August of 2023[1], with a 2-5 year time frame to go through the "trough of disillusionment" and then back to the "plateau of productivity", which seems pretty fair to me. [1]https://www.gartner.com/en/articles/what-s-new-in-the-2023-gartner-hype-cycle-for-emerging-technologies https://www.gartner.com/en/articles/what-s-new-in-the-2023-g...
  - red-iron-pine 2y ago
    The disruptive technology standard trend of: "the market would be huge" to "there is no market" to "there is a market, and it's growing"
dimask 2y ago
The problem is that there is little, incremental progress last 1 year or so after the big chatGPT boom to justify the hype, technically wise. Most of the "progress" going on is basically marketing, and making the models respond in ways humans like, or being more useful in certain practical applications. The basic, fundamental issues/limitations remain unanswered and unaddressed. As products, they have improved a lot and most probably are gonna improve more. But if we are talking for going towards AGI or more complex applications, I do not see evidence on that except as toys.

zombiwoof 2y ago

It’s because we don’t need Artificial we need Apple Intelligence

eurekin 2y ago

I'm using chatgpt daily, but cannot imagine it doing any real software development work for multiple reasons. It's great as a non offensive stack overflow replacement, but just look at aider benchmarks (amazing work by the way): most capable models really struggle to make basic changes to a real code base. Does anybody actually using it in practice believes the hype? I thought the hype is just another theatre for investors EDIT: just some points taken from: https://aider.chat/docs/leaderboards/#code-editing-leaderboard https://aider.chat/docs/leaderboards/#code-editing-leaderboa... - The metric "Percent completed correctly" maxes out at 72.9% with gpt4o, while at the same time, giving out correctly formatted output only 96.2% of the time. - Benchmark suite is based on https://github.com/exercism/python https://github.com/exercism/python, which very likely is a part of the training material already! In real code bases, no LLM would have the advantage of seeing new or proprietary code

mvkel 2y ago
Have built multiple MVP-scale node web apps, python scripts and two native iOS apps with it, not having known the languages beforehand. The only skill I brought to the table was app architecture. It's not just hype.
- jslakro 2y ago
  Did you use chatgpt, copilot or any open source model?
  - mvkel 2y ago
    Vanilla ChatGPT, not even Cursor
- eurekin 2y ago
  It's a great and very useful tool, but the hype around just feels significantly blown out of proportion. I also built multiple things with it and I always came to a point, where it just couldn't handle anything slightly larger than a mvp, or a non guided change requiring editing multiple files at once

deleted 2y ago

[deleted]

digitalsushi 2y ago

how could the hype be in control? we have a lot of people with a lot of investments that benefit greatly from hyping the products, an investment gold rush we simultaneously have an industry of people leveraging better jobs, job functions, job titles, by showing off their skills with this new technology making real things very quickly, and causing companies to have fomo and react without planning by overhiring and overpromoting and then finally we have a third class of people just talking about this new technology for the benefits of proximity. if there's a rumor george clooney is is crashing a party, half of people will make sure they're the loudest voice for the perceived benefits of that proximity. and even if it's not accurate, it's effective all of it's happening at the same time

deleted 2y ago
[deleted]

rafaelero 2y ago

Does this guy really believe that GPT-4's answers are of the same quality as GPT-3.5? That's ridiculous.

Zambyte 2y ago
It's pretty dependant on the questions you ask. Obviously when you push them to their limits, GPT 4 goes a lot further. But what about for your average queries though? I don't have the data to make a proper claim about that.
- rafaelero 2y ago
  That's why we have Lmsys for. And spoiler alert, it performs better than 3.5 no matter the complexity of the question.
  - Zambyte 2y ago
    And are you sure people ask the chatbot arena the same caliber of questions they ask a single model? Wouldn't you tend to ask more difficult questions to find where one fails but the other doesn't? I'm not convinced that the average query will get a significantly better result from GPT 4. I only know that is true for more difficult queries.
    - rafaelero 2y ago
      Do you have access to ChatGPT? You can try it yourself (you can query GPT-4o and GPT-3.5 afterwards). The difference is large enough to be noticeable at least 80% of the time in my experience.
- limfop 2y ago
  > It's pretty dependant on the questions you ask. That’s the description of a house of cards rather than some revolutionary new technology.
ado__dev 2y ago
I agree that GPT4 is better, but the amount of times it still hallucinates is still way too high to be useful for any real work. LLMs at present are super impressive, but have very very low trust.(This is true for all models, Claude, Opus, etc.)

righthand 2y ago

What is human intelligence? Are all humans, human intelligent? How do I know a machine has gotten there? Seems like a pretty poorly defined goal.

Zambyte 2y ago
Yeah, this is the question I always find myself asking after anything like this. Without establishing a definition of "intelligence", anything that follows is meaningless. And "intelligence" can be defined. People just think of different things when you use the word unless you define it. Here is how I like to define intelligence: the intersection between knowledge and reason. An example of a system that is highly knowledgeable but lacks intelligence due to its inability to reason is Wikipedia. A system that is highly reasonable but lacks intelligence due to its lack of knowledge is a calculator. It's obvious that language models encode more knowledge than humans do. Even a few GB model that I can download and run on my computer can spit out facts about random things. Way more than any human could. Sometimes they are wrong about facts, but so is any knowledge system, including humans. An LLMs reasoning capability is a bit less clear cut. When you have any understanding of how LLMs work, it seems like they should have no ability to do novel reasoning. Yet when you try to push them, they are often surprisingly good. I think that people who say the best LLMs have worse than human reasoning capabilities probably overestimate the average adults reasoning capability. Plus it is not hard at all to hook up a language model as a component in a greater intelligence system, by adding other components that specialize in reason. For example an environment to run code in.
- righthand 2y ago
  I will say that LLM reasoning is good because it’s using information that has already been reasoned. Whether that reasoned information is fact or fiction is another case all together, so since LLMs are good at regurgitating information then they are good at regurgitating reasoned information as well. I would expect reasoned regurgitation to inform me at best; I wouldn’t expect something regurgitating to be able to necessarily be able to reason information it can’t regurgitate already.
  - 2y ago

ugjka 2y ago

I'm fine with everything that can run locally, but honestly any product that adds AI on server side will want some sort of subscription from you eventually because it is not cheap by any means yet... So we will have every app on phone or desktop begging for money pretty soon when the hype starts to shift on the wrong side

netman21 2y ago

Remember when Cliff Stoll wrote a whole book about how the internet was over-hyped, Silicon Snake Oil? In 1995. We are in the same stage with AI.

Turing_Machine 2y ago

Am I the only one who thinks that ChatGPT-4o is worse in pretty much every respect than previous versions, and that the previous versions themselves seem to have been dramatically dumbed-down? I'm on the verge of canceling my subscription altogether, as trying to get the damned thing to do what I ask is starting to take more time than just doing it by hand. If anyone from OpenAI is reading this: I (and I suspect many others) would rather pay more money than use a shitty model.

jamil7 2y ago
Yes, I cancelled my subscription a while ago, I can't tell if it's getting objectively worse or I just ran up against the limitations in using it for software development. I spent so much time correcting code it had written or getting it to rewrite code or getting concrete answers from it that I didn't feel like I was saving as much time as I initially thought after the novelty wore off. I will try Claude now it's available in my region. Overall it's still a very useful tool and I still use it but have stopped paying.

Workaccount2 2y ago

Ironically, and I'm sure this will draw ire if Google takes it out of beta, but I "watched" this video by asking the AI overview to summarize the video then asked it specific questions about viewpoints and opinions expressed in the video (I pay for premium and opted into the test beta of this last week). So rather than watching the video for 22 minutes, I felt I was able to get the gist of it in 1 minute...and denied the creator a full view. Was it accurate? I skimmed the video too in order to see, and it seemed correct, but I am aware of the inaccuracy these models tend to have. I don't think the hype is necessarily accurate in it's specific predictions, but I do think it is accurate in the "things are going to change dramatically" way.

osigurdson 2y ago

I don't know. I struggle to recall anything as transformational as the recent advancements in AI. Talking to a computer and getting mostly correct answers is literally Star Trek level. I suspect we are getting at least single digit increases in global GDP now due to LLMs.

masswerk 2y ago
I think, the key word here is "mostly". If it doesn't do better, the transformation is really that we have managed to engineer reliability and repeatability out of machines, which have been their defining characteristic in the past. So, will they do better? I personally think that knowledge is about implications. Notably, things that are similar do not necessarily have identical implications. And I can't see LLM tackling this issue anytime soon. (I even doubt that LLMs provide a suitable approach to this, at all.) So we'll probably end up with "we managed to get rid of reliability", which may be just the opposite of the role, the computer plays in Star Trek scripts. *) To provide a bit of context to reliability and repeatability: These are really externalization of the major generalizations of Husserl's life-world (Lebenswelt), as in "and so on" and "I can do this again". We are generally fine with machines, because they embody these principles in confined parameters. However, there's also a certain danger in this, as we tend to generalize and idealize along these lines, as this is essential to how we construct and navigate the world, we live in. We'll also apply this to things that are just "mostly", and are apt to be fooled by this, as may be the case with LLMs. (And in the past, machines were a great way out of this trap, as long – and as soon – as we were able to verify the parameters and boundaries.)

dwallin 2y ago

Generative AI is progressing at an extremely rapid rate. What many are experiencing though, is that every company wants to be perceived as being on the cutting edge. Being on the cutting edge gets you mindshare, free publicity, investment, stock price bumps, etc. This incentivizes all this bluster and false promises. If you just take a shallow look at everything, it would seem like an unending wave of this, and it's easy to get jaded. However right behind that is a not so slow march of people doing actual work, building non-trivial products, or using the technology to create tons of small, sometimes invisible, but cumulatively transformational, improvements to their products.

ai4ever 2y ago

* red alert * narrative shift triggered. needs to be combated with more hype. VC money is at stake. co-pilots are threatened. sora-hype is exhausted. 4o-hype is tired. call-to-action: "thought-leaders" like a16z to put out giant "architecture of ai" articles to keep the fomo alive.

zoogeny 2y ago

This weekend I caught up with some old friends. One of them works as a Registered Nurse (RN) and the other has been working in marketing for a small local clothing company for over a decade. The RN was describing a new AI service that her practice is using. It is a plugin to their tele-medicine software that listens to an ongoing call and uses speech-to-text plus whatever AI magic sauce the product has to fill out a "call sheet". I don't know precisely what a call sheet is, but the RN said it was the worst part of her job and she was very happy that it would automate the paper-work that takes up a large part of her day. The marketing manager laughed and said she used GPT multiple times a day. As in, she doesn't even send emails now without running the text through GPT. She then relayed an anecdote about how she had taken a picture of her fridge and asked GPT to identify the food available in there and to make a weekly meal plan for her family. These anecdotes were just brought up spontaneously. That is, the RN was just excited and musing about a new huge time-saver that was making her work-life better. And the marketing manager, who has already completely integrated GPT into her work, was showing how she was now integrating GPT into her family life. Even though I understand the "facts based reasoning" that is advocated in this video - my personal experience is seeing excitement not at hype, but excitement by non-tech people in the current uses of the technology. The trend I see in the people I am talking to is that AI, even the weak LLM version we are seeing now, is being integrated into work-life and personal-life at an astonishing pace.

limfop 2y ago
The model suggested adding glue to her meal plan then everyone clapped. https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022 https://www.theonion.com/guy-who-sucks-at-being-a-person-see...