5 ms·

This is terrifying. Even though they acknowledge the issues with hallucinations/errors, that is going to be completely overlooked by everyone using this, and t

by hi_hi 2y ago

This is terrifying. Even though they acknowledge the issues with hallucinations/errors, that is going to be completely overlooked by everyone using this, and then injecting the outputs into their own powerpoints. Management Consulting was bad enough before the ability to mass produce these graphs and stats on a whim. At least there was some understanding behind the scenes of where the numbers came from, and sources would/could be provided. The more powerful these tools become, the more prevelant this effect of seepage will become.

cyanydeez 2y ago
[flagged]
tmnvdb 2y ago
> At least there was some understanding behind the scenes of where the numbers came from, and sources would/could be provided. Oh Sweet summer child.
- opdahl 2y ago
  Hi tmnvdb, since you seem to love these super smart LLMs I thought it would be fun to have openais o3-mini-high analyze your recent comments in contrast to the Hacker News Comment Guidelines. Here is the output it gave me, hope it helps you: ------ Hey, I've noticed a few things in your style that are both strengths and opportunities for improvement: Strengths: - You clearly have deep knowledge and back up your points with solid data and examples. - Your confidence and detailed analysis make your arguments compelling. Opportunities: - At times, your tone can feel a bit combative, which might shut down conversation. - Focusing on critiquing ideas rather than questioning someone's honesty can help keep the discussion constructive. - A clearer structure in longer posts could make your points even more accessible. Overall, your passion and expertise shine through—tweaking the tone a bit might help foster even more productive debates. ------ Just reply here if you want the full 500+ words analysis that goes into more detail.
autoconfig 2y ago
Either you care about being correct or you don't. If you don't care then it doesn't matter whether you made it up or the AI did. If you care then you'll fact check before publishing. I don't see why this changes.
- spaceywilly 2y ago
  I think a lot about how differentiating facts and quality content is like differentiating signal from noise in electronics. The signal to noise ratio on many online platforms was already quite low. Tools like this will absolutely add more noise, and arguably the nature of the tools themselves make it harder to separate the noise. I think this is a real problem for these AI tools. If you can’t separate the signal from the noise, it doesn’t provide any real value, like an out of range FM radio station.
  - WOTERMEON 2y ago
    Not only that: by publishing noise, you’re lowering the signal/noise ratio.
- layer8 2y ago
  People are much less scrupulous using LLM output than making up stuff themselves, because then they can blame the LLM.
- hi_hi 2y ago
  Because maybe you want to, but you have a boss breathing down your neck and KPIs to meet and you haven't slept properly in days and just need a win, so you get the AI to put together some impressive looking graphs and stats that will look impressive in that client showcase thats due in a few hours. Things aren't quite so black and white in reality.
  - dauhak 2y ago
    I mean those same conditions already just lead the human to cutting corners and making stuff up themselves. You're describing the problem where bad incentives/conditions lead to sloppy work, that happens with or without AI Catching errors/validating work is obviously a different process when they're coming from an AI vs a human, but I don't see how it's fundamentally that different here. If the outputs are heavily cited then that might go someway into being able to more easily catch and correct slip-ups
scarab92 2y ago
Think of it like a vaccine. The majority of human written consultant reports are already complete rubbish. Low accuracy, low signal-to-noise, generic platitudes in a quantity-over-quality format. LLMs are innoculating people to this kind of low information value content. People who produce LLM quality output, are now being accused of using LLMs, and can no longer pretend to be adding value. The result of this is going to be higher quality expectations from consultants and a shaking out of people who produce word vommit rather than accurate, insightful, contextually relevent information.
- layer8 2y ago
  This has been downvoted, but I think there’s actually a chance it might become true (until AGI comes along at least).
- DrSiemer 2y ago
  Exactly what will happen with art. The tolerance for low quality output will decrease.
- randcraw 2y ago
  I don't think so. Instead of SEO, I think we'll soon see 'LLMO' dominating such uses, where LLM summaries are reshaped by vendors and etailers to misrepresent facts in ways that favor them over others. I suspect this can be done simply by poisoning a query with supplemental suggestions of sources to use in a RAG, many of which don't even have to be publicly available but are made accessible to the LLM (perhaps by submitting hidden URLs that mislead the summary along with the query). But even after such a practice is uncovered and roundly maligned, that won't stop the infinite supply of net con men from continuing to inject their poisons into the background that drives deep research, so long as the LLM maker doesn't actively oppose this practice actively and publicly -- which none of them have been willing to do with any other LLM operational details so far. In fact, I predict that if a LLM summary like DR's does NOT soon provide references to the sources of the facts it relies on, in no time users will disregard such summaries to be yet more uselessly unreliable pfaff from yet another net disreputable -- as we do with search engine summaries now.
_bin_ 2y ago
let's be real for a sec, i've done consulting and have a lot of friends who still do. three times in four, your mckinsey report isn't super well-founded in reality and involves a lot of guesstimation.
n144q 2y ago
I think that ship has sailed many years ago since Facebook allowed false information to spread freely on their site (if not earlier).
anthonyshort 2y ago
Then the hallucinated research is published in an article which is then cited by other AI research, continuing the push the false information until it’s hard to know where the lie started.

by hi_hi 2y ago

cyanydeez 2y ago

[flagged]

tmnvdb 2y ago

> At least there was some understanding behind the scenes of where the numbers came from, and sources would/could be provided. Oh Sweet summer child.

opdahl 2y ago
Hi tmnvdb, since you seem to love these super smart LLMs I thought it would be fun to have openais o3-mini-high analyze your recent comments in contrast to the Hacker News Comment Guidelines. Here is the output it gave me, hope it helps you: ------ Hey, I've noticed a few things in your style that are both strengths and opportunities for improvement: Strengths: - You clearly have deep knowledge and back up your points with solid data and examples. - Your confidence and detailed analysis make your arguments compelling. Opportunities: - At times, your tone can feel a bit combative, which might shut down conversation. - Focusing on critiquing ideas rather than questioning someone's honesty can help keep the discussion constructive. - A clearer structure in longer posts could make your points even more accessible. Overall, your passion and expertise shine through—tweaking the tone a bit might help foster even more productive debates. ------ Just reply here if you want the full 500+ words analysis that goes into more detail.

autoconfig 2y ago

Either you care about being correct or you don't. If you don't care then it doesn't matter whether you made it up or the AI did. If you care then you'll fact check before publishing. I don't see why this changes.

spaceywilly 2y ago
I think a lot about how differentiating facts and quality content is like differentiating signal from noise in electronics. The signal to noise ratio on many online platforms was already quite low. Tools like this will absolutely add more noise, and arguably the nature of the tools themselves make it harder to separate the noise. I think this is a real problem for these AI tools. If you can’t separate the signal from the noise, it doesn’t provide any real value, like an out of range FM radio station.
- WOTERMEON 2y ago
  Not only that: by publishing noise, you’re lowering the signal/noise ratio.
layer8 2y ago
People are much less scrupulous using LLM output than making up stuff themselves, because then they can blame the LLM.
hi_hi 2y ago
Because maybe you want to, but you have a boss breathing down your neck and KPIs to meet and you haven't slept properly in days and just need a win, so you get the AI to put together some impressive looking graphs and stats that will look impressive in that client showcase thats due in a few hours. Things aren't quite so black and white in reality.
- dauhak 2y ago
  I mean those same conditions already just lead the human to cutting corners and making stuff up themselves. You're describing the problem where bad incentives/conditions lead to sloppy work, that happens with or without AI Catching errors/validating work is obviously a different process when they're coming from an AI vs a human, but I don't see how it's fundamentally that different here. If the outputs are heavily cited then that might go someway into being able to more easily catch and correct slip-ups

scarab92 2y ago

Think of it like a vaccine. The majority of human written consultant reports are already complete rubbish. Low accuracy, low signal-to-noise, generic platitudes in a quantity-over-quality format. LLMs are innoculating people to this kind of low information value content. People who produce LLM quality output, are now being accused of using LLMs, and can no longer pretend to be adding value. The result of this is going to be higher quality expectations from consultants and a shaking out of people who produce word vommit rather than accurate, insightful, contextually relevent information.

layer8 2y ago
This has been downvoted, but I think there’s actually a chance it might become true (until AGI comes along at least).
DrSiemer 2y ago
Exactly what will happen with art. The tolerance for low quality output will decrease.
randcraw 2y ago
I don't think so. Instead of SEO, I think we'll soon see 'LLMO' dominating such uses, where LLM summaries are reshaped by vendors and etailers to misrepresent facts in ways that favor them over others. I suspect this can be done simply by poisoning a query with supplemental suggestions of sources to use in a RAG, many of which don't even have to be publicly available but are made accessible to the LLM (perhaps by submitting hidden URLs that mislead the summary along with the query). But even after such a practice is uncovered and roundly maligned, that won't stop the infinite supply of net con men from continuing to inject their poisons into the background that drives deep research, so long as the LLM maker doesn't actively oppose this practice actively and publicly -- which none of them have been willing to do with any other LLM operational details so far. In fact, I predict that if a LLM summary like DR's does NOT soon provide references to the sources of the facts it relies on, in no time users will disregard such summaries to be yet more uselessly unreliable pfaff from yet another net disreputable -- as we do with search engine summaries now.

_bin_ 2y ago

let's be real for a sec, i've done consulting and have a lot of friends who still do. three times in four, your mckinsey report isn't super well-founded in reality and involves a lot of guesstimation.

n144q 2y ago

I think that ship has sailed many years ago since Facebook allowed false information to spread freely on their site (if not earlier).

anthonyshort 2y ago

Then the hallucinated research is published in an article which is then cited by other AI research, continuing the push the false information until it’s hard to know where the lie started.