4 ms·

pretty interesting but with how much current llms get wrong or hallucinate i’d be pretty wary of trusting the output, at least currently. amazing to think of w

pretty interesting but with how much current llms get wrong or hallucinate i’d be pretty wary of trusting the output, at least currently. amazing to think of where this could be in 10 or 20 years.

brookst 3y ago
You’re saying this brand new experimental technology may be imperfect?
- ShamelessC 3y ago
  Yeah...The research here doesn't even make the claim that it has no hallucinations. It seems to largely be exciting _despite_ hallucinations because it clearly does occasionally guess the correct words. They mention lots of issues but so long as it passes peer review, seems like a massive step forward. https://arxiv.org/pdf/2309.14030v2.pdf https://arxiv.org/pdf/2309.14030v2.pdf
- dang 3y ago
  I completely understand the reflex against shallow dismissal of groundbreaking work, but please don't respond by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
admax88qqq 3y ago
Combine hallucinations with police adopting this as the new polygraph and this could take a pretty bad turn. Cool tech though, lots of positive applications too.
- codedokode 3y ago
  Why only police? Install mind-readers at every home.
- Sanzig 3y ago
  If noninvasive mind reading ever becomes practical, we need to recognize the right to refuse a brain scan a universal human right. Additionally, it should be banned from being used for evidence in the courtroom. Unfortunately there will be authoritarian regimes that will use and abuse this type of tech, but we need to take a firm stand against it in liberal democracies at the very least.
  - jprete 3y ago
    That's not sufficient - it needs to be actually banned for any uses resembling employment purposes, because otherwise people will be easily pressured into it by the incentives of businesses who want their employees to be worker bees. Just look at how many businesses try to force people to waive their right to a trial by law as a condition of being a customer!
    - mentos 3y ago
      I have a feeling that by the time this is fully fleshed out AI will have taken all the jobs anyways.
    - Sanzig 3y ago
      Agreed.
  - swayvil 3y ago
    What if it went the opposite way? What if perfect brainreaders/liedetectors became as common as smartphones. Used on everybody all the time. From politicians and cops to schoolkids and your own siblings. What would be an optimistic version of that?
    - Sanzig 3y ago
      I don't think there is one, not for a version of humanity that is even remotely recognizable at least. We are not ready to hear each other's internal monologues. Most people have intrusive thoughts, some people (like those with OCD, for example) have really frequent and distressing intrusive thoughts. What are you going to think of the OCD sufferer in the cubicle next to you who keeps inadvertently broadcasting intrusive thoughts about violently stabbing you to death? Keep in mind, they will never act on those thoughts, they are simply the result of some faulty brain wiring and they are even more disgusted about them than you are. What are you going to think when you find out your sister-in-law had an affair with a coworker ten years ago, because her mind wandered there while you were having coffee with her? Humanity does not even come close to having the level of understanding and compassion needed to prevent total chaos in a world like that. People naïvely believed that edgy or embarrassing social media posts made by millenials in the late 2000s wouldn't be a big deal, because we'd all figure out that everyone is imperfect and the person you were 10 years ago is not the person you are today. Nope, if anything the opposite has happened: it's now a widely accepted practice to go on a fishing expedition through someone's social media history to find something compromising to shame them with. Now imagine that, but applied to mind reading. No, that's not a future that we can survive as a species, at least not without radical changes in our approaches to dealing with each other.
- spookybones 3y ago
  I wonder if a subject has to train it first, such as by reading a bunch of prompts while trying to imagine them. Or, are our linguistic neural networks all very similar? If the former is true, it would at least be a bit harder to work as a polygraph. You wouldn't be able to just strap on the helmet and read someone's thoughts accurately.
  - dexwiz 3y ago
    I wonder if you could develop techniques to combat it, like a psychic nail in the shoe. Or maybe an actual nail. How useful is a mind reader when all it reads is “PAIN!”
  - RaftPeople 3y ago
    Yes it requires training for each individual. In addition, they tested using a trained model from one person to try to decode a different person and the results were no better than chance. They also said that the person must cooperate for the decoding to work, meaning the person could reduce the decoding accuracy by thinking of specific things (e.g. counting). CORRECTION: The paper I read was not the correct paper, ignore this comment. The actual paper states that the model is transferrable across subjects.
  - admax88qqq 3y ago
    > and read someone's thoughts *accurately* Sadly accuracy has not been a big concern historically of polygraphs and much of forensic "science".
- andy99 3y ago
  Police is far down the list or realistic concerns. - insurance discount if you wear this while driving - remote work offered as a "perk" as long as you wear it - the "alladvantage ecg helmet" that pays you to wear it around while you're shown advertising - to augment one of those video interviews where you have to answer questions and a computer screens your behavior That's all stuff that already exists more or less and much more likely to be the form that the abuse of this technology takes
  - popcalc 3y ago
    Eventually it will become affordable for parents, evangelical churches, and spouses.

pretty interesting but with how much current llms get wrong or hallucinate i’d be pretty wary of trusting the output, at least currently. amazing to think of where this could be in 10 or 20 years.

brookst 3y ago

You’re saying this brand new experimental technology may be imperfect?

ShamelessC 3y ago
Yeah...The research here doesn't even make the claim that it has no hallucinations. It seems to largely be exciting _despite_ hallucinations because it clearly does occasionally guess the correct words. They mention lots of issues but so long as it passes peer review, seems like a massive step forward. https://arxiv.org/pdf/2309.14030v2.pdf https://arxiv.org/pdf/2309.14030v2.pdf
dang 3y ago
I completely understand the reflex against shallow dismissal of groundbreaking work, but please don't respond by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

admax88qqq 3y ago

Combine hallucinations with police adopting this as the new polygraph and this could take a pretty bad turn. Cool tech though, lots of positive applications too.

codedokode 3y ago
Why only police? Install mind-readers at every home.
Sanzig 3y ago
If noninvasive mind reading ever becomes practical, we need to recognize the right to refuse a brain scan a universal human right. Additionally, it should be banned from being used for evidence in the courtroom. Unfortunately there will be authoritarian regimes that will use and abuse this type of tech, but we need to take a firm stand against it in liberal democracies at the very least.
- jprete 3y ago
  That's not sufficient - it needs to be actually banned for any uses resembling employment purposes, because otherwise people will be easily pressured into it by the incentives of businesses who want their employees to be worker bees. Just look at how many businesses try to force people to waive their right to a trial by law as a condition of being a customer!
  - mentos 3y ago
    I have a feeling that by the time this is fully fleshed out AI will have taken all the jobs anyways.
  - Sanzig 3y ago
    Agreed.
- swayvil 3y ago
  What if it went the opposite way? What if perfect brainreaders/liedetectors became as common as smartphones. Used on everybody all the time. From politicians and cops to schoolkids and your own siblings. What would be an optimistic version of that?
  - Sanzig 3y ago
    I don't think there is one, not for a version of humanity that is even remotely recognizable at least. We are not ready to hear each other's internal monologues. Most people have intrusive thoughts, some people (like those with OCD, for example) have really frequent and distressing intrusive thoughts. What are you going to think of the OCD sufferer in the cubicle next to you who keeps inadvertently broadcasting intrusive thoughts about violently stabbing you to death? Keep in mind, they will never act on those thoughts, they are simply the result of some faulty brain wiring and they are even more disgusted about them than you are. What are you going to think when you find out your sister-in-law had an affair with a coworker ten years ago, because her mind wandered there while you were having coffee with her? Humanity does not even come close to having the level of understanding and compassion needed to prevent total chaos in a world like that. People naïvely believed that edgy or embarrassing social media posts made by millenials in the late 2000s wouldn't be a big deal, because we'd all figure out that everyone is imperfect and the person you were 10 years ago is not the person you are today. Nope, if anything the opposite has happened: it's now a widely accepted practice to go on a fishing expedition through someone's social media history to find something compromising to shame them with. Now imagine that, but applied to mind reading. No, that's not a future that we can survive as a species, at least not without radical changes in our approaches to dealing with each other.
spookybones 3y ago
I wonder if a subject has to train it first, such as by reading a bunch of prompts while trying to imagine them. Or, are our linguistic neural networks all very similar? If the former is true, it would at least be a bit harder to work as a polygraph. You wouldn't be able to just strap on the helmet and read someone's thoughts accurately.
- dexwiz 3y ago
  I wonder if you could develop techniques to combat it, like a psychic nail in the shoe. Or maybe an actual nail. How useful is a mind reader when all it reads is “PAIN!”
- RaftPeople 3y ago
  Yes it requires training for each individual. In addition, they tested using a trained model from one person to try to decode a different person and the results were no better than chance. They also said that the person must cooperate for the decoding to work, meaning the person could reduce the decoding accuracy by thinking of specific things (e.g. counting). CORRECTION: The paper I read was not the correct paper, ignore this comment. The actual paper states that the model is transferrable across subjects.
- admax88qqq 3y ago
  > and read someone's thoughts *accurately* Sadly accuracy has not been a big concern historically of polygraphs and much of forensic "science".
andy99 3y ago
Police is far down the list or realistic concerns. - insurance discount if you wear this while driving - remote work offered as a "perk" as long as you wear it - the "alladvantage ecg helmet" that pays you to wear it around while you're shown advertising - to augment one of those video interviews where you have to answer questions and a computer screens your behavior That's all stuff that already exists more or less and much more likely to be the form that the abuse of this technology takes
- popcalc 3y ago
  Eventually it will become affordable for parents, evangelical churches, and spouses.