5 ms·

It's always interesting to see how people engage with new technologies on HN. For example, here you note 3 things that LLMs are known for, but not all the thing

It's always interesting to see how people engage with new technologies on HN. For example, here you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them. Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions. Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, but not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer. Just interesting to see.

sorokod 3y ago
Perhaps an equivalent of "risk aversion", negative implications outweigh the positive even if the count of either is the same.
courseofaction 3y ago
The bias will be by design and known to the purchaser.
deleted 3y ago
[deleted]
sottol 3y ago
But is that really what you want to happen IN GOVERNMENT? For bureaucracy to adapt even mildly flawed technology?! Bureaucracy + blind trust in hierarchy/computers = Kafka-esque hellscape. "I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$. [1] https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/ https://www.washingtonpost.com/technology/2023/04/05/chatgpt...
- okennedy 3y ago
  Things are bad enough already, even with pre-LLM technology. "I'm sorry sir, your 8-year old son can't board this plane because the system says he's a terrorist" [1]. [1] https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html
  - adventured 3y ago
    You're using a straw example to try to claim a massive point. There are 340 million people in the US. The US has one of the largest government systems that humanity is likely to ever see. How many similar cases of children being put on the terrorist watch list have there been in the past 20 years? Unless there are a lot of cases of that happening, if it's not rare (hint: it's extraordinarily rare; your example story is from 13 years ago), your premise is pure straw.
    - jhncls 3y ago
      Wouldn't being innocently jailed for 9 days because of a wrong face recognition count as another example? [0]: https://www.wired.com/story/face-recognition-software-led-to-his-arrest-it-was-dead-wrong/ https://www.wired.com/story/face-recognition-software-led-to... [1]: https://www.theguardian.com/us-news/2023/apr/27/california-police-facial-recognition-software https://www.theguardian.com/us-news/2023/apr/27/california-p...
      - CDT-MLT 3y ago
        Jesus Christ
      - replygirl 3y ago
        An example against adoption of larger, more generalized, and less reliable models, yes
  - deleted 3y ago
    [deleted]
- 3y ago
parker_mountain 3y ago
> you conveniently leave out the positive benefits of these models that are causing people to promote them We are holding it to a higher standard because this is going to be used by agencies with serious power over people's lives. It is not just to spit out, for example, a form rejection letter from a credit card. And, nobody thinks that Microsoft wants to "screw" the government. But if something happens, the impact can be magnified significantly considering the reach and power of the state. I'm all for moving forward quickly with this tech, but there are a handful of spaces (government, medicine, aeronautics) where we need to be extremely deliberate.
- williamcotton 3y ago
  [flagged]
  - birdyrooster 3y ago
    We being the people upvoting the GP comment. Lol you really read into that.
jchw 3y ago
Not really that interesting, it's pretty much always what people do when the potential upside of something is limited but the potential disastrous consequences are basically limitless.
InCityDreams 3y ago
Well, if government*s can't even use MS excel correct, this is going to be soooo much fun. I feel for my kids, and all I can do is warn them... https://theconversation.com/excel-errors-the-uk-government-has-an-embarrassingly-long-history-of-spreadsheet-horror-stories-147606 https://theconversation.com/excel-errors-the-uk-government-h...
Solvency 3y ago
GPT is an incredible productivity tool for people who are informed about it and understand its limitations and use it wisely and responsibly. The vast majority of government workers are underpaid apathetic cogs in a machine who do not fit this bill. We just saw a LAWYER use hallucinated information in his practice. A lawyer who is ostensibly rigorously trained and tested in his field. The amount of misuse of GPT by way of sheer ignorance alone by the government will be astounding.
burnished 3y ago
Theres already precedent with police and judiciary misusing technology they don't understand, I suspect the underlying issue is that this person would not trust government employees generally with a tool that issues false positives and requires discretions.
romwell 3y ago
>you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them. When talking about the pitfalls of bringing GPT-4 as a tool for government officials, the positive features are outside of the scope. Obviously they exist, otherwise it wouldn't be proposed. But it's not our job to do a sales pitch for GPT-4. >Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions. No. The OP suggests that the primary users of the tool would be low-level, underpaid officials, who are mostly following directions. Not because it's their fault as human beings - but because their positions require them to do so, and perhaps for good reasons. In any case, direction-followers (of questionable competence and low pay grade) make up the bulk of large organizations in general as a consequence of growth. Which is why the tool is being pitched in the first place: to streamline work of people who would rather not do it in the first place. >Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, b As if the governments aren't prone to massive scandals even without GPT-4 in place. Try checking the news today, you may be surprised. >not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer. "By Microsoft" wasn't in OP's statement. Manipulating data and statistics to get the desired outcome from seemingly impartial algorithms is something our government has been engaging in for a very, very long time. Redlining[1] and gerrymandering[2] are just two prime examples. We don't have to make any assumptions or apply biased perspectives to say that we expect: 1)Training data to be manipulated by entities with interest in influencing the outcome; 2)Lives of countless people being adversely affected, and 3)There being a massive scandal as a result. The government has a long track history of 1, 2, and 3 happening any time a system is introduced which lacks transparency, and large language models are the epitome of that. The only unrealistic thing about OP's statement is that a scandal will actually happen, instead of people getting away with it. >It's always interesting to see how people engage with new technologies on HN. And it is even more interesting to see how people here engage with ethics. [1]https://www.nytimes.com/2021/08/17/realestate/what-is-redlining.html https://www.nytimes.com/2021/08/17/realestate/what-is-redlin... [2]https://www.brennancenter.org/our-work/research-reports/gerrymandering-explained https://www.brennancenter.org/our-work/research-reports/gerr...
TX81Z 3y ago
Meanwhile I know of highly skilled engineers in FEDGOV who not only experiment with the APIs for hobby projects but could give a very good lecture on how they work, their limitations, and affordances. But hey, I guess all those dumb dumbs at NASA can’t tie their shoelaces.
dontupvoteme 3y ago
It is. I'm from the north-middle, sometimes I'm surprised how little people on the west coast of America think of the people on the east coast. Above All, The NSA knows their shit. 44B+, Autonomy and Mathematicians Proves it above all, regardless of Morals. Don't attack them on knowledge. You have to use a different approach to climb Everest. I would only put the Mossad, KGB and <insert name of the chinese SIGINT agency> on their level.
ChatGTP 3y ago
Just interesting to see. Just communicate more clearly, you're clearly displeased and want to say something more?

sorokod 3y ago

Perhaps an equivalent of "risk aversion", negative implications outweigh the positive even if the count of either is the same.

courseofaction 3y ago

The bias will be by design and known to the purchaser.

deleted 3y ago

[deleted]

sottol 3y ago

But is that really what you want to happen IN GOVERNMENT? For bureaucracy to adapt even mildly flawed technology?! Bureaucracy + blind trust in hierarchy/computers = Kafka-esque hellscape. "I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$. [1] https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/ https://www.washingtonpost.com/technology/2023/04/05/chatgpt...

okennedy 3y ago
Things are bad enough already, even with pre-LLM technology. "I'm sorry sir, your 8-year old son can't board this plane because the system says he's a terrorist" [1]. [1] https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html
- adventured 3y ago
  You're using a straw example to try to claim a massive point. There are 340 million people in the US. The US has one of the largest government systems that humanity is likely to ever see. How many similar cases of children being put on the terrorist watch list have there been in the past 20 years? Unless there are a lot of cases of that happening, if it's not rare (hint: it's extraordinarily rare; your example story is from 13 years ago), your premise is pure straw.
  - jhncls 3y ago
    Wouldn't being innocently jailed for 9 days because of a wrong face recognition count as another example? [0]: https://www.wired.com/story/face-recognition-software-led-to-his-arrest-it-was-dead-wrong/ https://www.wired.com/story/face-recognition-software-led-to... [1]: https://www.theguardian.com/us-news/2023/apr/27/california-police-facial-recognition-software https://www.theguardian.com/us-news/2023/apr/27/california-p...
    - CDT-MLT 3y ago
      Jesus Christ
    - replygirl 3y ago
      An example against adoption of larger, more generalized, and less reliable models, yes
- deleted 3y ago
  [deleted]
3y ago

parker_mountain 3y ago

> you conveniently leave out the positive benefits of these models that are causing people to promote them We are holding it to a higher standard because this is going to be used by agencies with serious power over people's lives. It is not just to spit out, for example, a form rejection letter from a credit card. And, nobody thinks that Microsoft wants to "screw" the government. But if something happens, the impact can be magnified significantly considering the reach and power of the state. I'm all for moving forward quickly with this tech, but there are a handful of spaces (government, medicine, aeronautics) where we need to be extremely deliberate.

williamcotton 3y ago
[flagged]
- birdyrooster 3y ago
  We being the people upvoting the GP comment. Lol you really read into that.

jchw 3y ago

Not really that interesting, it's pretty much always what people do when the potential upside of something is limited but the potential disastrous consequences are basically limitless.

InCityDreams 3y ago

Well, if government*s can't even use MS excel correct, this is going to be soooo much fun. I feel for my kids, and all I can do is warn them... https://theconversation.com/excel-errors-the-uk-government-has-an-embarrassingly-long-history-of-spreadsheet-horror-stories-147606 https://theconversation.com/excel-errors-the-uk-government-h...

Solvency 3y ago

GPT is an incredible productivity tool for people who are informed about it and understand its limitations and use it wisely and responsibly. The vast majority of government workers are underpaid apathetic cogs in a machine who do not fit this bill. We just saw a LAWYER use hallucinated information in his practice. A lawyer who is ostensibly rigorously trained and tested in his field. The amount of misuse of GPT by way of sheer ignorance alone by the government will be astounding.

burnished 3y ago

Theres already precedent with police and judiciary misusing technology they don't understand, I suspect the underlying issue is that this person would not trust government employees generally with a tool that issues false positives and requires discretions.

romwell 3y ago

>you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them. When talking about the pitfalls of bringing GPT-4 as a tool for government officials, the positive features are outside of the scope. Obviously they exist, otherwise it wouldn't be proposed. But it's not our job to do a sales pitch for GPT-4. >Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions. No. The OP suggests that the primary users of the tool would be low-level, underpaid officials, who are mostly following directions. Not because it's their fault as human beings - but because their positions require them to do so, and perhaps for good reasons. In any case, direction-followers (of questionable competence and low pay grade) make up the bulk of large organizations in general as a consequence of growth. Which is why the tool is being pitched in the first place: to streamline work of people who would rather not do it in the first place. >Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, b As if the governments aren't prone to massive scandals even without GPT-4 in place. Try checking the news today, you may be surprised. >not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer. "By Microsoft" wasn't in OP's statement. Manipulating data and statistics to get the desired outcome from seemingly impartial algorithms is something our government has been engaging in for a very, very long time. Redlining[1] and gerrymandering[2] are just two prime examples. We don't have to make any assumptions or apply biased perspectives to say that we expect: 1)Training data to be manipulated by entities with interest in influencing the outcome; 2)Lives of countless people being adversely affected, and 3)There being a massive scandal as a result. The government has a long track history of 1, 2, and 3 happening any time a system is introduced which lacks transparency, and large language models are the epitome of that. The only unrealistic thing about OP's statement is that a scandal will actually happen, instead of people getting away with it. >It's always interesting to see how people engage with new technologies on HN. And it is even more interesting to see how people here engage with ethics. [1]https://www.nytimes.com/2021/08/17/realestate/what-is-redlining.html https://www.nytimes.com/2021/08/17/realestate/what-is-redlin... [2]https://www.brennancenter.org/our-work/research-reports/gerrymandering-explained https://www.brennancenter.org/our-work/research-reports/gerr...

TX81Z 3y ago

Meanwhile I know of highly skilled engineers in FEDGOV who not only experiment with the APIs for hobby projects but could give a very good lecture on how they work, their limitations, and affordances. But hey, I guess all those dumb dumbs at NASA can’t tie their shoelaces.

dontupvoteme 3y ago

It is. I'm from the north-middle, sometimes I'm surprised how little people on the west coast of America think of the people on the east coast. Above All, The NSA knows their shit. 44B+, Autonomy and Mathematicians Proves it above all, regardless of Morals. Don't attack them on knowledge. You have to use a different approach to climb Everest. I would only put the Mossad, KGB and <insert name of the chinese SIGINT agency> on their level.

ChatGTP 3y ago

Just interesting to see. Just communicate more clearly, you're clearly displeased and want to say something more?