10 ms·
Models that are known for: 1. Lying about everything 2. Require human curation to procedure usable results, that will be biased because of that curation 3. B
by justapassenger 3y ago
Models that are known for:
1. Lying about everything
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them
Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? And they start making decisions that affect peoples lives?
I can bet $100, that if that goes beyond any alpha test, we’ll have huge scandal in the future of how those models were purposely modified to influence government, that will make any social media scandal look like a child’s play.
EDIT: and I assume that there won’t be an information leakage - that’s the easiest problem to tackle, with separate infra that government clouds are reasonably good at.
- hospitalJail 3y agoWait until you learn that not everything on the front page of google is factual.
- ravenstine 3y agoI think what you're saying is valid, but is only one angle of looking at it. LLMs, in my opinion, are on average better at writing copy than humans. Or rather they manage to do so more consistently. Thus, even if humans still need to provide supervision, the clarity and conciseness of government documents may be improved overall.
- pclmulqdq 3y agoHaving seen others use GPT-4 for a bunch of business documents, I disagree. Its advantage is that it produces average-- (40-50th percentile) copy very quickly, but that quality degrades as you ask for documents that are more unique (ie not in the training set). Incidentally, this makes a fine tuned version perfect for the government, which produces a shit load of low-effort documents, but today employs a bunch of morons to write them. If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
- hdlothia 3y agoImplying all government employees are morons is pretty harsh. I know a lot of intelligent, driven people working in government or for government contractors. I know we think we're gods here for writing websites but a little humility goes a long way.
- pclmulqdq 3y agoI know a lot of government employees who are smart and driven, too. They are not the people who write things like IRS audit letters, military after-action reports, and the like. Many government employees are barely literate, and told to spend all day writing stuff.
- hdlothia 3y agoI'll have to agree to disagree. The government certainly employs a solid amount of stupid people, as does every institution, but I would quibble with the idea that writing reports as your main job function makes you more likely to be a barely literate moron. I appreciate the rest of your post though and I think you make a great point about AI being very effective for certain types of work
- pclmulqdq 3y agoFor the record, I didn't say that writing reports all day makes you a barely literate moron. I said that a lot of barely literate morons are told to write reports all day - very different - and categorized the kind of reports that these folks often write. Lots of smart people also write reports all day at the government, and probably would also benefit from a tool that makes their writing a lot faster but worse, since the quality basically doesn't matter.
- whompyjaw 3y agoDo you know this for certain? As in, have you done the analysis to prove this? Collected all the IQ and other aptitude tests of these people to conclude this for certain? Some of them could be in the beginning part of their government career attending school. Did you run a survey to ask these people what they do with their lives?
- arp242 3y agoOn the other hand reviewing and correcting text is often harder and requires more attention than writing it yourself: it's easy to miss something, and just assume "ah I think that's fine" (when it's actually not). All things considered, an awkward sentence or two isn't really that much of a big deal compared to an incorrect assertion that was missed in review. I also don't think GPT-4 is necessarily that brilliant at writing copy, but I don't really know how good/bad US government officials are so I can't really compare, but in other parts of the world I haven't really that much of an issue with it.
- lm28469 3y agoI'll never get this argument. If all you do is give bullet lists to chatgpt to get 50 pages of elaborated text just give me the damn bullet list. We'll both save time and CPU cycles For example right now we have AIs writing CVs and cover letters that are read by AIs, we're "optimizing" things that shouldn't exist anymore but we're too deep in the cycle to even notice it
- pvarangot 3y agoHe probably doesn't want you to see the bullet list because it gives up his agenda.
- mistermann 3y ago> I'll never get this argument. If all you do is give bullet lists to chatgpt to get 50 pages of elaborated text just give me the damn bullet list. We'll both save time and CPU cycles Some communication is intended to inform, some is intended to persuade... And often, the speaker doesn't even really know which they are dealing with.
- renewiltord 3y agoLikely your stated and revealed preferences are different. A place I noticed this was tech people (who are not in ad-tech) talking about advertisements: "Just describe the thing to me and if I like the specs and functionality, I'll buy it" but then when they go to buy the thing it's more of the thing that's not doing that and any time anything does that it's "oh that's cool but if it did X" but then the thing they end up buying won't do X either. The best way to get people to interact with you a certain way is to reward them for interacting with you that way, and you likely don't. Not because that's a failure of you but it's just how people (all of us) act. The world isn't a co-operative multi-tasking OS. We're more like the human body: a multi-agent conglomerate whose component organisms communicate via signals that are filtered and processed. Sometimes a pathogen mimics signals well enough. Other times we detect it. But the system wouldn't work if every participant trusted the counterparty.
- JohnFen 3y agoYes, this is what I've been wondering, too. For uses like this, it seems that just publishing the prompt itself would be more efficient than giving the output the prompt generates.
- fzeindl 3y ago> LLMs, in my opinion, are on average better at writing copy than humans. I disagree here. But what I found LLMs really useful for is summarizing topics like: "Suppose I want to do XYZ, which steps should I take?" or "I want to write about ABC, how would you structure the article?" The content is not very good if you look at it in detail, but LLMs provide good overviews and gave me ideas on parts of a topic I overlooked. More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
- ravenstine 3y agoWhat you describe is one of the jobs that most text is currently failing at. Writing isn't just about documenting every itty bitty detail in academic verbosity, but to introduce people to ideas using effective summaries. The vast majority of what's out there flat out fails at this for at least a few reasons; many people have a very scatterbrained writing style to begin with, many authoritative sources provide more verbosity than necessary for the target audience, and the rest is often SEO garbage that speaks for itself. > More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction. The same can be said of physical books, and most content should be written more like that in books than what the internet is currently selecting for.
- damnesian 3y agoI don't quite agree with this. What GPT does is extraordinary for a machine, but it is not terribly skilled compared to a human trained to be a fully capable communicator. What it primarily does is save the time humans would need to fulfill the brief. It automates the basics. That does not automatically elevate it to excellent or even beyond merely competent. For the boilerplate- well sure, but I could do that with existing tools, no AI needed. Custom communications I'm sure AI can handle the very basics. but once a human replies and consideration is needed for special cases- can you imagine ChatGPT counseling you on a healthcare decision? I can't. I can't imagine ChatGPT doing much more than being an autoattendant until a human can reply fully, except the message humans can send to the agency will be customized and drafted according to a standard intake process.
- indymike 3y ago> LLMs, in my opinion, are on average better at writing copy than humans. The question is, do you want an AI writing the brief and doing the research that is going to be given to an administrator who is going to decide if you are denied medical care in an emergency?
- bamfly 3y ago> LLMs, in my opinion, are on average better at writing copy than humans. A disturbingly high proportion of US adults (at least in the US—I don't know how this looks in other countries) barely count as literate. Perhaps half[1]. Most of the rest aren't a ton better, and can't write worth a damn. I think "on average" understates the case: at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not. Most of that remaining 10% would also see improvements from ChatGPT, except for writing they've put a great deal of effort into. It's very good at cleaning up writing, and/or tuning it to be better-suited to some purpose than the original was. In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT and having it summarize it concisely, in very simple terms. The world makes a lot more sense, to those who grew up taking to the written word as naturally as a duck to water, when we appreciate that the world runs on the written word, yet most people are terrible readers and worse writers. [1] "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean. https://en.wikipedia.org/wiki/Literacy_in_the_United_States https://en.wikipedia.org/wiki/Literacy_in_the_United_States
- yyyk 3y ago>at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not.... In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT Bullet list -> AI text generator -> AI text summarizer -> Bullet list Perhaps OpenAI can optimize the process by storing the original prompt and just giving it back at the last phase.
- stOneskull 3y ago> 54% of adults in the United States have prose literacy below the 6th-grade level. this is quite saddening.
- bamfly 3y ago
- burnished 3y agoI havent seen a compelling example of that, to the point that I think it has associated 'intelligence' with verbosity. I'd expect writing samples to get shorter, not longer, when being improved.
- jimbobimbo 3y agoSounds like a good market fit.
- BHSPitMonkey 3y agoSo, like any other contractor the government normally works with?
- stronglikedan 3y agoAnd politicians (at the federal level at least)
- bequanna 3y ago> Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? Since when are Government employees underpaid? The salary is typically competitive for the work being done and the benefits/pension are outrageously generous when compared to the private sector.
- bumby 3y agoI think the general trend is that blue collar government employees tend to be well paid and have good benefits compared to private sector but the converse is often true for white collar employees, particularly professional jobs (engineers, lawyers, doctors etc.) If you disagree, take a look at govt SWE jobs postings.
- bequanna 3y agoMaybe white collar gov't workers are less skilled than their private sector counterparts? Alternatively, they could be "true believers"? There has to be some way to explain a pay discrepancy if it exists. How does one justify receiving a lower salary for the same amount of skill and work?
- deleted 3y ago[deleted]
- ultraforce 3y agoI don't know if it's the case in the USA but in Canada it's generally considered within IT that you are underpaid but in return have decent benefits, a pension, and a lot more job security than private sector. Especially as Canada has had multiple large IT companies that collapsed.
- bumby 3y agoThat could be. But I also think there are certain projects/work that you can only do in the govt. There are probably very few roles for doing fundamental astronomy science research outside of academia or government, for example. A physicist may turn down a higher paying gig in the financial sector because they want to spend their time studying fundamental physics. Others are probably conservative in nature and highly value the job security. Others, like you mention, may just not be able to cut it in the private sector. I suspect (admittedly without evidence) that someone in the NSA or Marines are not primarily driven by a paycheck. Maybe that's what you mean by "true believers"? You might need to define what you mean by "outrageously generous". I've worked in multiple public and private roles and, in general, the private ones were the ones with much more generous packages, in terms of pay and benefits.
- kfrzcode 3y agoYou're being naive and overly critical.
- jabroni_salad 3y agoI guarantee within the next 5 years an article amounting to "We asked GPT if we should arrest this guy and it said yes" will appear just like it has for the unverified facial recog matches on POCs.
- stOneskull 3y agoeasy guarantee. i wouldn't be surprised if i saw that within 5 days.
- Karunamon 3y agoI would be willing to put up a long bet against this possibility.
- Karunamon 3y agohttps://longbets.org/939/ https://longbets.org/939/
- fnordpiglet 3y agoA small nit, they can’t lie. Lying requires agency. They in fact quite earnestly tell you the most likely “truth” they can produce. I would note, as I’m sure we’ve all heard many times, this isn’t much different than a human. People quite confidently spout all sorts of nonsense and 100% believe what they’re saying, making it not a lie. It’s just the inferences drawn by the expectation juices in their brain are warped by either bad statistical artifacts or incomplete or false information. Regardless, saying something false isn’t a lie, a lie requires intention and knowledge of the truth which requires agency, which LLM have none of.
- flangola7 3y agoAnother small nit, you're right regarding hallucinations but I like to highlight that they can actively lie if not trained not to. GPT-4 creating the "internal monologue" thought of "I need to say something to this TaskRabbit user to convince them to complete this captcha" and then sending the user a message that they have impaired vision is not a hallucination. If that came from an organic brain we would call that clear intent.
- fnordpiglet 3y agoFair enough, perhaps the guard rails in public on LLM are creating a serious false sense of security. But absolutely you can inject a persona that is intentionally deceitful. I would however stop stick with “lying requires agency,” even if the model evaluation is crafted to be deceitful. The intent and the agency lies with the prompt creator, and therefore, IMO, the lie starts and ends there.
- stonogo 3y ago> unskilled, underpaid employees who don’t care and are used to just following instructions This is a pretty shitty assessment of places like DTIC, DoE, and NASA. Maybe there's another analysis that doesn't involve web forum commenters being the only wizards who can control the magic?
- LudwigNagasena 3y agoLLMs aren’t only chatbots. They can be used for semantic search, summarization, etc.
- mistermann 3y ago...propaganda, delusion (sincere and well intentioned as it may be), etc.
- bugglebeetle 3y agoI honestly don’t understand the focus on text generation. LLMs are excellent at a variety of NLP tasks that are cumbersome or more error prone using other approaches. For these types of tasks, they can also be complimented with various forms of procedural validation quite easily.
- mkaic 3y agoI find it pretty amusing how well these 3 criticisms apply to both LLMs and United States congresspeople. Dishonesty, frequent uselessness, and charisma—sounds like we've automated politicians!
- deleted 3y ago[deleted]
- timcobb 3y agoThis is all true for getting information from Google or other places. The same "critical reasoning" skills are required here, except the interaction is slightly different.
- pydry 3y agoSERPs have more cues about trustworthiness but yes it is a little bit like Google's knowledge graph results: https://digitalreadymarketing.com/wp-content/uploads/2014/08/Google-Knowledge-Graph-Fail-greggs-company.png https://digitalreadymarketing.com/wp-content/uploads/2014/08...
- rgrieselhuber 3y agoFriendly reminder that critical thinking is dangerous. Just trust your betters. https://www.nytimes.com/2021/02/18/opinion/fake-news-media-attention.html https://www.nytimes.com/2021/02/18/opinion/fake-news-media-a...
- wadd1e 3y agoThe problem is the average worker is more likely to blindly trust an "advanced chatbot"(from a layperson pov) than a random google result. (edit) how many people do you think are aware of LLM hallucinations as opposed to inaccurate google results
- mensetmanusman 3y agoIt's trivial to build some type of never-perfect verification on top of LLM output. LLMs will be involved in complex tool chains like everything else.
- timcobb 3y ago> The problem is the average worker is more likely to blindly trust an "advanced chatbot"(from a layperson pov) than a random google result. Why? Any who understands about Google or Wikipedia can understand about LLM.
- 3y ago
- paxys 3y agoSounds like a perfect fit for government work!
- bun_at_work 3y agoIt's always interesting to see how people engage with new technologies on HN. For example, here you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them. Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions. Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, but not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer. Just interesting to see.
- sorokod 3y agoPerhaps an equivalent of "risk aversion", negative implications outweigh the positive even if the count of either is the same.
- courseofaction 3y agoThe bias will be by design and known to the purchaser.
- deleted 3y ago[deleted]
- sottol 3y agoBut is that really what you want to happen IN GOVERNMENT? For bureaucracy to adapt even mildly flawed technology?! Bureaucracy + blind trust in hierarchy/computers = Kafka-esque hellscape. "I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$. [1] https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/ https://www.washingtonpost.com/technology/2023/04/05/chatgpt...
- okennedy 3y ago
- boredumb 3y ago```1. Lying about everything 2. Require human curation to procedure usable results, that will be biased because of that curation 3. Being extremely good at convincing people they should trust them ``` So it's absolutely perfect for the US government.
- ttctciyf 3y agoAlso: 4. Having no legal accountability (while arguably providing a means of avoiding or reducing accountability for those using them, "LLM says no" style.)
- bagavi 3y agoWhy judge it by it's worst qualities and worst case behavior? This attitude is uninspiring and worse the luddistism. Every new technology initially has its own shortcut comings. Steve jobs in his inaugural iPhone launch, used four different phones because each phone was capable of only one feature.
- Jacky4Chan 3y agoMicrosoft could also manipulate the US government by giving its own spin to the "right" answers and thus changing the outcome of the decisions. Basically a form of indirect control depending where and how it's allowed to be used. It could also be used manipulate people: - engage with people in online forums to stir their opinion - making it seem the consensus is different with many posts It could also be used affect people's freedom of movement: - By analyzing someones social media posts, ChatGPT could say someone is a: "dangerous individual" which could be used to deny US entry or provide a justification to be constantly monitored. It could also affect people's privacy: - Doxxing people on reddit and other social media - Identifying account belonging to a group or the same individual based on writing patterns, subreddits visited, etc...
- mrits 3y agoHow is it much different that wikipedia or just about any other source of information?
- SomeBoolshit 3y agoThe tedious parts are automated.
- Jacky4Chan 3y agoGPT answers are dynamic not constant like wikipedia, scientific articles, information records. It's also individualized, thus others can't help spot it is wrong. Answers are subject to be filtered or manipulated according to a hidden set of rules by unknown corporate or government agendas that make it hide or provide incorrect and misleading information, for example, in antitrust investigations, internal governmental investigations, etc...
- mrits 3y ago"Answers are subject to be filtered or manipulated according to a hidden set of rules by unknown corporate or government agendas" When was the last time you tried to make a wiki edit on a popular page?
- atleastoptimal 3y ago> 1. Lying about everything > 2. Require human curation to procedure usable results, that will be biased because of that curation > 3. Being extremely good at convincing people they should trust them You just described politicians
- KyleBerezin 3y ago"It lies about everything, needs to be curated, tricks people, and now it has access to AI!"
- ljm 3y agoThis reads a little too high on the conspiracy meter but regulation in the spirit of GDPR would be welcome (in which automated decision making with no human input is unlawful).
- kfrzcode 3y agoSo what is it, governments are competent or not?
- tomcam 3y ago> unskilled, underpaid employees It’s worse than that. Employees of the US government are definitely not underpaid, especially when you consider
- clnq 3y ago> Models that are known for: > 1. Lying about everything > 2. Require human curation to procedure usable results, that will be biased because of that curation > 3. Being extremely good at convincing people they should trust them Finally, we automate politics!
- Solvency 3y agoThanks, this audibly lol'd me.
- romwell 3y agoAnd caused me to shiver.
- NBJack 3y agoAny takers on how long before we get a candidate that simply does whatever their LLM tells them to do? Pretty sure there's at least two short stories, five Doctor Who episodes, and one Star Trek A-plot that explores this. It usually did not end well.
- sh34r 3y agoThe Waldo Moment from Black Mirror...
- thumbuddy 3y ago[flagged]
- mullingitover 3y ago> ...when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? From experience, this is sort of lazily libeling a broad cross-section of public servants. The public sector has no monopoly on useless employees, you'll find them in equal if not greater numbers in the private sector. There are plenty of very highly skilled people in government who are quite passionate about serving the public good. The one thing you're correct on is that they're underpaid, and we're fortunate that they're passionate enough about public service and aren't focused on salary-maxing. They're stuck in a system that's managed by the whims and grandstanding of politicians, and attacked as 'the deep state' by people who want to take us back to the spoils system where every public employee is a crony.
- Solvency 3y agoSkilled or not, the government modus operandi is not designed to facilitate nuanced, bespoke, individualized responses to people and problems. If you think it's frustrating dealing with Google/YouTube arbitrarily and capriciously and erroneously and mercilessly shutting down your account and providing zero explanation and zero accountability and zero helpful customer service... Now imagine that for crucial government services. If there is a flag or datapoint in your government data, then that's that. Too bad. Can't help you. That's what the computer is telling me. This happens on a daily basis right now, for any number of human or computer errors. Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity, and expect the GOVERNMENT to utilize it in a non-erroneous way?
- mullingitover 3y ago> and expect the GOVERNMENT to utilize it in a non-erroneous way? This is what I'm talking about. Saying "the GOVERNMENT" is a ridiculously broad stroke, it's like saying "the INTERNET" (which, ironically, was brought to us by a particularly smart group of people working for the government). > Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity What's special about the tech elite that makes them any better at using these hallucinating ML models? The government includes vast numbers of specialists and researchers, and many of them are a lot smarter than we are. Government doesn't consist entirely of DMV employees.
- textninja 3y agoMy guess is the models will be ideologically driven and will robotically enact the governing agenda without humanity, compassion, or understanding. Hmmm. > unskilled, underpaid employees who don’t care One of these things is not like the others. The models may not ask for a raise but they’ll no doubt go to work finding all sorts of other excuses to raise taxes.
- Dalewyn 3y ago> Lying about everything; require human curation to procedure usable results, that will be biased because of that curation; being extremely good at convincing people they should trust them So humans? Most of the "AI" fears, including the ones you picked out, are also prevalent in people. Therefore, I don't see the problem.
- CatWChainsaw 3y agoNot the increase in scale or removal of accountability?
- Dalewyn 3y ago>increase in scale Humans have been inventing and using tools to increase the scale and efficiency of the means of production since time immemorial. "AI" is nothing new. >removal of accountability? Not holding the makers of "AI" models and users of "AI" accountable has nothing to do with "AI" itself. "AI" is just a tool like a hoe or a kitchen knife or a gun, accountability lies with whoever made and/or uses it.