6 ms·
There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so c
by paxys 4y ago
There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremely confident in all its responses.
- Pxtl 4y ago"mansplaining as a service" is the term I've seen used. Or Artificial Cliff Calvin.
- taskforcegemini 4y agoit is not limited to the internet. this is also applicable to any news company
- nuclearnice3 4y agoBefore the had the internet, back when we still had newspapers, we had a similar phenomenon referred to as Gell-Mann Amnesia. > Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. > In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know. https://www.epsilontheory.com/gell-mann-amnesia/ https://www.epsilontheory.com/gell-mann-amnesia/
- zoklet-enjoyer 4y agoThis is why I've said Trump is right about fake news. When I say that people think I'm a weirdo Trumper.
- nuclearnice3 4y agoOne nuance might be that those are two difference senses of fake news. In one case, we have news that is wrong. In another case, we have an insult aimed at reporting that we don't like, regardless of accuracy.
- zoklet-enjoyer 4y agoI haven't trusted the news since the time I participated in manipulating local news stations to create stories about jenkem. More modern equivalent is Momo. It's all fake
- kragen 4y agoThis sounds interesting! What did you do?
- zoklet-enjoyer 4y agoSome of us from the forum totse emailed local news stations, schools, and police departments pretending to be concerned parents. I wrote my high school and local news and said that my child was using jenkem and linked to an article from a local news station from somewhere else in the country to kind of back up my claims. https://www.snopes.com/fact-check/jenkem/ https://www.snopes.com/fact-check/jenkem/
- kragen 4y agoWow, so jenkem was largely a TOTSE creation? Do you have any idea how the original stories from Africa got started — were there TOTSE members in Zambia?
- zoklet-enjoyer 4y agoThere's a BBC article from 1999 about jenkem in Africa. It's messed up and sad. Someone saw that and turned it into a meme. Jenkem in America was a totse creation http://news.bbc.co.uk/2/hi/africa/406067.stm http://news.bbc.co.uk/2/hi/africa/406067.stm
- tshaddox 4y agoI think the effect is a bit overblown. Yes, I have noticed coverage of topics I know about that are oversimplified or straight up mistaken. But these are technical topics that I don’t expect journalists to get right (and in many cases I’d prefer they didn’t try to cover them at all). And I also won’t implicitly trust coverage about other technical subjects that I’m unfamiliar with. I would argue that national politics and international affairs as they are covered in mass news media is a different type of coverage. It’s not about learning technical details and explaining them. It’s more about observing actions and broad sentiment and literally reporting those observations. I wouldn’t trust a reporter to get scientific details correct about some chemical process that’s the subject of some debated EPA regulation, but I could trust a reporter to summarize the debate and report on the sentiment of politicians, public sentiment based on opinion polls, etc. It would be weird to say “that newspaper screwed up the difference between a dominant 7th chord and a major 7th chord in a story about a famous songwriter, therefore I can’t trust them to report which politicians gave the most heated arguments in the debate about the EPA bill.”
- zoklet-enjoyer 4y agoCheck out this excerpt from a local news story I saw yesterday. You know what else acetic anhydride is used in? Production of aspirin. This is basic scare tactics to manipulate people and I see it everyday. A combination of ignorance, laziness, and being the voice of those in power. Fake news and manipulation of the public. https://www.valleynewslive.com/2023/03/09/fargo-cbd-store-owner-says-proposed-bill-would-eliminate-almost-all-states-cbd-market/ https://www.valleynewslive.com/2023/03/09/fargo-cbd-store-ow... "“Semi-synthetic derivatives, or chemically derived cannabinoids, refer to certain types of substances that are produced by converting a cannabis extract into a different substance through chemical reaction,” said forensic scientist with the North Dakota State Crime Laboratory Charlene Rittenbach at the March 3 committee hearing of the bill. For example, according to the Addiction Prevention Coalition, to make THC-O: you need to extract CBD, extract Delta 8 THC from the CBD and then add acetic anhydride. The National Library of Medicine says acetic anhydride is corrosive to metals and tissue, and it’s used to make fibers, plastics, dyes and explosives."
- oefrha 4y agoIn the pre-LLM world we already had rumors/myths/lies/propaganda becoming "facts" after being reinforced through repetition without verification. (Relevant xkcd: Citogenesis https://xkcd.com/978/ https://xkcd.com/978/.) Now, instead of authoritative sounding humans who are possibly wrong or with an agenda, we have an "infallible", "impartial" oracle capable of inventing whatever it wants to. Yes, I have seen ChatGPT treated as infallible on this very forum: "you are wrong because ChatGPT says <insert nonsense here>". Can't wait to see LLMs taking "post-truth" to a whole new level. The propaganda potential is immense, to point out just one application.
- brewtide 4y ago100% on how it "takes over the world". We treat it like an oracle, hand it over willingly in mindset alone.
- theonlybutlet 4y agoPerhaps it's a good thing, everyone will have to become experts in discerning the truth. Much like Video Game graphics looked "real life" until the next gen came out, perhaps society will start to see through any rumours/myths/lies/propaganda. Either that or we descend into a state of anomie.
- oefrha 4y agoDoubt it. I can hardly verify things happening in my city even if I try, let alone e.g. things happening on the other side of the globe which the media keeps telling me about in order to drum up the next cold/hot war. I used to be able to tell Photoshop jobs by non-experts, but soon enough everyone and their mum will probably be able to produce completely fake videos indistinguishable from real ones, so even "video evidence" will mean little.
- danielvaughn 4y agoI think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to seed a database. Normally not a big deal, but these records need to have foreign keys pointing to one another. I wondered whether ChatGPT could help me out, so I briefly described the fields I needed, their format and data type, the type of information they contained, etc etc. It did it almost perfectly, and fixing the little errors was trivial. I was dreading having to do this because it's such a pain in the ass. To me that kind of thing is going to be the most useful application. Everyone's freaking out (in both the "scared" and "excited" sense) over AI's ability to replace creativity, but I'm focused on it's ability to replace toil.
- paxys 4y agoWhile I agree with your premise, the "toil" that you mention is a very small use case for a tiny minority of people. And automating it away has always been trivial. What would have taken 30 minutes to write a Python script now takes 5 minutes to come up with the perfect prompt. And then you have to test and be sure that it does what you want. Neat, but not world-changing. Meanwhile ChatGPT is already being hailed as the Google killer. People are using it to write essays and form opinions. Plenty of online and offline debates contains phrases from ChatGPT repeated verbatim. It's being adopted in newsrooms around the world. So what it is good for in theory and what people are actually using it for are worlds apart. IMO the technology is going to have its "self driving car kills pedestrian" moment very soon because of over-eager and careless users, and that is going to set the entire field back.
- deleted 4y ago[deleted]
- turtlebits 4y agoI disagree. Toil is part of everyone’s job and what creates procrastinators. I used it recently to help me write a browser extension, where I find writing vanilla js and xpath expressions very tedious (just a bunch of syntax lookup) and would not have even bothered in the first place. If you approach it with the right mindset, it’s extremely valuable.
- kristopolous 4y agoThat's almost every reddit thread I've been involved in. Oratory prowess is way more important than accuracy. If you ever want to "lose a debate" on reddit, pick something that's counter-intuitive or widely misunderstood and use nuance and citations in your defense. You'll get cyberbullied every time - sometimes even banned from the sub.
- smogcutter 4y agoFrom those examples, I think you’re maybe also running up against people generally having a low threshold for giving a stranger on the internet the benefit of the doubt. It’s hard to be surprised that “eugenics, but the good kind, just hear me out!” and “state’s rights, but not the disingenuous lost cause kind, just hear me out!” are a tough sell with complete strangers who have no prior reason to trust you or be interested in your opinions.
- kristopolous 4y agocorrect! Btw, I removed them from my examples, didn't want to "discuss" that here. I'm kind of a history nerd and it's like a fishing hook with delicious bait when I see stuff like that. "Oh wait! You think it was just the south that argued states rights?! Actually (brings up fugitive slave clause jurisprudence)" Or how manner and conduct books of the 1800s, eugenics texts, and jordan peterson's 12 rules for life are basically dots in a straight line. That stuff is just catnip. I just need to stop being distracted and go write actual books or something.
- pfannkuchen 4y agoI’m no fan of Peterson, but I’m curious— how are intentionally altering the gene pool of a group and cleaning your room colinear?
- kristopolous 4y agoLook at the table of contents of this book titled "Eugenics" https://archive.org/details/naturessecretsre00shan/page/6/mode/2up https://archive.org/details/naturessecretsre00shan/page/6/mo... There were multiple points of interest for the Eugenicists. One of them was for the general public and how to get them to have a proscribed conduct and behavior. Maybe you've seen those diagrams where they have 2 life paths depicted from the Victorian era. That's from Eugenics. Example: https://ia800301.us.archive.org/BookReader/BookReaderImages.php?zip=/16/items/naturessecretsre1919shan/naturessecretsre1919shan_jp2.zip&file=naturessecretsre1919shan_jp2/naturessecretsre1919shan_0020.jp2&id=naturessecretsre1919shan&scale=2&rotate=0 https://ia800301.us.archive.org/BookReader/BookReaderImages.... Here's "The Road to Success" for "The Young Man" https://archive.org/details/naturessecretsre00shan/page/244/mode/2up https://archive.org/details/naturessecretsre00shan/page/244/... Again, this is part of Eugenics. We focus on the part that was removed (race science) and leave mostly unexamined the parts that were reinvented or repurposed as say, "The Power of Positive Thinking". Eugenics books, for the general public, were a different beast. Probably the best succinct description for how they thought of themselves can be summarized in this 1921 diagram: https://images.prismic.io/wellcomecollection/efc97918-8073-4fc5-95f3-34d75eb286ef_eugenics-tree-1921-tree-only.jpg https://images.prismic.io/wellcomecollection/efc97918-8073-4... It is a dangerous psuedoscientific epistemology - a practice with a bunch of faulty logic and ways of knowing that can lead to disasters and it's alive and well - for instance, assuming a social structure in lobsters applies to humans. That these adherents tend to be revealed as secretly racist and thus subscribe to the part which was removed, should be of no shock. We've mischaracterized Eugenics[1] and allowed it to fester by other names or as Maurice Bardèche, the french neo-fascist observed about fascism in the 60s, "With another name, another face, and with nothing which betrays the projection from the past, with the form of a child we do not recognize and the head of a young Medusa, the Order of Sparta will be reborn" The point is that if you don't draw the perimeter right around an idea, you can't reign it in. It'll just escape from your hands and re-emerge as something else. So, for instance, the slave plantation becomes the prison farm (https://en.wikipedia.org/wiki/Prison_farm https://en.wikipedia.org/wiki/Prison_farm). We tend to, for good reason, ignore societal trainwrecks and blights on our past. But trainwrecks are the most important thing to study if we strive to build safer trains and seek to avoid similar disasters in the future. Things have to be accurately tackled with in order to defeat them. --- [1] it's worth noting it's a big topic and some "real" science has also come out of it, such as biosocial theory and biostatistics studies. Peer reviewed journals such as https://en.wikipedia.org/wiki/Journal_of_Heredity https://en.wikipedia.org/wiki/Journal_of_Heredity used to be eugenics journals. This is, however, a very small minority.
- pyuser583 4y agoI’ve been quizzing ChatGPT about it’s knowledge of the oil industry - it has no idea what’s talking about. For example, I ask it to simulate a discussion between a flow engineer and a geologist, and it doesn’t seem to understand they work together regularly. Not like there’a a stack overflow for this sort of thing.
- stevenhuang 4y agoyou need to prompt it with more specific knowledge; if you're generic in your prompts it will be generic in its responses. inject some field-specific terminology and be esoteric in your queries, etc its latent representation is vast, I find you can get much higher quality+interesting responses if the prompts are tuned appropriately
- aliasxneo 4y agoCan you give an example? I see this advice a lot but usually devoid of practical tips.
- stevenhuang 4y agoHere's an attempt. Looks pretty good to me. It's sort of like, how would you imagine an expert would respond if all you said was "simulate a discussion between a flow engineer and geologist". Without providing any further context or hints into what you want, I also wouldn't be surprised if I get a generic response. But instead if you sort of know what you want to ask, they'd have more to work with and be able to better answer your query. Note I know nothing about this field, I just looked at https://en.wikipedia.org/wiki/Engineering_geology#Soil_and_rock_mechanics https://en.wikipedia.org/wiki/Engineering_geology#Soil_and_r... and understood this was for oil and gas, so: > Simulate a discussion between a flow engineer and a geologist in the planning phases of a drill operation. Use field-specific terminology and aim for accuracy, the scenario will be used to teach a petroleum engineering class focused on deformation monitoring. Flow Engineer: Good morning, I'm the flow engineer for this drill operation. Nice to meet you. Geologist: Hi, I'm the geologist in charge of the geological analysis for this project. Flow Engineer: Great, I understand we need to work together to ensure the success of this drill operation. Can you tell me more about the geological features of the area we're drilling in? Geologist: Sure. We're drilling in an area with significant deformation, caused by tectonic activity. There are a variety of rock formations, including sandstone, shale, and limestone. There's also a possibility of encountering fault zones, which could pose a risk to the drilling operation. Flow Engineer: Thanks for the information. We'll need to take into account the deformation and rock properties when designing the wellbore trajectory. Have you conducted any geomechanical analysis? Geologist: Yes, we've conducted a rock mechanics study to determine the stress and strain characteristics of the formations in the area. Based on the results, we recommend using a multilateral well design with a horizontal section to avoid encountering the fault zones and minimize the impact of deformation. Flow Engineer: That's helpful. We'll need to ensure that our drilling fluid properties are appropriate for the formations we'll be encountering. Can you tell me more about the porosity and permeability of the rock formations? Geologist: The porosity and permeability of the sandstone formations vary, but in general, they're relatively high, which means we'll need to be careful when selecting our drilling fluid to avoid fluid loss into the formation. The shale formations are less porous and permeable, which means we'll need to use a different type of fluid to avoid damaging the formation. Flow Engineer: Thanks for the information. We'll take that into account when selecting our drilling fluid. We'll also need to monitor deformation during the drilling process. Do you have any recommendations for deformation monitoring tools we should use? Geologist: Yes, we recommend using acoustic emission sensors to monitor rock fracturing and deformation, as well as tilt sensors to monitor any changes in the wellbore inclination. We also recommend conducting periodic borehole imaging to ensure that the wellbore is remaining stable. Flow Engineer: Excellent. We'll ensure that we incorporate those tools into our drilling plan. Thank you for your time and expertise. I'm looking forward to working with you on this project.
- squarefoot 4y agoThat was also common on TV. Everyone seems credible, until we spot errors when someone in our field starts talking. edit: minor typo
- alextheparrot 4y agoMy first recollection of this sentiment pertained to the New York Times, for what it’s worth.
- nonethewiser 4y agoTherefor you have to verify the answer or accept that it may be wrong. I’m finding that it’s fairly good for code snippets. You can test that pretty quickly. But asking it something you don’t know and can’t verify is a huge risk.
- chrstphrknwtn 4y agoThis generalises very broadly. I have some expertise and knowledge is one or two specialised fields, and when I read Wikipedia, news articles and blogs about those fields, it's maddening how incorrect many people are, whilst sounding perfectly knowledgeable.
- _0ffh 4y agoI don't think it's an internet phenomenon. It's a human phenomenon. It doesn't matter if it's a conversation or a piece in a newspaper (c.f. the Gell-Mann amnesia effect).
- LordAtlas 4y agoReddit version of Gell-Man Amnesia: https://www.johndcook.com/blog/2021/01/18/gell-mann-amnesia/ https://www.johndcook.com/blog/2021/01/18/gell-mann-amnesia/
- WelcomeShorty 4y ago> There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. Sounds like the old days to me, where we would meet up , discuss the world and what have you, without any means to fact check anything online. Changes there was a real expert on the subject in your circle was at best 20% So we just enjoyed / continue our life. No harm done as long as nobody took the discussion as gospel.
- helsinkiandrew 4y agoWhen I've asked ChatGPT about things I know a little about I find its response is often wrong. So much so that I always respond with "Are you sure that's correct?" it then often responds "I'm sorry you are right, I should have said... " and then says something a little closer to the truth. I wouldn't rely on ChatGPT or any system that relies on it, except perhaps for summarising/rephrasing a single, known document. It worries me that people are using it for software development - creating stubs, bootstrap code. It's perhaps OK if you know the language/platform well enough to fix issues, but if you then use it on something that your not an expert in, what is the outcome. For example: Finding the source of literary quotes - miss attributing them - took 5 or 6 QA responses for it to get close. Explaining how a p-channel enhancement mode MOSFET works - explained n-channel Explain regex - often gets the technical parts right but can make very bad suggestions on what the regex aim is. Name some classic cocktails created in Europe - suggested Manhattan, which it said was acceptable because it was popular in Europe Suggest some gin based cocktails that don't include citrus, suggested those with lemon juice.
- dragonelite 4y agoI compared ChatGPT to a middle tech manager that just heard he has to create a last minute presentation. He knows all the current buzz words to sounds smart.
- Kiro 4y ago> common on Reddit and other such forums It's especially common on Hacker News.
- remarkEon 4y agoGell-Mann Amnesia Effect but at scale. I find this all the time when reading the NYT, or Washington Post, or whatever, about something I'm intimately familiar with. ChatGPT just does this as a sort of carnival trick, and people seem to think it's funny for some reason. https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnesiaEffect https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes... >Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. >In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.
- gonzo41 4y agoSame effect happens with newspapers. Check your sources.
- lordnacho 4y agoSounds like internet version of Gell-Mann effect