13 ms·
ChatGPT turned generative AI into an “anything tool”
- lakomen 3y agoI'd like to have LawGPT, where it reads all laws of a certain country, and I can ask it things like, "what are my options when the neighbors tree crosses over into my property?"
- consumer451 3y ago> I'd like to have LawGPT I tried ChatGPT 4 for a specific TX county's laws. This was after I had spent hours doing traditional search. ChatGPT got everything correct, and even gave me some more direction.
- xmprt 3y agoThis only shows that ChatGPT is as good as or better than your traditional search and not that it's gotten the law right. Unless you go to a lawyer and confirm this doesn't say a whole lot especially considering how prone LLMs are to hallucinating incorrect answers that look correct (even if you found it matched up with Google search, the answers might still be wrong in whatever blog or article you read).
- consumer451 3y agoThat's a fair point. This happened to all be in an area of the law, estate law, which was heavily documented for the public in that jurisdiction. The estate was tiny, and the actual legal advice was locked behind a paywall ($7k minimum in legal fees) which would have otherwise taken the majority of the estate. Still a huge win for $20/month.
- xmprt 3y agoYeah. The results are impressive and I can't deny the value when ChatGPT is able to get things like this right. But especially given how much disinformation is starting to be spread on the internet these days and how it's all being scraped for training LLMs (and especially in the case of law where different jurisdictions can have very different laws but ChatGPT could easily get its wires crossed), I'm worried that it might lead to more harm than good when people make decisions based on incorrect outputs of LLMs.
- Alupis 3y ago> ChatGPT got everything correct My anecdote with technical information is exactly the opposite, unfortunately. I routinely use it to help guide my research - generally about topics I know something about but want to know more. It often will provide some response that is entirely wrong but looks very good. When called out, it apologizes, then proceeds to produce more information that's also not entirely correct. There's an artform to teasing out the right answer... but of course you have to be knowledgeable enough to know what is right in the first place. The subtleties of correct or wrong can be very difficult to determine for a layperson. Without deep knowledge, one might be inclined to believe the first response it provides. That's pretty terrifying when you think of the possible implications revolving around research, law, etc.
- consumer451 3y agoI have had similar experiences in other areas, that's why I didn't try an LLM before spending hours researching. But I must say that I was shocked at the accuracy in that one particular area... Harris County estate law. What is the difference here? No hallucinations, everything correct… is it just random chance?
- LouisSayers 3y agoI'm having the same issues with it getting things confidently incorrect. The flow I've been going through: * Do this task with this constraint > GPT4 Does task * Analyse all of the items and whether they adhere to the constraint > GPT4 "Apologies, I got some things wrong, here's what I did incorrect" * Fix the things you did wrong, remember the constraint is of upmost importance > GPT4 "I have completed the task and fixed the constraint * Analyse all of the items and whether they adhere to the constraint > GPT4 "Apologies, I got some things wrong, here's what I did incorrect" ... At least it can analyse the results, but even then if you don't ask in the right way it will blatantly lie to you and tell you it's all correct when it's not.
- nicce 3y agoI have found the very similar issues. But on programming language and other logical question areas it does not matter much, as you can verify with logic if something is correct, and then it is incredible useful
- Turskarama 3y agoThe issue is that you cannot _guarantee_ it will get everything correct, the fact that it managed it in your case is largely a matter of luck. In law, being correct 99% of the time is not really good enough.
- skissane 3y ago> I tried ChatGPT 4 for a specific TX county's laws. This was after I had spent hours doing traditional search. ChatGPT got everything correct, and even gave me some more direction. Some months back, I tried asking it detailed questions about Australian drug laws. Initially it responded accurately, but then told me some absolute crazy nonsense (that LSD was a completely legal drug in Australia). I just tried it again and it isn't doing that any more. I still wouldn't trust anything it says about legal topics. IBM mainframe assembly language remains a topic on which ChatGPT (the GPT-3.5 version at least) rather consistently hallucinates. For example, I just asked it to explain the difference between SVC (Supervisor Call) and PC (Program Call) instructions. It wrongly claimed PC is used to make calls within the current program. On the contrary, the PC instruction is basically an LPC/IPC mechanism, it is used to make a call to another process running in a different address space.
- tbrownaw 3y agoRelying on that too hard might be a bit hazardous: https://reason.com/volokh/2023/08/21/another-note-from-a-judge-about-generative-ai-programs/ https://reason.com/volokh/2023/08/21/another-note-from-a-jud... https://reason.com/volokh/2023/06/22/sanctions-issued-in-case-where-lawyers-cited-chatgpt-hallucinated-precedents/ https://reason.com/volokh/2023/06/22/sanctions-issued-in-cas...
- menro2 3y agoWe are working on LawGPT. Today's foundational models like GPT 4 have some Laws, but not all. Sadly some US States still require the purchase of printed copies and others are blocked by TOS.
- bronco21016 3y agoI’m curious of methods other commenters to you are using. I recently vectorized a PDF of material that contains a lot of complex scheduling language. It covers legal regulations as well as contractual constraints. Using langchain’s QA function I queried the vector DB. I then tested it against a sample of questions from a Facebook group on the subject. It did shockingly well. It seems so much has to do with feeding it the right chunks of source material. When I experimented with just copy and paste small chunks on my own it rarely provided useful feedback. I love the law idea. I just bought a house and would like to do an addition. Feeding local code and regulation for QA could be a god send in navigating this stuff for a non-expert.
- PeterisP 3y agoLots and lots of organizations would want a strong ChatGPT-like tool which is trained on a specific (likely not open) dataset of their choosing in addition to all the large general data that grants "common sense" and language understanding. Your example for all laws of a certain country is a good one, and the legal department of some (for example) chemical megacorp would want that and also all the court cases that are even so slightly relevant, and all their contracts they have made since 1900, and all their internal policy documents - and that has quite some potential. But it really does need to be a large language model, for such scenarios, the differences between e.g. ChatGPT 3.5 or ChatGPT 4 or Llama or Llama 2 are meaningful.
- yreg 3y agoHow do you build that in a way that you can rely on the results? The safest approach currently is to use GPT-4 and tell it something like: "I will give you a block of text and a question. Please answer the question using only the information in the provided text. Don't make any guesses and admit if you are not sure about your answer. If you don't know the answer, say you don't know the answer. Question: X Text: Y" Trouble is that you cannot put the entire books of law into Y because the context is limited. So currently you need to do something else to sieve through the law material and find only the passages most relevant to the question through embeddings or full text search or some other method. It's not very reliable.
- shagie 3y agoThere was an example project a bit ago... https://www.chatpdf.com https://www.chatpdf.com - https://news.ycombinator.com/item?id=35626312 https://news.ycombinator.com/item?id=35626312 and also https://askyourpdf.com https://askyourpdf.com https://news.ycombinator.com/item?id=35430409 https://news.ycombinator.com/item?id=35430409 I don't recall which one it was (or if it might have been another one) that had a rental contract as the example document. The interesting part (to me) was the ability to ask it questions like "can you bring a dog to a party" and have it answer and show the passages that restricted pets and parties.
- hnfong 3y agoI suspect having a database of legislation and case law, and a good vector database, then feeding relevant context into a language model with a large context window will work best.
- party_zebra 3y agoI bet we're going through the cycle netflix went through. Cable was fractured and expensive, then netflix made it cheap and monolithic, then everyone realized there was money to be made in this new paradigm, then there was a legal fight over usage rights, then it got split up and partitioned into a bunch of different services, the combined cost of which is pretty expensive and individually each service kind of sucks. So I'm betting we're a few years away from stackoverflow, wikipedia, imdb, goodreads?, etc all having their own ai interfaces. Fine by me. chatgpt is a way better user experience than stackoverflow, and I'd probably use the wikipedia one, don't really care about anything else. It being an "anything tool" is kind of a novelty, I liked that I could make it "write an advertisement for a housecat in the style of a ford truck commercial, and use plenty of made up trademarks and patents for normal cat behaviors/features", but then lost interest when it refused to do the same thing for boobs.
- BlueTemplar 3y agoThese are all sites relying on volunteers to contribute the data, how would that work if their input becomes "invisible" ? Even volunteers want credit / to see the final result... (Mostly unrelated : note that the last two are now owned by Amazon.)
- fardo 3y ago> but then lost interest when it refused to do the same thing for boobs. There’s a cottage industry of unaligned local LLM and image models from company leaks or released retrained public models dedicated specifically around letting you do this for “boobs” (or anything else, for that matter). It’s not hard to find these communities if you go looking (searching for E.G “local models general” will get you in the right direction), but be advised while they’re a great resource, they’re unabashedly NSFW and unabashedly boundary-pushing, as the subset of the populace dedicated to generating their own porn and distributing porn-related models tend to be uniquely dedicated individuals.
- jeremyjacob 3y ago> but then lost interest when it refused to do the same thing for boobs. GTP-4 from the API seems to have no problem with it: Narrator: (Deep, gruff southern voice) [Soundtrack: American Roots Rock tune in the background] (SFX: Eagle's cry) “Presenting the all-new Boobiemax® Wonder Duo! Crafted with the finesse only Mother Nature could inspire! These ain't your average pair of breasts. They're like no other, perfect blend of allure and comfort, built for the hardworking women who seize the day. No fabrication - only 100% organic design with ComfortPlush™ technology, defining the balance between aesthetics and luxury…” and so on.
- supportengineer 3y agoI asked it to plan a date night in my city. What it came up with was... not terrible.
- DrCats 3y agoI would posit that ChatGPT (or perhaps GPT in general) seems to be an "Anything tool" because the task it's trained on, human language, is a sort of "Anything tool" itself.
- HPsquared 3y ago"In the beginning there was the word."
- hyperthesis 3y agoAI hypes every twenty years. Instead of moving goalposts, let's accept that each one succeeded at mechanizing intelligence. Including boolean algebra, writing and arithmetic.
- kromem 3y agoWhen was the last time an AI product reached 100 million users in a few months? I seem to have forgotten that locust cycle.
- itissid 3y agoOne thing that is surprising is just how vast is the chasm on Google/Bing Search quality for developer resources(and probably many engineering field with vast user documentation) and how good ChatGPT-4 has become on it. As a developer ChatGPT and OpenAI API tools are essential and useful. Though if the gap is that big it probably means that other players can chip away at it.
- heroku 3y agoWhy can't someone publish this AI open source and let people build and run it on their own computer again?
- healthyusa 3y agoI have thousands of images and zillions of lines of code out there. AI take it all. Train away. The rewards I get from using these tools is orders of magnitude more valuable than my inputs. Copyright is dumb. It just slows down progress. We’re all in this together, and share away we should. I never made my art to make money, I made art to make cool stuff, not for the almighty $$$.
- falloutx 3y agoIts hard to give up code/art which has gathered a lot of value. Copyright isn't dumb because once your code/art amasses a value of more than $1m or whatever is the threshold for you, you would absolutely want to protect it.
- coltongaodgd 3y agoEngage in effortless interaction with ChatGPT, seamlessly embedded within the Utopia P2P browser. This cutting-edge platform offers unparalleled privacy features, enabling efficient and secure communication for users.
- Someone1234 3y agoAlthough let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the competition for Google Assistant/Siri/Cortana/Alexa was super hot, the responses to the various voice assistance was almost eerie in that it could infer what you wanted/needed. Then as things cooled off, they got gradually dumber/worse every year since then. They're legitimately bad today (Siri in particular, but even Google Assistant is much worse than itself from back then). I suspect it is because hardware costs were too high so they found cheaper models that could run on a potato/locally. From what I've read to get ChatGPT responses in the times it currently takes, they may need to be running something like two 3080 GPUs, 64 GB of RAM, and a high end CPU, with the power draw associated. So to make the economics work (even with ad revenue for example), either a technological breakthrough has to occur to make running today's models much cheaper OR they will cut back making the responses objectively worse (they arguably already did this once with a 3.5 model change).
- sharemywin 3y agoisn't ChatGPT just an investor demo anyway? I doubt they run out of cash at least until they fund building GPT 5. If it flops then who knows.
- stavros 3y agoNo, that was GPT, GPT-2, and GPT-3.
- malwrar 3y agoThe only moat openai has is their financial resources to pay for training and their vast troves of data scraped before reddit/twitter/etc started trying to lock things down. I think at this point there are enough people knowledgeable and interested in open LM technology that, if openai begins to degrade in quality, we won’t lose this technology forever. I’ve personally been obsessively reading papers about new models and transformer modifications, and having succeeded in implementing some I think the experience has taught me that this tech is probably where web was in the 90s: easy in concept, expensive and hard to scale due to a few missing easy pieces thatll come later.
- atemerev 3y agoThis is what we used to call an "AGI" back in 2013-2014. Generalized artificial intelligence. An AI which is general enough to be used for diverse tasks in multiple fields, unlike specialized AIs of that time (e.g. for image recognition or self-driving cars). Nothing more than that. Now the goalposts are moved way forward, and now "AGI" is something that have conscious, with superhuman intelligence, and probably evil intent. But back in a day, AGI was just what we have today -- an universal chat bot. We live in the magic land.
- Sohcahtoa82 3y ago> But back in a day, AGI was just what we have today -- an universal chat bot. I don't know anybody that used AGI to mean that. I've always thought of AGI as an AI that can learn how to perform tasks with very simple instructions, rather than spending weeks training a neural network on a data center full of GPGPUs. Basically, my idea of AGI has always been what you're calling the "new" definition of AGI.
- danielbln 3y agoAren't you effectively describing few-shot learning in LLMs? Teach the model a new task by describing it and maybe give a few examples?
- Kerb_ 3y agoRemember that people have decades of generalized training before becoming able to perform tasks with very simple instructions. I would consider years of pre-training to be acceptable if the model is usable and trainable quickly in practice as well
- ChatGTP 3y agoMostly “training” to do abstract things, not really for survival or fun. I could catch fish since a fairly early age, mostly because I enjoyed it. At about age 12 I used to drive stolen cars around without any training at all except observation. It too me decades to become a corporate robot though :) I think training is a weird word, life is an experience, not just training. I think your comment demonstrated the limitations of language to describe things.
- TySchultz 3y agoI've been building a daily news digest with intent of learning something new and creating something cool. Although AI is the heart of the product, I have noticed that I have been using it as an "anything tool". Though the articles talks about more broad use cases like hardware and robotics, I've found its an anything tool for most of the boring use cases. Using python for the first time in a while, ask GPT. Need a tweak on the app store copy, ask GPT. What are the top news sources, ask GPT. Data formatting & manipulation, save 5 minutes ask GPT. https://apps.apple.com/us/app/quill-news-digest/id1669557131 https://apps.apple.com/us/app/quill-news-digest/id1669557131
- famouswaffles 3y agoLarge Language Models learn to complete arbitrary token sequences, not just about language. https://arxiv.org/abs/2307.04721 https://arxiv.org/abs/2307.04721
- yenwodyah 3y agoWhen you have a hammer…
- josefresco 3y agoTried using Bing ChatGTP to create a logo for me because I saw a cool demo on Instagram. It couldn't write text correctly and I tried numerous times. Not only that, the logos were terrible and didn't "mimic" the image I uploaded as a guide (which was in the demo) Today I needed to transcribe an MP3. Nope, can't do that either. However on Monday I had it write some PHP and that was neat! 50/50 for me.
- kenjackson 3y agoTranscribing an MP3 is a solved problem, in some contexts. What were you doing to make this a fail?
- josefresco 3y agoGave Bing Chat an MP3 URL and it just refused. I know I can do this with software on my PC, or maybe there's an online service where I can upload but I was hoping this would be something Bing Chat/ChatGTP would excel at. I tried first giving it the URL of the article where the MP3 was embedded, and then tried a direct link.
- BoorishBears 3y agoI'm still lost on how you reached the conclusion Bing Chat should do be able that? Does MS imply it can do that somewhere? Did you assume Whisper is from OpenAI => OpenAI is related to Microsoft => Bing Chat can transcribe MP3s? I'm really just lost on the thought process here.
- scrollaway 3y agoPeople hear “ai” and think it’s a near-human that cannot merely answer to text but also see, hear and do shopping.
- josefresco 3y agoYou can upload images to Bing Chat for analysis (see), you can talk to it (hear) and yes, it will also shop for you lol.