7 ms·

Am I the only one who isn't using ChatGPT on a regular basis? Though I've made a few attempts to use it, generally I already know the answers to most trivial q

by sgt 3y ago

Am I the only one who isn't using ChatGPT on a regular basis? Though I've made a few attempts to use it, generally I already know the answers to most trivial questions. And by going into more complex questions and scenarios, it becomes apparent that ChatGPT lacks a deep understanding of the suggestions it provides. It turns out to be a pretty frustrating experience. Also I've noticed that by the time I've crafted intricate prompts, I could have easily skimmed through a few pages of official docs and found a solution. That said, considering the widespread buzz surrounding ChatGPT, it's entirely possible that I may simply be using it incorrectly.

Surplus7421 3y ago
Could you provide an example prompt illustrating the general types of questions that you are asking, with which it seems to struggle?
- sgt 3y ago
  It's been a few weeks ago, but I recall asking ChatGPT about models in Django, which on the surface it often gets perfect. However, when I asked that the model in question needs to be unmanaged and backed by a SQL view (which I pasted), which had previously been applied from a migration script, it became pretty obvious that ChatGPT is just putting together sentences that are highly likely to be true, but not really understanding the architecture of Django. So that's one example - and it's not even a particularly complex scenario.
  - Surplus7421 3y ago
    Can you provide the exact prompt?
    - sgt 3y ago
      I would have to redo it.. I cleaned up all my prompts.
  - gremlinsinc 3y ago
    try asking phind.com, if it brings up outdated info just give it a link to catch up on the latest stuff, it helped me grok next js server actions and convert some code to use server actions over api calls.
  - jenergyla 3y ago
    I've been using version 4 for Django quite a bit and it has saved me quite a bit of coding. I'll just throw in my model and explain in detail what views and serializers to write and the code is near perfect. I didn't have that same experience with GPT 3.5. And wow, is it convenient for writing tests. I just copy the entire views.py in and list all of the tests I want. The key is to be explicit with what you want.
tallowen 3y ago
I've been using it to learn programming concepts I'm unfamiliar with and I perceive a huge difference in quality of chat-gpt answers between: - Salesforce API - AWS - React Native The react native docs are good enough that I really empathize with what you're saying - wouldn't I just be faster reading the docs? However, for salesforce I feel exactly the opposite - the docs are all over the place and message boards tend to be of low quality for the subject. For what it's worth, AWS felt somewhere in the middle depending on the service.
- nirav72 3y ago
  >- Salesforce API I've used it to write unit test methods for Salesforce apex classes & triggers. For the most part, the unit test code it spits out is actually decent. Occasionally I'll have to clean it up a bit or make minor corrections. But it has greatly helped free up some time doing mundane stuff.
gooob 3y ago
no, i don't use it regularly either. my main use case is asking bing bot how to use a linux command that i don't remember off the top of my head.
voidhorse 3y ago
This aligns perfectly with my experience. I feel like I haven't found the correct way to utilize these tools yet as, just like you, whenever I attempt to use them for some intellectual work it proves to be a huge waste of time and the old manual hunt for information and DIY approach is much more effective. I know a lot of people leverage gpt for basic writing tasks. I'm confident in my writing ability and enjoy writing so I don't use it for that either. I might try using image models at some point to generate some pixel art, but on the whole I've found these tools pretty useless and am left wondering what I'm missing. To me it seems like they only work for a very specific case: "I have no background knowledge in domain X and am ok with a quick solution that is adequate but likely not optimal, and I don't need to worry about about correctness." Unfortunately, that's completely out of the realm of the sort of work I like to do.
- balder1991 3y ago
  One thing I like to do with it is telling it to act as a machine that, for every text I paste, it outputs specific academic subjects and their respective courses that dive deep into that subject, so I know where to learn more. Another option is you want to learn about X, you can ask ChatGPT to give you a list of authors or subjects that discuss that topic deeper, which give you a head start, as you can then ask follow up questions such as where their central argument fit in the bigger scheme, which were more influential etc.
djbusby 3y ago
Its not fixing things for me (yet) but I'm using it for brainstorming, for transforming long-form content to tweets, and prototype code generation.
epolanski 3y ago
I have used it at the very beginning but I've been mostly disappointed aside of few examples. Things it proved useful for: - learning new programming languages and solving hackerrank or codewars katas in languages I didn't know. It sped up and lowered my learning curve. - writing recommendation letters based on my inputs ("write a recommendation letter for X, knowing those are his strengths and what he's good at). Things it proved terrible for: - personal assistant. I was hoping to use ChatGPT as a tool that would help me reason. I did try to use it as a companion learning the new Epic Games programming language "Verse". ChatGPT proved absolutely terrible for this task. I would feed him paragraphs about verse calculus, from the pdf and then question him about what I fed him, but have few exchanges more and he would start allucinating. - text editing. I tried to feed him paragraphs and then tell him, modify this or that. But it's capped in the number of tokens and it would make mistakes. Say I told him "modify this line", he would reprint it, but if I told him "hey, so what's paragraph 2", again he would get stale versions - calendar assistant. I would tell him what my priorities and appointments where but it was very unhelpful to provide help and required much too much interaction. In the end I just stopped using it, besides few niche use cases (which haven't happened recently) it just does not provide value where I want to find this value: as an assistant. I'm still moderately optimistic about such use cases being a reality in the future, because the idea of having a companion I can converse with about my daily tasks and work tasks (sort of a jarvis) is really where I expect to find value. Where I don't expect it to provide value is overhyped things like google or bing search, there I want to find websites.
- wahnfrieden 3y ago
  Try w 4
throwuwu 3y ago
It’s very important to qualify this with which model you are using. 3.5 and 4 are very different in the quality of answers they give, the version numbering is deceptive. As a side note, although I can’t confirm this it seems like the general quality of answers for both 3.5 and 4 decreased over time. I suspect they are doing further RLHF which has shown to make the model stupider for lack of a better word.
- sgt 3y ago
  Using 3.5
  - Matticus_Rex 3y ago
    There are probably things you could do to get more out of 3.5, but 4 is waaaaaay, way better for every complex thing I've tried. 3.5 was helpful on some margins when I remembered it, but 4 is a huge part of my workflows now.
  - jarek83 3y ago
    3.5 vs 4 is like day and night. Try to get you impressions on v4
    - Nicholas_C 3y ago
      Not the OP but good to know. How does 4 compare to Bard?
      - muzani 3y ago
        3.5 is arguably better than Bard on some coding use cases and higher accuracy. 4 is just much, much better than both of them.
    - MattPalmer1086 3y ago
      I must have a play with GPT4 at some point. For me, 3.5 generates working code more often than not and rarely hallucinates.
codyb 3y ago
I’ve never even tried it, so no. Docs are usually sufficient, and reworking prompts just doesn’t look that enjoyable to me.
ghaff 3y ago
I've played with it for writing text and it was fine for some boilerplate definitional stuff but I could have copy/pasted that pretty trivially too. Maybe could have saved me a few minutes with some starting point listicle stuff too--but would have taken a bunch of rework and additions to be publishable. Anything that's not straightforward? Even if not flat-out wrong, it lacks any real nuance. I also played with it for some trivial coding and, again, could maybe have saved me a few minutes but nothing earth shattering. So, yeah, maybe I'm doing it wrong but it's not remotely a go-to tool.
- emehex 3y ago
  ChatGPT4 has replaced ~99% of my Google/StackOverflow searches...
  - unshavedyak 3y ago
    Yea, i'm similar. I use GPT4 as a sort of "thought/idea search engine". Where i use Kagi (my Search engine) when i need to find something i already know or when i know there is a concrete answer that i might (definitely) not trust GPT on. However i use GPT when i am exploring new concepts, learning terminology, relationships, etc. GPT is super prone to error on any niche thing, be it subtle nuance or concrete details, but i don't actually find it to be any more or less reliable to the state of the web. Ie most content i consume from the web is full of errors, incorrect facts, etc. I don't find my suspicion of this to be any different than that of GPT. However GPT has a great way of having the relationships to error-ridden results in the web already established. I can ask it about this web of things to get a better understanding when i drill into concrete facts via standard search. GPT is far from a gimmick in my usage, but i also don't think it's revolutionary yet. I use it because i'm exploring and finding it useful, but also because each new leap OpenAI makes is pretty damn impressive. This week they're rolling out Plugins i believe. Every step they take is fun to be with, fun to use, fun to experience. That alone is worth the $20/m for me.
  - chankstein38 3y ago
    I agree. Especially because it seems like, almost simultaneously, Google's result quality has just tanked. It's difficult to find good answers to most things that aren't either behind a paywall or written by the company that made them etc. It helped me debug my tractor last week and helped advise me on the correct part numbers and stuff. Things I was unable to find on the open internet without confusion because everyone just had product pages with zero information.
linsomniac 3y ago
One mistake I've seen people who are unhappy with GPT make is they ask it questions they already know the answer to, topics they are an expert in, and then tear down the response. Try asking it things you would google for. I'm finding it to be a pretty good "I'd like an answer to this question, I don't want to wade through a bunch of google results trying to find the one that answers this question" engine. Sometimes it fails, but often it's really good. This is probably why google has an "all hands on deck about AI" going on right now, I've been using google a lot less lately between bing and chatgpt. However, last week I noticed that Google Bard provides an answer PLUS gives reference links, which ChatGPT can't do.
- elicksaur 3y ago
  If the answer is wrong for topics you do know, why trust the answers for things you don’t? What’s going to be the term for Gell-Mann amnesia[1] but applied to LLMs instead of the media? [1] https://web.archive.org/web/20190808123852/http://larvatus.com/michael-crichton-why-speculate/ https://web.archive.org/web/20190808123852/http://larvatus.c...
  - TeMPOraL 3y ago
    I think GP means people are being too perfectionist with GPT-4. Between things you're an expert on, and things you have no first clue about, there's also a vast space of things you understand to some degree - enough to let you apply intuition, common sense. In this space, you'll be able to spot rather quickly if GPT-4 is making things up, while still benefiting greatly from when it's being right.
    - historygurl 3y ago
      Indeed, plus what are you going to do otherwise? Google it. And how will you vet the pages Google returns? Intuition and common sense, and perhaps an appeal to authority.
      - TeMPOraL 3y ago
mnky9800n 3y ago
Perhaps that is because you haven't used my top ten prompts for software devs who are worried about losing their jobs! But also yes, it is not the greatest thing ever. It is mostly mediocre for me as well. And lots of the most exciting things, like DMing a dungeons and dragons campaign, is actually mostly terrible because it recreates the same scenarios over and over and it never remembers any of the information that a DM should know like anything on your character sheet, when to ask you to roll the dice, or even which version of the rules its supposed to be using even if you ask it directly to use that particular version of the rules. But in fact, it is wild that it has read all of the rules. I was using it to run a star wars saga edition game for myself (I'm not that lonely I promise). And at some point it occured to me this is all copywrited information, is it allowed to read this and then regurgitate it to me? How much of this is fair use? If I wrote a campaign for the system could I regurgitate rules like this in teh campaign book? Who owns all this material that is created not from imagination but from highly complex combinatorics?
- joshuacc 3y ago
  Game rules are generally not copyrightable. The particular written expression of game rules may be, though. So it becomes a question of how much it took from the game books and whether it presented them verbatim or in a new way.
- gautamsomani 3y ago
  Please share your top 10, or for that matter top 20 prompts of ChatGPT.
- hungryforboba 3y ago
  I don't want to come off as self-promoting, but the DnD DM problem is exactly what we're working on solving. If you build the infrastructure around the ruleset and leverage the AI for the story telling then it becomes much better than the vanilla ChatGPT experience, i.e it can reference your character sheet, ask you to roll for skill checks, do combat, etc. It's far from perfect but we're continually working on improving it and you can check it out here if you're interested: https://www.fables.gg/ https://www.fables.gg/
javier123454321 3y ago
I've tried! I want it to fix issues for me in the code. I would love for it to answer questions. But I use it very seldom because it kind of sucks.
herbst 3y ago
Mostly same experience here. However there are a few things I did it worked kinda well for: - Suggesting and writing YouTube shorts scripts about facts. However these turned out very sensationalised and less factual. - I have a website with categories based on real world things. In the past I paid hundreds of dollars for mediocre short descriptions. With ChatGPT I paid $0.04 for good and long descriptions. - I experimented with automated blog articles. It doesn't work for me because of their 'morals' but if you write boring articles about non controversial things without any opinions it works better than any (cheap) paid writer I had so far. So far I haven't had a single coding issue I thought GPT could help me. I don't understand the buzz either.
- MentallyRetired 3y ago
  It's not necessarily about getting help. For me, at least, it's about speed. It can look things up faster than I can. I can copy/paste/modify the results pretty quickly and every now and then it surprises me with a solution I hadn't thought of.
  - herbst 3y ago
    I gave it a try a few hours ago and it hallucinated a method with invalid syntax. Likely better with a more optimized prompt but definitely not what I expected. However it was close, close enough that I could find the right method trough GitHub that I didn't find before. Not bad, not amazing, but I see the potential
jjoonathan 3y ago
ChatGPT has zones of competence and your opinion of ChatGPT is likely to be a function of whether or not its competence zones overlap with what you are doing. Early on, ChatGPT knocked a bunch of highly technical questions I sent it out of the park. It trivially reproduced insight from hours of diving through old forums and gave me further info that eventually proved out. More recently, it has completely hallucinated responses to the last 3 technical questions I gave it in its familiar "invent an API that would be convenient" style. It's the same ChatGPT, but two very different experiences as a function of subject.
- historygurl 3y ago
  > Early on, ChatGPT knocked a bunch of highly technical questions I sent it out of the park I hear this all the time but never with a transcript. I wonder how much experts “read into” responses to make them seem more impressive. (Accidentally you understand, no malice). Or if in the moment it feels impressive but on review it’s mostly banal / platitudes / vague. The few times I’ve used it for precise answers they were wrong in subtle but significant ways.
  - jasonm23 3y ago
    Ask it about cleaning a toilet, and then go deep on bacteria and fungal growth etc. For areas in which you have no expertise, but have a simple understanding, it will grow that knowledge tree. My apologies if you are an expert toilet cleaner, the point is it's more useful than how-to-wiki or YouTube for getting you up and running or refreshing info you may have forgotten. Avoid asking it for VERY obscure things you know nothing about, because it probably doesn't either, but won't say, and the hallucinations start. The falsified stuff can be pretty awful, and it has a tendency to double down.
achenatx 3y ago
I use it extensively. For example I was getting an error that no variable X was on an object. I could see the object in the debugger and could see the variable. I put the section of code into chat gpt and the error and it said that the function I was using returned a collection of the object and I was using it as if it was the actual object. Obviously I had not noticed the punctuation that the debugger used to indicate a collection. I recently was looking for the melting points and hardness of various materials. Obvious ones were easily available, but what is the melting point of brick or granite? I put in a big list of materials and chat gpt got them all. Finally today I had a shopping list and I asked chat GPT to organize by aisle for the store I was going to. It did a bad job, but it was mostly correct. I could easily fix the mistakes myself. I think for many things chat gpt gets 90% of the way there and the 10% you have to fix is no big deal. I really like it for generating sql queries and regular expressions. Two things that take me a lot of time that I do infrequently enough that I can never remember how it works. A friend of mine said he feels like chat GPT will enable a resurgence in the generalist programmer. Im a solid developer but I dont do enough to be completely immersed. Chat GPT is amazingly productive for new areas or areas I dont do that often. Recently Ive used it for linux networking, oauth, some reasonably complex SQL queries, ruby threading, and working with the Ruby 2D API.
simonw 3y ago
Using it effectively is hard. It requires developing pretty strong intuition as to what it's useful for and what it isn't, which means you have to work with it quite a bit before it "clicks" and you start finding it useful. This is pretty unintuitive, especially given the hype around it. Once you DO learn how to use it the productivity boost it can provide for all sorts of different things is substantial. I use it a dozen or so times a day at this point. I've written a bit about how I use it here: https://simonwillison.net/series/using-chatgpt/ https://simonwillison.net/series/using-chatgpt/
jspdown 3y ago
I only use it for fun, just to see how it would respond to a particular question. At the moment, I can't find a serious use case for me. I find it really hard to guide the AI in the direction I want or requires extra work I'm not ready to put in [1]. [1] https://martinfowler.com/articles/2023-chatgpt-xu-hao.html https://martinfowler.com/articles/2023-chatgpt-xu-hao.html
rep_movsd 3y ago
I asked ChatGPT about a very subtle python bug - it had to do with how python default params are actually mutable and will retain their mutated values across invocations. I'd forgotten about this weirdness, but ChatGPT explained it. I also managed to get ChatGPT to write two pieces of fairly complex C++ boilerplate - one was a std:vector that used mmap() and mremap() to grow linearly rather than by a fixed factor (also avoiding memory copy on resize) Then I made it write a vector whose iterator was an integer index rather than a pointer. I made it write all the unit tests and benchmarks for these and it did everything correctly except not knowing that munmap() needs the size parameter rounded to the nearest page. Obviously I hardly managed to get everything correct on a single prompt. It took an iterative conversation and successive refinement
suprjami 3y ago
This is almost the same thing I say to people. To ask GPT about something and be able to determine whether its answer is false or not, you already need to be competent and knowledgeable in that topic. If you're already competent and knowledgeable in a topic, you don't need to ask GPT about it because you already know it. ChatGPT is not an expert, it's not even written to try to be one, it's fancy phone keyboard prediction with a random element added. It's great for creative stuff like "write a cut poem about a puppy" but so far literally every technical question I've asked it has been answered incorrectly.
- d0mine 3y ago
  GPT-4 is decent [1] at generating SQL queries from a short text description. [1] assuming you write raw SQL rarely if ever by hand
ldehaan 3y ago
[dead]
tmaly 3y ago
I have found it to be good in cases where I need a rough outline of something. For programming, I often ask it for some modules in language X that can do Y. Sometimes it surprises me and lists something I have not heard of. Other times it makes stuff up. I think to use it well, you have to be some what of a subject matter expert in the topic you are asking about.
mmchicago 3y ago
I felt that way until I stumbled upon a usage I didn't anticipate: Helping my teenage son with his math and physics homework. Generally, I know the concepts he's working on but I've run into a couple problems. First, they don't use textbooks at his school. It's just poorly made slideshows from the teacher that don't clearly explain any topics. So, to get to parity with his understanding, I'd have to hear the teacher explain it. Second, his work will generally reference names of laws, theorems, etc. that I'm not familiar with. Usually, I'd search for a document, Khan academy video, or some other YouTube video but this has been time-consuming. I started asking ChatGPT: "Take on the role of a high school teacher in an advanced physics course. Give me a basic explanation of [insert law name here]. Please provide at least two examples and a reference URL." This has been incredibly useful.

by sgt 3y ago

Surplus7421 3y ago

Could you provide an example prompt illustrating the general types of questions that you are asking, with which it seems to struggle?

sgt 3y ago
It's been a few weeks ago, but I recall asking ChatGPT about models in Django, which on the surface it often gets perfect. However, when I asked that the model in question needs to be unmanaged and backed by a SQL view (which I pasted), which had previously been applied from a migration script, it became pretty obvious that ChatGPT is just putting together sentences that are highly likely to be true, but not really understanding the architecture of Django. So that's one example - and it's not even a particularly complex scenario.
- Surplus7421 3y ago
  Can you provide the exact prompt?
  - sgt 3y ago
    I would have to redo it.. I cleaned up all my prompts.
- gremlinsinc 3y ago
  try asking phind.com, if it brings up outdated info just give it a link to catch up on the latest stuff, it helped me grok next js server actions and convert some code to use server actions over api calls.
- jenergyla 3y ago
  I've been using version 4 for Django quite a bit and it has saved me quite a bit of coding. I'll just throw in my model and explain in detail what views and serializers to write and the code is near perfect. I didn't have that same experience with GPT 3.5. And wow, is it convenient for writing tests. I just copy the entire views.py in and list all of the tests I want. The key is to be explicit with what you want.

tallowen 3y ago

I've been using it to learn programming concepts I'm unfamiliar with and I perceive a huge difference in quality of chat-gpt answers between: - Salesforce API - AWS - React Native The react native docs are good enough that I really empathize with what you're saying - wouldn't I just be faster reading the docs? However, for salesforce I feel exactly the opposite - the docs are all over the place and message boards tend to be of low quality for the subject. For what it's worth, AWS felt somewhere in the middle depending on the service.

nirav72 3y ago
>- Salesforce API I've used it to write unit test methods for Salesforce apex classes & triggers. For the most part, the unit test code it spits out is actually decent. Occasionally I'll have to clean it up a bit or make minor corrections. But it has greatly helped free up some time doing mundane stuff.

gooob 3y ago

no, i don't use it regularly either. my main use case is asking bing bot how to use a linux command that i don't remember off the top of my head.

voidhorse 3y ago

This aligns perfectly with my experience. I feel like I haven't found the correct way to utilize these tools yet as, just like you, whenever I attempt to use them for some intellectual work it proves to be a huge waste of time and the old manual hunt for information and DIY approach is much more effective. I know a lot of people leverage gpt for basic writing tasks. I'm confident in my writing ability and enjoy writing so I don't use it for that either. I might try using image models at some point to generate some pixel art, but on the whole I've found these tools pretty useless and am left wondering what I'm missing. To me it seems like they only work for a very specific case: "I have no background knowledge in domain X and am ok with a quick solution that is adequate but likely not optimal, and I don't need to worry about about correctness." Unfortunately, that's completely out of the realm of the sort of work I like to do.

balder1991 3y ago
One thing I like to do with it is telling it to act as a machine that, for every text I paste, it outputs specific academic subjects and their respective courses that dive deep into that subject, so I know where to learn more. Another option is you want to learn about X, you can ask ChatGPT to give you a list of authors or subjects that discuss that topic deeper, which give you a head start, as you can then ask follow up questions such as where their central argument fit in the bigger scheme, which were more influential etc.

djbusby 3y ago

Its not fixing things for me (yet) but I'm using it for brainstorming, for transforming long-form content to tweets, and prototype code generation.

epolanski 3y ago

I have used it at the very beginning but I've been mostly disappointed aside of few examples. Things it proved useful for: - learning new programming languages and solving hackerrank or codewars katas in languages I didn't know. It sped up and lowered my learning curve. - writing recommendation letters based on my inputs ("write a recommendation letter for X, knowing those are his strengths and what he's good at). Things it proved terrible for: - personal assistant. I was hoping to use ChatGPT as a tool that would help me reason. I did try to use it as a companion learning the new Epic Games programming language "Verse". ChatGPT proved absolutely terrible for this task. I would feed him paragraphs about verse calculus, from the pdf and then question him about what I fed him, but have few exchanges more and he would start allucinating. - text editing. I tried to feed him paragraphs and then tell him, modify this or that. But it's capped in the number of tokens and it would make mistakes. Say I told him "modify this line", he would reprint it, but if I told him "hey, so what's paragraph 2", again he would get stale versions - calendar assistant. I would tell him what my priorities and appointments where but it was very unhelpful to provide help and required much too much interaction. In the end I just stopped using it, besides few niche use cases (which haven't happened recently) it just does not provide value where I want to find this value: as an assistant. I'm still moderately optimistic about such use cases being a reality in the future, because the idea of having a companion I can converse with about my daily tasks and work tasks (sort of a jarvis) is really where I expect to find value. Where I don't expect it to provide value is overhyped things like google or bing search, there I want to find websites.

wahnfrieden 3y ago
Try w 4

throwuwu 3y ago

It’s very important to qualify this with which model you are using. 3.5 and 4 are very different in the quality of answers they give, the version numbering is deceptive. As a side note, although I can’t confirm this it seems like the general quality of answers for both 3.5 and 4 decreased over time. I suspect they are doing further RLHF which has shown to make the model stupider for lack of a better word.

sgt 3y ago
Using 3.5
- Matticus_Rex 3y ago
  There are probably things you could do to get more out of 3.5, but 4 is waaaaaay, way better for every complex thing I've tried. 3.5 was helpful on some margins when I remembered it, but 4 is a huge part of my workflows now.
- jarek83 3y ago
  3.5 vs 4 is like day and night. Try to get you impressions on v4
  - Nicholas_C 3y ago
    Not the OP but good to know. How does 4 compare to Bard?
    - muzani 3y ago
      3.5 is arguably better than Bard on some coding use cases and higher accuracy. 4 is just much, much better than both of them.
  - MattPalmer1086 3y ago
    I must have a play with GPT4 at some point. For me, 3.5 generates working code more often than not and rarely hallucinates.

codyb 3y ago

I’ve never even tried it, so no. Docs are usually sufficient, and reworking prompts just doesn’t look that enjoyable to me.

ghaff 3y ago

I've played with it for writing text and it was fine for some boilerplate definitional stuff but I could have copy/pasted that pretty trivially too. Maybe could have saved me a few minutes with some starting point listicle stuff too--but would have taken a bunch of rework and additions to be publishable. Anything that's not straightforward? Even if not flat-out wrong, it lacks any real nuance. I also played with it for some trivial coding and, again, could maybe have saved me a few minutes but nothing earth shattering. So, yeah, maybe I'm doing it wrong but it's not remotely a go-to tool.

emehex 3y ago
ChatGPT4 has replaced ~99% of my Google/StackOverflow searches...
- unshavedyak 3y ago
  Yea, i'm similar. I use GPT4 as a sort of "thought/idea search engine". Where i use Kagi (my Search engine) when i need to find something i already know or when i know there is a concrete answer that i might (definitely) not trust GPT on. However i use GPT when i am exploring new concepts, learning terminology, relationships, etc. GPT is super prone to error on any niche thing, be it subtle nuance or concrete details, but i don't actually find it to be any more or less reliable to the state of the web. Ie most content i consume from the web is full of errors, incorrect facts, etc. I don't find my suspicion of this to be any different than that of GPT. However GPT has a great way of having the relationships to error-ridden results in the web already established. I can ask it about this web of things to get a better understanding when i drill into concrete facts via standard search. GPT is far from a gimmick in my usage, but i also don't think it's revolutionary yet. I use it because i'm exploring and finding it useful, but also because each new leap OpenAI makes is pretty damn impressive. This week they're rolling out Plugins i believe. Every step they take is fun to be with, fun to use, fun to experience. That alone is worth the $20/m for me.
- chankstein38 3y ago
  I agree. Especially because it seems like, almost simultaneously, Google's result quality has just tanked. It's difficult to find good answers to most things that aren't either behind a paywall or written by the company that made them etc. It helped me debug my tractor last week and helped advise me on the correct part numbers and stuff. Things I was unable to find on the open internet without confusion because everyone just had product pages with zero information.

linsomniac 3y ago

One mistake I've seen people who are unhappy with GPT make is they ask it questions they already know the answer to, topics they are an expert in, and then tear down the response. Try asking it things you would google for. I'm finding it to be a pretty good "I'd like an answer to this question, I don't want to wade through a bunch of google results trying to find the one that answers this question" engine. Sometimes it fails, but often it's really good. This is probably why google has an "all hands on deck about AI" going on right now, I've been using google a lot less lately between bing and chatgpt. However, last week I noticed that Google Bard provides an answer PLUS gives reference links, which ChatGPT can't do.

elicksaur 3y ago
If the answer is wrong for topics you do know, why trust the answers for things you don’t? What’s going to be the term for Gell-Mann amnesia[1] but applied to LLMs instead of the media? [1] https://web.archive.org/web/20190808123852/http://larvatus.com/michael-crichton-why-speculate/ https://web.archive.org/web/20190808123852/http://larvatus.c...
- TeMPOraL 3y ago
  I think GP means people are being too perfectionist with GPT-4. Between things you're an expert on, and things you have no first clue about, there's also a vast space of things you understand to some degree - enough to let you apply intuition, common sense. In this space, you'll be able to spot rather quickly if GPT-4 is making things up, while still benefiting greatly from when it's being right.
  - historygurl 3y ago
    Indeed, plus what are you going to do otherwise? Google it. And how will you vet the pages Google returns? Intuition and common sense, and perhaps an appeal to authority.
    - TeMPOraL 3y ago

mnky9800n 3y ago

Perhaps that is because you haven't used my top ten prompts for software devs who are worried about losing their jobs! But also yes, it is not the greatest thing ever. It is mostly mediocre for me as well. And lots of the most exciting things, like DMing a dungeons and dragons campaign, is actually mostly terrible because it recreates the same scenarios over and over and it never remembers any of the information that a DM should know like anything on your character sheet, when to ask you to roll the dice, or even which version of the rules its supposed to be using even if you ask it directly to use that particular version of the rules. But in fact, it is wild that it has read all of the rules. I was using it to run a star wars saga edition game for myself (I'm not that lonely I promise). And at some point it occured to me this is all copywrited information, is it allowed to read this and then regurgitate it to me? How much of this is fair use? If I wrote a campaign for the system could I regurgitate rules like this in teh campaign book? Who owns all this material that is created not from imagination but from highly complex combinatorics?

joshuacc 3y ago
Game rules are generally not copyrightable. The particular written expression of game rules may be, though. So it becomes a question of how much it took from the game books and whether it presented them verbatim or in a new way.
gautamsomani 3y ago
Please share your top 10, or for that matter top 20 prompts of ChatGPT.
hungryforboba 3y ago
I don't want to come off as self-promoting, but the DnD DM problem is exactly what we're working on solving. If you build the infrastructure around the ruleset and leverage the AI for the story telling then it becomes much better than the vanilla ChatGPT experience, i.e it can reference your character sheet, ask you to roll for skill checks, do combat, etc. It's far from perfect but we're continually working on improving it and you can check it out here if you're interested: https://www.fables.gg/ https://www.fables.gg/

javier123454321 3y ago

I've tried! I want it to fix issues for me in the code. I would love for it to answer questions. But I use it very seldom because it kind of sucks.

herbst 3y ago

Mostly same experience here. However there are a few things I did it worked kinda well for: - Suggesting and writing YouTube shorts scripts about facts. However these turned out very sensationalised and less factual. - I have a website with categories based on real world things. In the past I paid hundreds of dollars for mediocre short descriptions. With ChatGPT I paid $0.04 for good and long descriptions. - I experimented with automated blog articles. It doesn't work for me because of their 'morals' but if you write boring articles about non controversial things without any opinions it works better than any (cheap) paid writer I had so far. So far I haven't had a single coding issue I thought GPT could help me. I don't understand the buzz either.

MentallyRetired 3y ago
It's not necessarily about getting help. For me, at least, it's about speed. It can look things up faster than I can. I can copy/paste/modify the results pretty quickly and every now and then it surprises me with a solution I hadn't thought of.
- herbst 3y ago
  I gave it a try a few hours ago and it hallucinated a method with invalid syntax. Likely better with a more optimized prompt but definitely not what I expected. However it was close, close enough that I could find the right method trough GitHub that I didn't find before. Not bad, not amazing, but I see the potential

jjoonathan 3y ago

ChatGPT has zones of competence and your opinion of ChatGPT is likely to be a function of whether or not its competence zones overlap with what you are doing. Early on, ChatGPT knocked a bunch of highly technical questions I sent it out of the park. It trivially reproduced insight from hours of diving through old forums and gave me further info that eventually proved out. More recently, it has completely hallucinated responses to the last 3 technical questions I gave it in its familiar "invent an API that would be convenient" style. It's the same ChatGPT, but two very different experiences as a function of subject.

historygurl 3y ago
> Early on, ChatGPT knocked a bunch of highly technical questions I sent it out of the park I hear this all the time but never with a transcript. I wonder how much experts “read into” responses to make them seem more impressive. (Accidentally you understand, no malice). Or if in the moment it feels impressive but on review it’s mostly banal / platitudes / vague. The few times I’ve used it for precise answers they were wrong in subtle but significant ways.
- jasonm23 3y ago
  Ask it about cleaning a toilet, and then go deep on bacteria and fungal growth etc. For areas in which you have no expertise, but have a simple understanding, it will grow that knowledge tree. My apologies if you are an expert toilet cleaner, the point is it's more useful than how-to-wiki or YouTube for getting you up and running or refreshing info you may have forgotten. Avoid asking it for VERY obscure things you know nothing about, because it probably doesn't either, but won't say, and the hallucinations start. The falsified stuff can be pretty awful, and it has a tendency to double down.

achenatx 3y ago

I use it extensively. For example I was getting an error that no variable X was on an object. I could see the object in the debugger and could see the variable. I put the section of code into chat gpt and the error and it said that the function I was using returned a collection of the object and I was using it as if it was the actual object. Obviously I had not noticed the punctuation that the debugger used to indicate a collection. I recently was looking for the melting points and hardness of various materials. Obvious ones were easily available, but what is the melting point of brick or granite? I put in a big list of materials and chat gpt got them all. Finally today I had a shopping list and I asked chat GPT to organize by aisle for the store I was going to. It did a bad job, but it was mostly correct. I could easily fix the mistakes myself. I think for many things chat gpt gets 90% of the way there and the 10% you have to fix is no big deal. I really like it for generating sql queries and regular expressions. Two things that take me a lot of time that I do infrequently enough that I can never remember how it works. A friend of mine said he feels like chat GPT will enable a resurgence in the generalist programmer. Im a solid developer but I dont do enough to be completely immersed. Chat GPT is amazingly productive for new areas or areas I dont do that often. Recently Ive used it for linux networking, oauth, some reasonably complex SQL queries, ruby threading, and working with the Ruby 2D API.

simonw 3y ago

Using it effectively is hard. It requires developing pretty strong intuition as to what it's useful for and what it isn't, which means you have to work with it quite a bit before it "clicks" and you start finding it useful. This is pretty unintuitive, especially given the hype around it. Once you DO learn how to use it the productivity boost it can provide for all sorts of different things is substantial. I use it a dozen or so times a day at this point. I've written a bit about how I use it here: https://simonwillison.net/series/using-chatgpt/ https://simonwillison.net/series/using-chatgpt/

jspdown 3y ago

I only use it for fun, just to see how it would respond to a particular question. At the moment, I can't find a serious use case for me. I find it really hard to guide the AI in the direction I want or requires extra work I'm not ready to put in [1]. [1] https://martinfowler.com/articles/2023-chatgpt-xu-hao.html https://martinfowler.com/articles/2023-chatgpt-xu-hao.html

rep_movsd 3y ago

I asked ChatGPT about a very subtle python bug - it had to do with how python default params are actually mutable and will retain their mutated values across invocations. I'd forgotten about this weirdness, but ChatGPT explained it. I also managed to get ChatGPT to write two pieces of fairly complex C++ boilerplate - one was a std:vector that used mmap() and mremap() to grow linearly rather than by a fixed factor (also avoiding memory copy on resize) Then I made it write a vector whose iterator was an integer index rather than a pointer. I made it write all the unit tests and benchmarks for these and it did everything correctly except not knowing that munmap() needs the size parameter rounded to the nearest page. Obviously I hardly managed to get everything correct on a single prompt. It took an iterative conversation and successive refinement

suprjami 3y ago

This is almost the same thing I say to people. To ask GPT about something and be able to determine whether its answer is false or not, you already need to be competent and knowledgeable in that topic. If you're already competent and knowledgeable in a topic, you don't need to ask GPT about it because you already know it. ChatGPT is not an expert, it's not even written to try to be one, it's fancy phone keyboard prediction with a random element added. It's great for creative stuff like "write a cut poem about a puppy" but so far literally every technical question I've asked it has been answered incorrectly.

d0mine 3y ago
GPT-4 is decent [1] at generating SQL queries from a short text description. [1] assuming you write raw SQL rarely if ever by hand

ldehaan 3y ago

[dead]

tmaly 3y ago

I have found it to be good in cases where I need a rough outline of something. For programming, I often ask it for some modules in language X that can do Y. Sometimes it surprises me and lists something I have not heard of. Other times it makes stuff up. I think to use it well, you have to be some what of a subject matter expert in the topic you are asking about.

mmchicago 3y ago

I felt that way until I stumbled upon a usage I didn't anticipate: Helping my teenage son with his math and physics homework. Generally, I know the concepts he's working on but I've run into a couple problems. First, they don't use textbooks at his school. It's just poorly made slideshows from the teacher that don't clearly explain any topics. So, to get to parity with his understanding, I'd have to hear the teacher explain it. Second, his work will generally reference names of laws, theorems, etc. that I'm not familiar with. Usually, I'd search for a document, Khan academy video, or some other YouTube video but this has been time-consuming. I started asking ChatGPT: "Take on the role of a high school teacher in an advanced physics course. Give me a basic explanation of [insert law name here]. Please provide at least two examples and a reference URL." This has been incredibly useful.