9 ms·

I have been using ChatGPT pretty consistently for various summarization type of tasks for a bit. But this weekend I tried using it to learn a technical concept

I have been using ChatGPT pretty consistently for various summarization type of tasks for a bit. But this weekend I tried using it to learn a technical concept I've struggled with. Previously, I could have spoken about the topic intelligently, but could not have put it to any practical use. In a few quick prompts and < 1 hr of synthesis, I was able to get to practical knowledge to understand it enough to try and build something from scratch. And building something in another 2-3 hours. I think this would have taken me 1-2 months of dedicate hitting my head on the wall time to learn previously. In that moment, I had a realization similar to the author. Basic flow of the prompt went something like: 1. Tell me about this thing that's part of the concept that I don't understand? 2. Explain this new term you threw at me? Do it in the context on an industry I understand well. 3. Explain yet another concept you mentioned in the same industry concept? 4. This sounds a lot like this other concept I know. How do these differ in practical use? 5. Write me some sample code in a language I understand very well, that does X use case the you mentioned? 6. What are these other concepts you used? 7. How can I do this new concept to do something I've implemented before. Essentially, my knowledge of other concepts, industries and being able to draw parallels allowed me to learn much much faster.

Cyberthal 3y ago
Bullshido blitzkrieg
quonn 3y ago
I find these stories strange. If it would be like that, wouldn‘t you learn from an expert friend just as fast? My experience is that for anything worth learning it takes a long time of thinking about it from various angles.
- mdrzn 3y ago
  Is there an endless amount of "expert friend" available somewhere that I don't know about? How can I learn something none of my friends know about?
  - ThalesX 3y ago
    A book?
    - layer8 3y ago
      You can’t ask a book for clarification, or to fill the gaps it leaves.
      - ThalesX 3y ago
        I agree, but it does give you access to field experts. Presumably a good book covers the learning approach in whichever way they consider it best, from the position of a field expert. It's also vetted information by other field experts. At the moment, ChatGPT's answers are between ChatGPT and the asker. It can be a total hallucination or useful information. By the way, my point is just that access to experts is not that hard if you consider their books as access.
- jimbokun 3y ago
  Now ChatGPT is everyone's "expert friend".
- dougmwne 3y ago
  While you are seeking out your expert friends and taking a long time, I’ll be on to the next 10 things I can learn from the AI.
  - quonn 3y ago
    I‘m talking to them while I‘m having a drink on the beach. How about that?
    - renewiltord 3y ago
      I mean, if you have an expert friend on any arbitrary topic that you meet at the beach with a drink in the moment, then more power to you! Perhaps try using ChatGPT to identify whether others can do this and for some ideas for why they may not be able to.
- ericmcer 3y ago
  You have expert friends who will spend an hour tutoring you on demand? Are you a celebrity or something?
valenterry 3y ago
So what was this concept that you struggled with?
- arolihas 3y ago
  I feel like I always hear these types of anecdotes pop up but concrete specific details are rarely shared.
  - roflyear 3y ago
    Yeah. Makes me wonder if it was a really simple concept, like something like recursion or something.
    - anongraddebt 3y ago
      Also, even for relatively simple and beginner things like recursion there are levels of understanding. Some of the anecdotes I see from people using ChatGPT to learn a new technical concept suggest a false sense of deeper levels of understanding. This seems to be a result of beginner naïveté together with the supreme confidence ChatGPT exudes.
  - kirrent 3y ago
    I've been using GPT-4 ChatGPT in this style so here's my specific use case. I'm currently studying MIT's 8370 quantum information science course because my background in understanding the fundamentals of error correction is pretty poor and I need it for work. I have a bachelors in physics but I wasn't a great maths student in university (the folly of youth) so my linear algebra could be better. On the other hand, I'm not going to redo linear algebra as a pre-requisite for this course. I also haven't done the pre-requisite QC courses, though I'm much more comfortable in that domain. I don't use ChatGPT to teach me the concepts in the course, I use it as an empathetic tutor to fill in the blanks. If I see a linear algebra identity I'm not familiar with I can ask ChatGPT, telling it the context and what I already know and it'll give me an answer in that context with solid mathematical grounding but (because I ask for it) intuitive justifications. The alternative of stackoverflow, wikipedia, or other online notes can be hard to search for and even when I find a good answer, it's often needlessly mathy and complex. I don't have to worry about hallucination because it is just a gap and once I understand it, usually what the lecturer said makes perfect sense. In one lecture the lecturer had a throw-away line about stabilisers corresponding to syndrome measurements but didn't elaborate much further. I didn't get how the corresponding circuit would be constructed so I went to ChatGPT and asked it my question with what I knew and the context. In that case, it pushed back and told me that what I knew was wrong. I had formed a misconception which it corrected and gave me an example to show why I was wrong. I guess I don't use it as a lecturer, or even a tutor. I use it as the really smart guy you sit next to in a lecture theatre who you whisper questions to.
dougmwne 3y ago
I had a similar experience recently having GPT-4 teach me double entry bookkeeping for a project I am ramping up on. I understood it in a vague sense from some Google, but wouldn’t be able to talk to a room full of accountants in an intelligent way. I was feeling like I needed to read a book or take a class, but tried using GPT-4 as an interactive tutor instead. After about an hour of delving into details, examples, history and analogies, I get it on a deep level that would have normally taken many days to develop. Most of the material I found before took an approach of “just memorize this” while I was able to get more parsimonious core concepts that were hidden behind 500 years of practice.
- quonn 3y ago
  Teach yourself French in 10 days. It includes 3 audio books which make it really easy, too.
- Animats 3y ago
  That's a good case. There's a huge amount of material available on double-entry bookkeeping, and there's a consensus on how it works. So a LLM can handle that.
- drc500free 3y ago
  That's great! I didn't really grok accrual accounting until I realized that non cash "accounts" are explanations for why there is extra cash or missing cash. When I stopped thinking about an asset as "a truck" and started thinking of it as "an explanation that cash is missing because we bought a truck" it all clicked. The "asset account" is the negative space around the asset, so that it can be matched up later with the cash that is generated by using up the asset. Before that I thought of assets and revenues as good, and liabilities and expenses as bad. I couldn't make sense of why an asset turns into a liability instead of into a revenue. It's similar to learning intro physics and trying to understand how anything moves if every force generates an opposite force.
Parmenidea 3y ago
I’d be worried about using ChatGPT for this given its tendency to hallucinate. Did you have to verify each response it gave you, or were you willing to trust the output?
- metalliqaz 3y ago
  I wouldn't verify each response, but I would (sorta) verify at the end by moving on to read within the topic. As you start to grok a topic, you should start to recognize internal inconsistencies and then you can probe those details. When you feel you have some understanding, you can then start consuming media on the topic at a higher level and any problems in your understanding should fall out fairly quickly.
FredPret 3y ago
This is exactly what I do as well. I make it generate sample code, ask it lots of "why" questions, ask it to help with the bugs. You can learn a huge amount in record time. Is this the end of Stackoverflow?
seba_dos1 3y ago
I have recently spent a lot of time reading the USB 2.0 specification and figuring out a race condition I've been experiencing that turned out to actually be a deficiency of the spec itself. Just for fun, I asked ChatGPT about it. This is its answer: > According to the USB 2.0 specification, a remote wakeup event can be initiated by a USB device in a suspended state to request the host to resume communication with the device. The device sends a remote wakeup signal to its hub, which then forwards the signal to the host. > However, if the upstream port of the hub is in the process of being suspended, the remote wakeup signal from the downstream device may not be received by the host. This is because the hub's upstream port is no longer able to forward the signal to the host during the suspension process. > As a result, the remote wakeup event may fail, and the downstream device may remain in a suspended state indefinitely. To avoid this race condition, the USB 2.0 specification recommends that downstream devices wait for a certain period before sending a remote wakeup signal to their hubs to ensure that the hub's upstream port is not being suspended. It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. We're going to see a huge spike of confidently wrong "experts" if they learn by asking ChatGPT to explain things to them.
- orbifold 3y ago
  I've tested it on a various areas of expertise in physics that I am familiar with and it often makes up content which sounds very plausible except that it gets pretty much always details wrong in very important ways. On the other hand I've found it very useful in providing reference articles.
- PragmaticPulp 3y ago
  > It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. This has been my experience as well. As I work on some projects I’ve been asking it basic questions for things I’ve already learned or know quite well. Once you deviate past the basics (content you’d find in common tutorials) the hallucination rate is out of control. The surreal part is that it all sounds so confident and plausible. The way it puts words together satisfies my “that sounds about right” reflex, but the actual content is incorrect or illogical quite frequently. If I continue prodding I can get it to come up the right answer many times, but I have to ask it a lot of leading questions to get there.
ericmcer 3y ago
This was the killer feature to me. It is an amazing teacher, it’s like having an infinitely patient mentor who can answer all your questions. I feel way more confident about being able to onboard with unfamiliar tech and be able to become productive quickly.
ornornor 3y ago
How can you trust it though? It notoriously hallucinates and makes things up in an otherwise plausible way, and the only way to tell is to already be knowledgeable about what you’re asking about… which makes the whole exercise useless.
- j2bax 3y ago
  I recently asked it to provide the meaning of a song. I gave it the artist and the song. It responded with a thoughtful(ish) explanation of the potential meaning of the song. I then asked it if it could share the lyrics of the song with me because some of the quotes it referenced weren't in the song. It provided lyrics that weren't the song I was asking about at all. I then asked what song it was, it apologized and it told me the artist and title of the song... I looked up that artist and title and it was also not the song that it provided the lyrics/meaning to. Ultimately, I determined that it just made up lyrics to a song. It was very apologetic.
  - throwaway4aday 3y ago
    Sounds like turbo aka the free (and dumb) 3.5 version. GPT-4 would deliver better results with less hallucination but if you want it to really work well you should include the lyrics in the prompt as context. It's not a search engine, if you want that then use Bing.
- RecycledEle 3y ago
  Q: How can You trust StackOverlfow, Reddit, or anything on the Internet? The stuff I read online is often wrong. A: Check it yourself.
  - j2bax 3y ago
    Well, you still can't trust everything, but voting mechanisms tend to help filter out some of the BS.
    - robocat 3y ago
angarg12 3y ago
Lately I've been reading seminal papers in distributed systems, and it occurred to me that there is a niche for LLMs to "democratize" highly technical content. Trying to understand those papers is a slog, but it doesn't need to be. For a casual reader, you could drop all the jargon and proofs. You could also rewrite the concepts in a less obscure and obtuse prose to make it more accessible, without dumbing down the content. Essentially it's like having a tech writer/communicator in your browser.
- codeisawesome 3y ago
  I often wonder why academia loves obtuse prose; the only dialect I’ve seen equally enthusiastically adopted while being just as useless is Corpspeak.
marak830 3y ago
This has been my experience as well. Also, I'm less afraid to ask the stupid questions, as I would be if asking a more experienced dev.
roflyear 3y ago
What was the thing? It's really hard to put a value to what you're describing without knowing what it was.
ModernMech 3y ago
I did the same thing to implement an automatic differentiation engine in my programming languages project. I’ve been mulling over the idea for months, but I was a little unsure how to do it. Sat down with chatgpt and we got it done mostly in an afternoon. I’m so stoked about the future of coding.
moneywoes 3y ago
How do you account for hallucinations
potsandpans 3y ago
how do you know what it's telling you is correct and not (as the open ai paper has labeled it) "hallucinating" answers? over Christmas, I used chat gpt to model a statically typed language I was working on. it was helpful, but a lot of what it gave me was sort of, in a very deceptive way wrong. It's tough to describe what I mean by "wrong" here. Nothing it spit out was just 100% blatantly incorrect. instead it was subtly incorrect, and gave inconsistent evaluations / overviews. not knowing a bit about type theory, I wouldn't actually be able to evaluate how good the information I got out of it was. I'd be hesitant to take anything chatgpt gave me at face value, or feel confident in being able to speak precisely about a given topic. did you run into similar problems? and if so, how'd you overcome them?
- Baeocystin 3y ago
  Not the person you asked, but chiming in. Two things: First, GPT-4 is far more capable than 3.5 when it comes to not hallucinating. The 'house style' of the response is, of course, very similar, which can lead one to initially think that the difference between the two models isn't as significant as it is. Since I don't have API access yet, I do a lot of my initial exploration in 3.5-Legacy, and once I've narrowed things down a bit, I have GPT-4 review and correct 3.5's final answers. It works very well. Second, and this is more of a meta comment: How people use ChatGPT really exposes whether they use sanity checks as a regular part of their problem solving technique. That all of the LLMs are confidently incorrect at times doesn't slow me down much at all, and sometimes their errors actually help me think of new directions to explore, even if 'wrong' on first take. Conversely, several of my friends find the error rate to be an impediment to how they work. They're good at what they do, they're just more used to taking things as correct and running with it for a longer length of time before checking to see if what they're doing makes sense. I do think that people who are put off by this significantly underestimate how often people are confidently incorrect as well. There's a reason the saying trust but verify is a common one.
  - throwaway4aday 3y ago
    Third, you just glue a vector search onto it and stuff it with a bunch of textbooks and documentation.
waynesonfire 3y ago
Yeah, chatGPT is a game changer for learning. Because of chatGTP, "I know GNU Make." I just love that it's helping me conquer technologies that I've been wanting to learn my entire career but I guess just didn't have the discipline to do it. I'm getting an itch to tackle Emacs next.
SoftTalker 3y ago
I find hitting my head on the wall is essential for learning. Hitting my head on the wall helps me learn where the boundaries of functionality are, or where the actual operation of the thing differs from my intuition. If I just learn some example use cases that work, that tells me something but I won't really have mastered whatever it was that I set out to learn.
UlisesAC4 3y ago
Yeah, Chatgpt is as good as you know how to ask it questions. I have been studying clojure concepts and I too would have taken more time than an afternoon in understanding them.
nonethewiser 3y ago
> This sounds a lot like this other concept I know. How do these differ in practical use? How did you test the answer it gave you? It sounds like the end product is code, which is testable, so maybe it’s moot. But that type of question is not useful in my experience because you can’t really disprove it’s answer. It’s also a leading question. It might not be similar to that other thing at all, but ChatGPT would manufacture something in that case.
kneebonian 3y ago
Only if you can trust what it told you. And how can you trust it without knowing the material. For all you know you've not only not gained knowledge you've worse gained false knowledge.
deleted 3y ago
[deleted]
yeck 3y ago
I have had many similar experiences with just the free version of ChatGPT. Now I also ask questions more immediately. In the past, when a bunch of new terms or concepts were thrown at me, the effort to stop and dig into each of them via Google search was great enough that I would leave many stones unturned. When using GPT while studying up on a new thing, the friction is very little so fewer stones left unturned. And like you mentioned, being able to translate concepts from a novel domain to one I'm very familiar with can be like translating an unfamiliar language to english.
alecfreudenberg 3y ago
this is the way

Cyberthal 3y ago

Bullshido blitzkrieg

quonn 3y ago

I find these stories strange. If it would be like that, wouldn‘t you learn from an expert friend just as fast? My experience is that for anything worth learning it takes a long time of thinking about it from various angles.

mdrzn 3y ago
Is there an endless amount of "expert friend" available somewhere that I don't know about? How can I learn something none of my friends know about?
- ThalesX 3y ago
  A book?
  - layer8 3y ago
    You can’t ask a book for clarification, or to fill the gaps it leaves.
    - ThalesX 3y ago
      I agree, but it does give you access to field experts. Presumably a good book covers the learning approach in whichever way they consider it best, from the position of a field expert. It's also vetted information by other field experts. At the moment, ChatGPT's answers are between ChatGPT and the asker. It can be a total hallucination or useful information. By the way, my point is just that access to experts is not that hard if you consider their books as access.
jimbokun 3y ago
Now ChatGPT is everyone's "expert friend".
dougmwne 3y ago
While you are seeking out your expert friends and taking a long time, I’ll be on to the next 10 things I can learn from the AI.
- quonn 3y ago
  I‘m talking to them while I‘m having a drink on the beach. How about that?
  - renewiltord 3y ago
    I mean, if you have an expert friend on any arbitrary topic that you meet at the beach with a drink in the moment, then more power to you! Perhaps try using ChatGPT to identify whether others can do this and for some ideas for why they may not be able to.
ericmcer 3y ago
You have expert friends who will spend an hour tutoring you on demand? Are you a celebrity or something?

valenterry 3y ago

So what was this concept that you struggled with?

arolihas 3y ago
I feel like I always hear these types of anecdotes pop up but concrete specific details are rarely shared.
- roflyear 3y ago
  Yeah. Makes me wonder if it was a really simple concept, like something like recursion or something.
  - anongraddebt 3y ago
    Also, even for relatively simple and beginner things like recursion there are levels of understanding. Some of the anecdotes I see from people using ChatGPT to learn a new technical concept suggest a false sense of deeper levels of understanding. This seems to be a result of beginner naïveté together with the supreme confidence ChatGPT exudes.
- kirrent 3y ago
  I've been using GPT-4 ChatGPT in this style so here's my specific use case. I'm currently studying MIT's 8370 quantum information science course because my background in understanding the fundamentals of error correction is pretty poor and I need it for work. I have a bachelors in physics but I wasn't a great maths student in university (the folly of youth) so my linear algebra could be better. On the other hand, I'm not going to redo linear algebra as a pre-requisite for this course. I also haven't done the pre-requisite QC courses, though I'm much more comfortable in that domain. I don't use ChatGPT to teach me the concepts in the course, I use it as an empathetic tutor to fill in the blanks. If I see a linear algebra identity I'm not familiar with I can ask ChatGPT, telling it the context and what I already know and it'll give me an answer in that context with solid mathematical grounding but (because I ask for it) intuitive justifications. The alternative of stackoverflow, wikipedia, or other online notes can be hard to search for and even when I find a good answer, it's often needlessly mathy and complex. I don't have to worry about hallucination because it is just a gap and once I understand it, usually what the lecturer said makes perfect sense. In one lecture the lecturer had a throw-away line about stabilisers corresponding to syndrome measurements but didn't elaborate much further. I didn't get how the corresponding circuit would be constructed so I went to ChatGPT and asked it my question with what I knew and the context. In that case, it pushed back and told me that what I knew was wrong. I had formed a misconception which it corrected and gave me an example to show why I was wrong. I guess I don't use it as a lecturer, or even a tutor. I use it as the really smart guy you sit next to in a lecture theatre who you whisper questions to.

dougmwne 3y ago

I had a similar experience recently having GPT-4 teach me double entry bookkeeping for a project I am ramping up on. I understood it in a vague sense from some Google, but wouldn’t be able to talk to a room full of accountants in an intelligent way. I was feeling like I needed to read a book or take a class, but tried using GPT-4 as an interactive tutor instead. After about an hour of delving into details, examples, history and analogies, I get it on a deep level that would have normally taken many days to develop. Most of the material I found before took an approach of “just memorize this” while I was able to get more parsimonious core concepts that were hidden behind 500 years of practice.

quonn 3y ago
Teach yourself French in 10 days. It includes 3 audio books which make it really easy, too.
Animats 3y ago
That's a good case. There's a huge amount of material available on double-entry bookkeeping, and there's a consensus on how it works. So a LLM can handle that.
drc500free 3y ago
That's great! I didn't really grok accrual accounting until I realized that non cash "accounts" are explanations for why there is extra cash or missing cash. When I stopped thinking about an asset as "a truck" and started thinking of it as "an explanation that cash is missing because we bought a truck" it all clicked. The "asset account" is the negative space around the asset, so that it can be matched up later with the cash that is generated by using up the asset. Before that I thought of assets and revenues as good, and liabilities and expenses as bad. I couldn't make sense of why an asset turns into a liability instead of into a revenue. It's similar to learning intro physics and trying to understand how anything moves if every force generates an opposite force.

Parmenidea 3y ago

I’d be worried about using ChatGPT for this given its tendency to hallucinate. Did you have to verify each response it gave you, or were you willing to trust the output?

metalliqaz 3y ago
I wouldn't verify each response, but I would (sorta) verify at the end by moving on to read within the topic. As you start to grok a topic, you should start to recognize internal inconsistencies and then you can probe those details. When you feel you have some understanding, you can then start consuming media on the topic at a higher level and any problems in your understanding should fall out fairly quickly.

FredPret 3y ago

This is exactly what I do as well. I make it generate sample code, ask it lots of "why" questions, ask it to help with the bugs. You can learn a huge amount in record time. Is this the end of Stackoverflow?

seba_dos1 3y ago

I have recently spent a lot of time reading the USB 2.0 specification and figuring out a race condition I've been experiencing that turned out to actually be a deficiency of the spec itself. Just for fun, I asked ChatGPT about it. This is its answer: > According to the USB 2.0 specification, a remote wakeup event can be initiated by a USB device in a suspended state to request the host to resume communication with the device. The device sends a remote wakeup signal to its hub, which then forwards the signal to the host. > However, if the upstream port of the hub is in the process of being suspended, the remote wakeup signal from the downstream device may not be received by the host. This is because the hub's upstream port is no longer able to forward the signal to the host during the suspension process. > As a result, the remote wakeup event may fail, and the downstream device may remain in a suspended state indefinitely. To avoid this race condition, the USB 2.0 specification recommends that downstream devices wait for a certain period before sending a remote wakeup signal to their hubs to ensure that the hub's upstream port is not being suspended. It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. We're going to see a huge spike of confidently wrong "experts" if they learn by asking ChatGPT to explain things to them.

orbifold 3y ago
I've tested it on a various areas of expertise in physics that I am familiar with and it often makes up content which sounds very plausible except that it gets pretty much always details wrong in very important ways. On the other hand I've found it very useful in providing reference articles.
PragmaticPulp 3y ago
> It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. This has been my experience as well. As I work on some projects I’ve been asking it basic questions for things I’ve already learned or know quite well. Once you deviate past the basics (content you’d find in common tutorials) the hallucination rate is out of control. The surreal part is that it all sounds so confident and plausible. The way it puts words together satisfies my “that sounds about right” reflex, but the actual content is incorrect or illogical quite frequently. If I continue prodding I can get it to come up the right answer many times, but I have to ask it a lot of leading questions to get there.

ericmcer 3y ago

This was the killer feature to me. It is an amazing teacher, it’s like having an infinitely patient mentor who can answer all your questions. I feel way more confident about being able to onboard with unfamiliar tech and be able to become productive quickly.

ornornor 3y ago

How can you trust it though? It notoriously hallucinates and makes things up in an otherwise plausible way, and the only way to tell is to already be knowledgeable about what you’re asking about… which makes the whole exercise useless.

j2bax 3y ago
I recently asked it to provide the meaning of a song. I gave it the artist and the song. It responded with a thoughtful(ish) explanation of the potential meaning of the song. I then asked it if it could share the lyrics of the song with me because some of the quotes it referenced weren't in the song. It provided lyrics that weren't the song I was asking about at all. I then asked what song it was, it apologized and it told me the artist and title of the song... I looked up that artist and title and it was also not the song that it provided the lyrics/meaning to. Ultimately, I determined that it just made up lyrics to a song. It was very apologetic.
- throwaway4aday 3y ago
  Sounds like turbo aka the free (and dumb) 3.5 version. GPT-4 would deliver better results with less hallucination but if you want it to really work well you should include the lyrics in the prompt as context. It's not a search engine, if you want that then use Bing.
RecycledEle 3y ago
Q: How can You trust StackOverlfow, Reddit, or anything on the Internet? The stuff I read online is often wrong. A: Check it yourself.
- j2bax 3y ago
  Well, you still can't trust everything, but voting mechanisms tend to help filter out some of the BS.
  - robocat 3y ago

angarg12 3y ago

Lately I've been reading seminal papers in distributed systems, and it occurred to me that there is a niche for LLMs to "democratize" highly technical content. Trying to understand those papers is a slog, but it doesn't need to be. For a casual reader, you could drop all the jargon and proofs. You could also rewrite the concepts in a less obscure and obtuse prose to make it more accessible, without dumbing down the content. Essentially it's like having a tech writer/communicator in your browser.

codeisawesome 3y ago
I often wonder why academia loves obtuse prose; the only dialect I’ve seen equally enthusiastically adopted while being just as useless is Corpspeak.

marak830 3y ago

This has been my experience as well. Also, I'm less afraid to ask the stupid questions, as I would be if asking a more experienced dev.

roflyear 3y ago

What was the thing? It's really hard to put a value to what you're describing without knowing what it was.

ModernMech 3y ago

I did the same thing to implement an automatic differentiation engine in my programming languages project. I’ve been mulling over the idea for months, but I was a little unsure how to do it. Sat down with chatgpt and we got it done mostly in an afternoon. I’m so stoked about the future of coding.

moneywoes 3y ago

How do you account for hallucinations

potsandpans 3y ago

how do you know what it's telling you is correct and not (as the open ai paper has labeled it) "hallucinating" answers? over Christmas, I used chat gpt to model a statically typed language I was working on. it was helpful, but a lot of what it gave me was sort of, in a very deceptive way wrong. It's tough to describe what I mean by "wrong" here. Nothing it spit out was just 100% blatantly incorrect. instead it was subtly incorrect, and gave inconsistent evaluations / overviews. not knowing a bit about type theory, I wouldn't actually be able to evaluate how good the information I got out of it was. I'd be hesitant to take anything chatgpt gave me at face value, or feel confident in being able to speak precisely about a given topic. did you run into similar problems? and if so, how'd you overcome them?

Baeocystin 3y ago
Not the person you asked, but chiming in. Two things: First, GPT-4 is far more capable than 3.5 when it comes to not hallucinating. The 'house style' of the response is, of course, very similar, which can lead one to initially think that the difference between the two models isn't as significant as it is. Since I don't have API access yet, I do a lot of my initial exploration in 3.5-Legacy, and once I've narrowed things down a bit, I have GPT-4 review and correct 3.5's final answers. It works very well. Second, and this is more of a meta comment: How people use ChatGPT really exposes whether they use sanity checks as a regular part of their problem solving technique. That all of the LLMs are confidently incorrect at times doesn't slow me down much at all, and sometimes their errors actually help me think of new directions to explore, even if 'wrong' on first take. Conversely, several of my friends find the error rate to be an impediment to how they work. They're good at what they do, they're just more used to taking things as correct and running with it for a longer length of time before checking to see if what they're doing makes sense. I do think that people who are put off by this significantly underestimate how often people are confidently incorrect as well. There's a reason the saying trust but verify is a common one.
- throwaway4aday 3y ago
  Third, you just glue a vector search onto it and stuff it with a bunch of textbooks and documentation.

waynesonfire 3y ago

Yeah, chatGPT is a game changer for learning. Because of chatGTP, "I know GNU Make." I just love that it's helping me conquer technologies that I've been wanting to learn my entire career but I guess just didn't have the discipline to do it. I'm getting an itch to tackle Emacs next.

SoftTalker 3y ago

I find hitting my head on the wall is essential for learning. Hitting my head on the wall helps me learn where the boundaries of functionality are, or where the actual operation of the thing differs from my intuition. If I just learn some example use cases that work, that tells me something but I won't really have mastered whatever it was that I set out to learn.

UlisesAC4 3y ago

Yeah, Chatgpt is as good as you know how to ask it questions. I have been studying clojure concepts and I too would have taken more time than an afternoon in understanding them.

nonethewiser 3y ago

> This sounds a lot like this other concept I know. How do these differ in practical use? How did you test the answer it gave you? It sounds like the end product is code, which is testable, so maybe it’s moot. But that type of question is not useful in my experience because you can’t really disprove it’s answer. It’s also a leading question. It might not be similar to that other thing at all, but ChatGPT would manufacture something in that case.

kneebonian 3y ago

Only if you can trust what it told you. And how can you trust it without knowing the material. For all you know you've not only not gained knowledge you've worse gained false knowledge.

deleted 3y ago

[deleted]

yeck 3y ago

I have had many similar experiences with just the free version of ChatGPT. Now I also ask questions more immediately. In the past, when a bunch of new terms or concepts were thrown at me, the effort to stop and dig into each of them via Google search was great enough that I would leave many stones unturned. When using GPT while studying up on a new thing, the friction is very little so fewer stones left unturned. And like you mentioned, being able to translate concepts from a novel domain to one I'm very familiar with can be like translating an unfamiliar language to english.

alecfreudenberg 3y ago

this is the way