13 ms·

Hi HN! I've been playing around with ChatGPT a bunch since it came out. This experiment has a little bit of a backstory. Some friends and I were out at a pho re

Hi HN! I've been playing around with ChatGPT a bunch since it came out. This experiment has a little bit of a backstory. Some friends and I were out at a pho restaurant; one of us put the whole bill on his card, so the rest of us needed to figure out how much to Venmo him. We were talking about how many bill-splitting apps there are, and I made a joke about doing it with ChatGPT. Then I actually tried it out. I OCR'd the text with Google Lens, described who had what, and after a bit of prompt engineering (e.g., adding "Be sure to get your math correct" to make the AI's arithmetic check out, and convincing the AI to split shared items evenly), it totally worked: https://gist.github.com/spinda/967322dda1c04d9864f3efd45addca13 https://gist.github.com/spinda/967322dda1c04d9864f3efd45addc... Then I started experimenting with describing a hypothetical check-splitting app to the AI, and asking it to feed me JSON commands to update the UI in response to messages from me telling it what the user was doing. The results were promising! And then the similarity to the Redux data loop jumped out, and I built this generic plugin to wire ChatGPT up to apps for real.

vinhboy 4y ago
Wow that's an impressive backstory. Probably one of the more useful things I've seen come out of chatGPT
quadcore 4y ago
Pretty sure you've invented something here. A coolest form of copilot. A new way of programming. This is really lispy, this is really cool. Love it 10 times.
- quadcore 4y ago
  Edit: you ask the AI to run the program in its head, its a new form of computer actually afaik.
  - visarga 4y ago
    Next up: I asked the AI to pretend to be the backend, but also front end! Apps spawn into existence to be used once and discarded.
- suction 4y ago
  [dead]
Gigachad 4y ago
Cool project, but it reminds me of one of the golden rules of AI “if you can solve it with a simple rules based system, you should” Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring.
- mintplant 4y ago
  I see your point, but I think that's kinda neat, personally. Fixing a bug by asking nicely for the bug not to happen. The pinnacle of declarativeness.
  - MereInterest 4y ago
    It makes me think of INTERCAL, that you need to prepend commands with "please" some of the time, but not so often that it chastises you for being overly polite.
  - ncr100 4y ago
    Wild times to be alive and a software engineer. Wonderful example @mintplant.
- lelandfe 4y ago
  It also did not work perfectly for me, running GP's prompt. ChatGPT left off the shared "Egg Rolls" entry entirely :(
- MikeTheGreat 4y ago
  Can I ask why it's effective to tell it to get it's math correct? Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why would telling it to check it's math actually make it's predictions more accurate? 2) Why would asking it to check it's math change the results it produces? Shouldn't it just do the same thing a second time and produce answers that are just as (in)accurate as it did the first time?
  - mintplant 4y ago
    My intuition is: the AI is trying to predict a likely response to the input query given what it's seen in the dataset it's been trained on. That dataset will include instances where math has been done incorrectly, so the AI producing a response with incorrect math isn't inherently "wrong" from its perspective. However, queries containing reminders to be careful with the math are more likely to be followed by responses with correct numbers.
willnonya 4y ago
So, you needed a multiple step process of technology transfer to an ai or an app to figure out how to split the check? I find this fascinating but not terribly impressive. Its fascinating that such a rudimentary skill prompted you to dive down this rabbit hole. The unimpressive part is that it's a rudimentary skill yet your over engineered solution still only required rudimentary skills.
- mintplant 4y ago
  It was more of a fun exercise, and an excuse to play with some cool tech! The part I was most impressed with was that ChatGPT could make sense of the mashed-up OCR text from the receipt—I can see that being super handy for speeding up expense reporting data entry. Everything else was extra.
- emodendroket 4y ago
  It's not, like, a practical way to solve problems, but it's interesting in its own right.
kristiandupont 4y ago
This seems like it could be perfect for making prototypes. "Pretend that you are a backend for a text messaging app that supports group chats. The data model should look something like this: ..."
supermatt 4y ago
Why use chatGPT instead of using the GPT-3 api directly? For the type of queries you are doing (sending whole context), the output is comparable (and just as wrong) between chatGPT and GPT-3.
- j5155 4y ago
  ChatGPT is currently free, GPT-3 is not
- moffkalast 4y ago
  Afaik there's no way to build upon your prompt to have it change the output in specific ways in GPT-3?
  - supermatt 4y ago
    You feed it back in as the final tokens of the next prompt.

vinhboy 4y ago

Wow that's an impressive backstory. Probably one of the more useful things I've seen come out of chatGPT

quadcore 4y ago

Pretty sure you've invented something here. A coolest form of copilot. A new way of programming. This is really lispy, this is really cool. Love it 10 times.

quadcore 4y ago
Edit: you ask the AI to run the program in its head, its a new form of computer actually afaik.
- visarga 4y ago
  Next up: I asked the AI to pretend to be the backend, but also front end! Apps spawn into existence to be used once and discarded.
suction 4y ago
[dead]

Gigachad 4y ago

Cool project, but it reminds me of one of the golden rules of AI “if you can solve it with a simple rules based system, you should” Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring.

mintplant 4y ago
I see your point, but I think that's kinda neat, personally. Fixing a bug by asking nicely for the bug not to happen. The pinnacle of declarativeness.
- MereInterest 4y ago
  It makes me think of INTERCAL, that you need to prepend commands with "please" some of the time, but not so often that it chastises you for being overly polite.
- ncr100 4y ago
  Wild times to be alive and a software engineer. Wonderful example @mintplant.
lelandfe 4y ago
It also did not work perfectly for me, running GP's prompt. ChatGPT left off the shared "Egg Rolls" entry entirely :(
MikeTheGreat 4y ago
Can I ask why it's effective to tell it to get it's math correct? Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why would telling it to check it's math actually make it's predictions more accurate? 2) Why would asking it to check it's math change the results it produces? Shouldn't it just do the same thing a second time and produce answers that are just as (in)accurate as it did the first time?
- mintplant 4y ago
  My intuition is: the AI is trying to predict a likely response to the input query given what it's seen in the dataset it's been trained on. That dataset will include instances where math has been done incorrectly, so the AI producing a response with incorrect math isn't inherently "wrong" from its perspective. However, queries containing reminders to be careful with the math are more likely to be followed by responses with correct numbers.

willnonya 4y ago

So, you needed a multiple step process of technology transfer to an ai or an app to figure out how to split the check? I find this fascinating but not terribly impressive. Its fascinating that such a rudimentary skill prompted you to dive down this rabbit hole. The unimpressive part is that it's a rudimentary skill yet your over engineered solution still only required rudimentary skills.

mintplant 4y ago
It was more of a fun exercise, and an excuse to play with some cool tech! The part I was most impressed with was that ChatGPT could make sense of the mashed-up OCR text from the receipt—I can see that being super handy for speeding up expense reporting data entry. Everything else was extra.
emodendroket 4y ago
It's not, like, a practical way to solve problems, but it's interesting in its own right.

kristiandupont 4y ago

This seems like it could be perfect for making prototypes. "Pretend that you are a backend for a text messaging app that supports group chats. The data model should look something like this: ..."

supermatt 4y ago

Why use chatGPT instead of using the GPT-3 api directly? For the type of queries you are doing (sending whole context), the output is comparable (and just as wrong) between chatGPT and GPT-3.

j5155 4y ago
ChatGPT is currently free, GPT-3 is not
moffkalast 4y ago
Afaik there's no way to build upon your prompt to have it change the output in specific ways in GPT-3?
- supermatt 4y ago
  You feed it back in as the final tokens of the next prompt.