5 ms·
It knows that this sentence structure closely resembles a simple algebra word problem, because it's read hundreds of thousands of simple algebra word problems.
by jonfw 4y ago
It knows that this sentence structure closely resembles a simple algebra word problem, because it's read hundreds of thousands of simple algebra word problems. I think you could see how somebody could tokenize that request and generate an equation like this- 250 = 4*1.4*X
And then all that's left is to solve for X
- schiffern 4y ago>It knows that... Isn't affirming this capacity for knowing exactly GP's point? Our own capacity for 'knowing' is contingent on real-world examples too, so I don't think that can be a disqualifier. Jeremy Narby delivers a great talk on our tendency to discount 'intelligence' or 'knowledge' in non-human entities.[0] [0] https://youtu.be/uGMV6IJy1Oc https://youtu.be/uGMV6IJy1Oc
- jonfw 4y agoIt knows that the sentence structure is very similar to a class of sentences it has seen before and that the expected response is to take tokens from certain locations in that sentence and arrange it in a certain way, which resembles an algebra equation It doesn't understand credit card rewards, it understands how to compose an elementary word problem into algebra
- scarface74 4y agoCheck out my other reply https://news.ycombinator.com/item?id=34811353 https://news.ycombinator.com/item?id=34811353 The relevant part “However, keep in mind that the value of the rewards can vary depending on how you redeem them, and your personal spending habits and travel preferences may also play a role in determining which card is right for you.”
- visarga 4y ago> It doesn't understand credit card rewards Probe it, go in and ask all sorts of questions to check if it understands credit card rewards, credit cards, rewards, their purpose, can solve math problems on this topic, etc.
- scarface74 4y agoExamples? I’m giving questions that I usually see in r/creditcards.
- inimino 4y agoThen don't. Instead of breathlessly trying to prove your theory, try and do some science by falsifying it. (cf. Wason test) Think of things it would get right only if it truly understood, not "common questions on reddit".
- scarface74 4y agoThe entire idea of solving math problems in middle school was that you didn’t have to know the domain and that all of the necessary information was there. When I wrote code for the health care industry, if you had asked me anything deeper about the industry or how to do brain surgery, I couldn’t have answered your question.
- inimino 4y agoYou're still trying to prove your position. Look, you're all over this thread misunderstanding LLMs and rejecting the relatively correct explanations people are giving you. The comment by joe_the_user upthread that you called an oversimplification was in fact a perfect description (randomly sampling from a space of appropriate inputs). That's exactly the intuition you should have. Do you know the Wason test? The point is that people do not intuitively know how to correctly pick which experiments to do to falsify an assumption. My point is that you are not picking the right experiments to falsify your assumptions, instead you're confirming what you think is going on. You're exactly failing the Wason task here. Really want to understand language models? Go build a few from scratch. Don't have time for that? Read Wolfram's post or any of the other similar good recent breakdowns. Only interested in understanding by playing with it? Great! An experimentalist in the true scientific tradition. Then you're going to have to do good experimental science. Don't be fooled by examples that confirm what you already think is going on! Try to understand how what people are telling you is different from that, and devise experiments to distinguish the two hypotheses. If you think ChatGPT "understands" word problems, figure out what "understanding" means to you. Now try your best to falsify your hypothesis! Look for things that ChatGPT can't do, that it should be able to do if it really "understood" by your definition (whatever you decide that is). These are not hard to find (for most values of "understand"). Finding those failures is your task, that's how you do science. That's how you'll learn the difference between reality and what you're reading into it.
- schiffern 4y agoOne can equally say, "Human brains only know that a neuron is activated by a pattern of axon firing in response to physical inputs from nerve endings." Does any of that change anything? Not really. >It doesn't understand credit card rewards Is this assertion based on anything but philosophical bias surrounding the word "understand"? >it understands how to compose an elementary word problem into algebra That's exactly how a human, who may or may not have understood rewards programs beforehand, would solve that word problem. Dan Dennett is informative here.[0] Yes, consciousness (even your own!) can just be a bag of tricks. [0] https://youtu.be/fjbWr3ODbAo?t=25s https://youtu.be/fjbWr3ODbAo?t=25s
- scarface74 4y agoIsn’t that the point? Isn’t that the same thing we test middle schoolers on in pre-algebra classes? My more complicated question comparing the Amex Hilton Aspire to the Amex Gold was inspired by this question asked over at r/creditcards https://www.reddit.com/r/CreditCards/comments/112udp3/what_is_more_valuable_100k_amex_hilton_points_or/?utm_source=share&utm_medium=ios_app&utm_name=iossmf https://www.reddit.com/r/CreditCards/comments/112udp3/what_i...
- scarface74 4y agoThis is the actual answer it gave. —— To break even on the $250 annual fee, you need to earn Membership Reward points that are worth $250. Since one Membership Reward point is worth 1.4 cents, we can calculate the number of points we need to earn by dividing $250 by 0.014: $250 / 0.014 = 17857.14 So, we need to earn 17,857 Membership Reward points to offset the $250 annual fee. Since we earn 4 Membership Reward points for every dollar we spend on groceries, we can calculate the amount we need to spend on groceries to earn 17,857 points by dividing 17,857 by 4: 17,857 / 4 = 4,464.25 Therefore, you would need to spend $4,464.25 on groceries in a year to earn enough Membership Reward points to break even on the $250 annual fee.
- williamcotton 4y agoIf you give it the right prompt, it'll give you back this: { thunk: "(async function(query, dispatch){dispatch({type: 'compute'});const fee = 250;const pointValue = 0.014;const breakEven = fee / (pointValue * 4);dispatch({type: 'compute_response'});return {answer: breakEven, solvedProblems: [], computed: true, query: false};})(query, dispatch)", en: 'You need to spend ${answer} on groceries to break even.' }
- jacquesm 4y agoThen why does a puzzle like that count towards my childs 'reading comprehension skills' score on a test? Rules for thee but not for me?
- taneq 4y agoThat’s precisely why. Humans have a long, well established record of making shit up to make themselves feel special. They do it about animals, they do it about other humans, they do it about themselves. Doing it about AI is inevitable.
- scarface74 4y agoI’m working on a relatively complex DevOps project right now that consists of over a dozen 10-30 line Python scripts involving JSON and Yaml data wrangling and AWS automation. I’ve been able to just throw my requirements into ChatGPT like I would give it to a junior dev and it came back with the correct answer 99% of the time with code quality and commenting I would expect from a junior dev. It has an “understanding” of the AWS SDK, Cloudformation, the CDK, etc. Once it generated code that had duplicate code blocks that were only different by its input. I asked it “can you remove duplicated code” and it did the refactoring. I’ve also I asked it what amounts to your standard middle school math problems and it solved the problem with explanations
- deleted 4y ago[deleted]