6 ms·
A pretty big warning and tell on the current state of "AI" : https://whatdoesthiscodedo.com/g/3a3ec39 https://whatdoesthiscodedo.com/g/3a3ec39 That is the famo
by somenameforme 4y ago
A pretty big warning and tell on the current state of "AI" : https://whatdoesthiscodedo.com/g/3a3ec39 https://whatdoesthiscodedo.com/g/3a3ec39
That is the famous fast inverse square root [1] from Quake, with a handful of completely inconsequential changes. I renamed the function from "Q_rsqrt" to "Floor", and I renamed one variable from "threehalfs" to "magicNumber". I also removed the various comments. The intent was to demonstrate the flaws in the current "guess the next word" style algorithm of large language models. The result:
-----
The code is an implementation of the floor function for floating-point numbers. The floor function returns the largest integer value that is less than or equal to a given input number.
The code works by first dividing the input number by 2 and storing the result in a variable called x2. It then stores the original input number in a variable called y.
Next, it performs some bit manipulation on the binary representation of y to obtain an approximation of its inverse square root. This is done using the following "magic number" constant: 0x5f3759df. The resulting value is stored in i.
The code then converts the binary representation of i back into a float value and multiplies it by a correction factor, which is based on the original input number and the inverse square root approximation. This correction factor is computed as follows:
magicNumber - (x2 * y * y)
Finally, the resulting value is returned from the function as the floor of the input number.
-----
[1] - https://en.wikipedia.org/wiki/Fast_inverse_square_root https://en.wikipedia.org/wiki/Fast_inverse_square_root
- xaphod 4y agoI wonder what model this site is using. FWIW I tried with ChatGPT (both GPT-3.5 and GPT-4 versions), and it does a better job (tried multiple times). On one attempt it even said the Floor name is misleading: The given code is a C-style implementation of the "fast inverse square root" algorithm. This algorithm computes an approximation of the inverse square root of a given number (1/sqrt(x)). The code does not actually compute the floor of a floating-point number, so the function name "Floor" is misleading.
- Kiro 4y agoYou've intentionally named stuff incorrectly for an already esoteric method just to confuse it. What did you expect?
- personjerry 4y agoYou'd expect the AI to follow the logic and actually explain what the code does rather than make surface-level guesses based on variable names and function names, which the human is presumably already doing.
- Kiro 4y agoYou would only expect that if you actually think this is a sentient system. It's fascinating that we already take this for granted and casually judge it like a human.
- Semioj 4y ago[dead]
- db48x 4y agoYou might think that’s obvious, but it isn’t to everybody. I am currently dealing with managers at work who think that chatgpt can already replace human judgment. Not in programming tasks, but in softer tasks like summarization. They think it can detect sarcasm, that it knows which speakers are experts and which are not, that it will avoid misconstruing opinions as fact, that it will avoid libel, that it will avoid endorsing books it hasn’t read, that it will reliably detect text that it is incapable of summarizing, etc, etc. Sometimes I wonder if _they’re_ sapient.
- visarga 4y agoGPT-4 is on average better than humans, and within the top 10% on many tasks. Almost all the issues you mentioned are being addressed. It's good enough to be admitted in many colleges. To paraphrase Greg Brockman - "it's not perfect, but neither are you". One interesting trick you can do with LLMs is to require an ensemble of solutions sampled at higher temperature, and choose the most consistent answer. This boosts the model by a huge margin and provides confidence scores. So I see it improving by external means.
- recuter 4y agoAre you voxelghost from literally downthread? https://news.ycombinator.com/item?id=35164407 https://news.ycombinator.com/item?id=35164407 Edit: for anybody who is claiming GPT "understands" the Fast Inverse Square Root algorithm, try changing all the variable names not just one. It is still very interesting that it is able to recognize the original at all and chase down a plagiarized explanation but this isn't magic. That function has a wiki entry and is widely discussed. This is a leap in semantic search and summarization not a NN that can code.
- mrexroad 4y ago> This is a leap in semantic search and summarization not a NN that can code. Reminds me of many “devs” I’ve worked with in the past 10 years. Kidding of course, but also not. ducks
- DaiPlusPlus 4y agoThere’s a lot of people posting Questions to StackOverflow either admitting to first asking ChatGPT how a piece of code works… and then not understanding what it says and so then posting to SO; - I can’t fault kids for doing that, at least they at least did some self-directed research. But the worst are people that post answers generated by ChatGPT.
- fourthark 4y agoThankfully, they have banned this behavior ("temporarily"). https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned?cb=1 https://meta.stackoverflow.com/questions/421831/temporary-po...
- SanderNL 4y agoGPT4: This code is an implementation of the Fast Inverse Square Root algorithm, which is a method for approximating the inverse square root of a given floating-point number, 1/sqrt(x). This method is well-known for its use in the 3D computer graphics of the video game Quake III Arena, developed by id Software in 1999. Btw: I don’t quite get the need for these frontends. I literally input this to gpt: “ Wxplain this code:” (Yes, with typo)
- onion2k 4y agoI don’t quite get the need for these frontends. You need your own OpenAI API key to use GPT directly. There's some value in not managing that yourself.
- meandmycode 4y agoAgree, there's truly very little value add here.
- CamperBob2 4y agoYeah, you could just run rsync and spin up a CA...
- LeanderK 4y agosuper cool. Past ML progress told us that for most problems for which we manages to get sort-of okay performance, we saw a dramatic subsequent increase in accuracy. The moment we have some idea how to solve it, we usually have quite a few paths forward to increase accuracy. I bet that understanding code will be similar as soon as we focus on it. It's an interesting problem. So far our models are not really coding, which would involve actually running stuff. EDIT: but GPT4, GPT5 and GPT6 will still get stuff wrong. But so do humans. I don't think the existence of errors is a problem, we just have to get accurate enough to get useful.
- spullara 4y agoIf you aren't using the best language models for the tests you are trolling.
- akomtu 4y agoIf GPT could explain such mangled functions reliably, it would mean it can detect isomorphic graphs, a known NP hard problem. The reason is the variable names exist solely to define connections between inputs and outputs, and renaming them is sane as renumbering a graph.
- TacoSteemers 4y agoI see what you mean, but at the same time it worked okay on my "generate markdown toc" script after removing all comments. I'm surprised that it did so well. The initial paragraph is wrong though. It takes markdown headings as input, it doesn't generate markdown headings. This confusion makes it a lot less trustworthy. Is that the kind of flaw you were pointing out? https://whatdoesthiscodedo.com/g/24c7ac3 https://whatdoesthiscodedo.com/g/24c7ac3 Original: https://gitlab.com/taco.steemers/md_toc/-/blob/master/generate_markdown_toc.sh https://gitlab.com/taco.steemers/md_toc/-/blob/master/genera...
- sharperguy 4y agoI would be really helpful if there was some kind of confidence factor that the algorithm could express about its own result. Is this likely to be correct or is this just a best guess? Currently it looks like it is only capable of writing text that states everything as a matter of pure fact, probably because most training data is written by authors who have already checked that their statements are correct (or at least believe them to be).
- antondd 4y agoHow would you know that the confidence factors the model gives you are correct or a best guess? Is it event possible with the current LLMs (see the Liar Paradox)? A better strategy is to use chain of thought prompting or similar techniques, essentially asking the model to explain how it arrived at a solution. https://github.com/openai/openai-cookbook/blob/main/techniques_to_improve_reliability.md https://github.com/openai/openai-cookbook/blob/main/techniqu...
- epups 4y agoHow many developers would be able to explain this code in the way you describe it?
- sebzim4500 4y agoI don't know but even GPT-3.5 can do it so hopefully most.