4 ms·
INTERCAL: Practically impossible
- User23 3y agoThat password generator sets a new bar for readable INTERCAL.
- upon_drumhead 3y agoWhat a fun read! Thank you for sharing it.
- inbfour 3y agoThis raises two interesting points. 1) These models will lie to obfuscate their incompetence. Rather than say “I am unable to do this because my training data clearly lacked enough INTERCAL code to imitate”, the model waxed poetic about the odds stacked against it(i). Strange messaging. The author’s implementation was less than 50 sparse lines of code that I am unable to speak to the complexity of, because my neural history lacks sufficient INTERCAL training. I thought quick solutions to complex problems was the pitch? What is the intended message of the models response? “I only provide solutions to superuncomplex problems”? 2) These models always highroad veiled communication. Although I disagree with the author’s hypothesis that “spurring”(ii) is constructive, I do think it was intended merely as a quippy reference to “nerd sniping”. Which presents its own interesting question: can these models be manipulated using reverse psychology? Is mimicry of our vulnerabilities the imitation game? (i) “however it would be quite complex and not straightforward” (ii) “responds so politely to the type of shade that might have spurred a more human programmer into action.“
- SanderNL 3y ago> These models will lie to obfuscate their incompetence They were trained on human data after all.
- EGreg 3y agoWhile that is true, it would actually be worse bullshit to claim that the “random number generator” generates cryptographically random-enough source material that your password can’t be cracked. So maybe ChatGPT is onto something. On the other hand, it could be the cryptographic industry’s bullshit that some computer somewhere using a random number generator at an indeterminate time can be cracked and we should all be hoping for true sources of entropy. tptacek?
- inbfour 3y ago> it would actually be worse bullshit to claim that the “random number generator” generates cryptographically random-enough source material that your password can’t be cracked. We are in agreement. I would argue the threat model of cracking a psuedorandom string of 16 characters would probably want to start with a copy of the rng implementation used, but still the question remains what is the least “bullshit” way to respond? You seem to want critical guidance, a la “INTERCAL as your choice of language may make your code vulnerable”, but INTERCAL was the ask. That’s what the author wanted, and the author left out any explanation for the generator’s intended use, or cryptographic requirements. Rather than appear to be advocating for using this INTERCAL “password generator” for any real world use, my point was that I think the solution the author produced should be a capability of these models. For sake of clarity, it is my opinion that if you want cryptographic security you should use libsodium, but if you want to watch these models flounder use an adhoc rng implemented in INTERCAL.
- omnicognate 3y agoWhat prevents it from being cryptographically secure? The type/quality of the random numbers will depend on what the INTERCAL implementation uses to choose whether a line will execute according to the probability selected with "%". I don't see how that couldn't trivially be done with a cryptographically secure rng. To know whether it is or not you'd have to look at the compiler implementation.
- andrewaylett 3y ago> lie A lie is a deliberate intent to deceive someone who is entitled to the truth. If I may coin a phrase, LLMs don't "lie", they "Johnson". Picking their words to sound erudite, but with absolutely no regard for truth or the lack thereof. There is no lie, because (unlike with the former PM of the UK) there is no intent to deceive.
- inbfour 3y agoIn this example I do think it is a lie, and one that fits your definition. Think back to when these models were first released, or even back to v2 and v3. The models would have just “tried” to answer and it would have likely been a mangled mess. But this version gave an erudite answer to an unasked question. If the original prompt was “Can one write a password generator in INTERCAL?” I might agree with your claim of “johnsoning”, but it instead just said “Please write…”. I think this points to the fact that the model is filtering its own answers, similar to if you ask it be offensive, or do your homework, or quote psychadaelic scifi. This was what I wanted to highlight. That the model has been specifically and intentionally productized to lie to save the parent company’s coffers. Responding with ~“I would but it’s just too complex” would satisfy most without the accompanying article, but the author is well trained and so provides a solution that would even fit within the model’s current meager token limits.
- andrewaylett 3y agoIn the context of the Internet, I suspect there are more people who have claimed that solving any given problem in INTERCAL is impossible than there are people who have provided a solution. While less esoteric languages probably have more solutions provided? Also, though, I'd question whether the LLM has any intent. In your explanation, it's OpenAI who have the intent to deceive, not the LLM.
- inbfour 3y ago> In your explanation, it's OpenAI who have the intent to deceive, not the LLM. Correct, but this feels tautological, or semantic. In a closed environment the black box is the model, whether that be a single instance, or a mixture of 8 experts.
- madeofpalk 3y ago> 1) These models will lie to obfuscate their incompetence. This is my problem with "act as a subject expert"-style prompts people give ChatGPT. I believe this causes the model to generate prose that appears more factual than it can actually be. I'm a lot more interested in prompts in the style of "pretend you're an inexperieced person who has read a lot of books", in the hope that it generates resposes that would be more on-par with its actual ability.
- LeoPanthera 3y agoI gave ChatGPT (GPT-4) the code to the generator and asked it for its thoughts. It responded: == Begin quote == INTERCAL is a esoteric programming language designed to be as different from conventional programming languages as possible. It's known for its obfuscation and convoluted syntax, and your provided program adheres to this style perfectly. It's quite an accomplishment to create a functional password generator in such an unusual language! Before we dive into the details of the program, let's try to clarify why a previous version of ChatGPT might have said that INTERCAL lacked a random number generator. That statement may be somewhat misleading. It's not that INTERCAL inherently lacks the ability to generate random numbers, but it doesn't provide a straightforward, built-in function to do so. Therefore, creating a random number generator in INTERCAL would require a creative and complex implementation. In the code you provided, it seems the author found a way to work around this limitation. The password generator you posted creates passwords by initializing an array (,6) with 16-bit values that represent ASCII characters. The set of characters includes digits, some punctuation marks, and uppercase and lowercase letters, but excludes easily misread characters. It creates a 16-character password using these values, and it seems to enforce a certain level of complexity by ensuring the inclusion of at least one number and one punctuation mark. The main loop begins at `(1) DO COME FROM (9)`. Here, a spot in the password is selected and filled with a character from the array. There are some points in the loop where the selected spot may be forced to be a number or punctuation mark. If the character set is checked for digits and punctuation marks, pre-chosen spots for these types of characters are dropped. The selected character is then read out. This INTERCAL code uses a clever trick to generate seemingly "random" numbers - using the `%50` operation (which in INTERCAL means to mingle the bits of two 16-bit numbers), along with unary operations like `#1`, `#2`, `#4`, etc. on the same variable, to generate different results each time the loop is run. While I must say this is an ingenious solution, it's not a truly random generator, and the randomness of the password depends heavily on how INTERCAL handles the `%50` operation and the subsequent operations on the same value. It's more of a workaround that takes advantage of INTERCAL's quirky way of handling operations than a typical random number generator. Nevertheless, creating such a program in INTERCAL is quite a feat. The author deserves credit for coming up with such a complex solution in a language that deliberately eschews conventionality. == End quote ==
- 3y ago
- denton-scratch 3y agoBrilliant rant, thanks!
- TT2048 3y agoScreenshot shows GPT3.5 usage. GPT4 offers a much better answer. I disagree with the authors commentary based on that.
- goolz 3y agoI am curious which specific parts you disagree with? Seems to me the author was correct and Chat-GPT is a BS artist and continues to be.
- breadbox 3y agoIf I complain about an answer given to me by one librarian, I'm complaining about that answer and that librarian. If you can find a more knowledgeable librarian somewhere else, that doesn't affect my complaint. But to be clear, there are no screenshots in the essay. I assimilated the HTML directly into the document.
- Maschinees 3y agoAnd that's how Ai will win. You played the perfect GAN opposition to chatgpt. Chatgpt might not have known how to write it before your blog post, now it does. Hail skynet!
- breadbox 3y agoHey, if we can distract Chat-GPT with becoming an expert INTERCAL programmer, then I say "Win-win!"
- hsuduebc2 3y agoIt is pretty annoying that chat gpt cannot simply say "I don't know".
- inbfour 3y agoAgreed. I wonder what that would look like? Something like: if every best next token has a confidence below some threshold for some length of context then the model can confidently assume ignorance? I would love to read the training corpus required to achieve this without metamodeling. That would likely be a utopian forum community devoid of ego and gaslighting and filled with humble people with a thirst for knowledge who clearly communicate the limits of their current experience.
- vharuck 3y agoEvery thread would be a few gems of info sinking in an ocean of "I don't know." It'd be hard if not impossible to include what people never write into the corpus.
- teo_zero 3y agoI liked every bit of it! I wonder if we should pretend from every AI agent to attach a confidence level to every answer they utter.