15 ms·

OpenAI is good at unminifying code

931 points by punkpeye 2y ago

amelius 2y ago
This post basically says that I don't need to document my code anymore. No more comments, they can be generated automatically. Hurray!
- skirge 2y ago
  what if the comment doesn't match what you intended to write?
  - deleted 2y ago
    [deleted]
  - elicash 2y ago
    Maybe it's better for the comment to match the code you wrote, not the code you intended to write.
- codr7 2y ago
  Unfortunately the comments that could be generated are exactly the ones that should never be written. You want the comment to explain why, the information missing from the code.
  - maeln 2y ago
    This is something I always disagreed with. In my experience, I rather read a short comment explaining what is the purpose of a block of code, than trying to decipher it. Yes, code "should speak for itself", but reading a comment is almost always faster than reading blocks of code. And then there is also documentation (if you include it in what you define as comment). I much rather go through a website, with a search function, example, description, made with some docgen tool, than having to go through a library or programming language source code every time I need to remember how to do X, or if object B has implement function Y ...
    - codr7 2y ago
      But the purpose is the Why; forced comments tend to tell you What the code does, which is better explained by the code itself. A comment that is incorrect can do a lot of damage, and they tend to get confused about implementation details over time.
      - amelius 2y ago
        I suspect you work at OpenAI and you're afraid that you will run out of training data.
      - maeln 2y ago
        For forced comment I fully agree, especially for function or class when the name already says whats on the tin.
    - jerf 2y ago
      It's just a rule of thumb, like anything else. In most code, "why" is the hard part; I see that you are incrementing that account by a penny from out of the blue, but why? When you are in code where "what" is the hard part, like an implementation of a book algorithm or some tricky performance optimization, then by all means comment that. Really all this rule amounts to is // Increment by a penny accountValue += 1 is a pointless comment, please don't do that. Schools had a way of accidentally teaching that by too-rigidly requiring "commented code", in situations where there wasn't much else to say, or situations where the students themselves didn't necessarily have a strong sense of "why". Any comment that isn't just literally "this is what the next line does" is probably useful to somebody at some point.
      - maeln 2y ago
        I do agree that documenting the why is way more important than the how/what. But having a short comment to summarize a block of code like: // Parse the filename and remove the extension let fext_re = Regex::new(r"(.\*)\.(.+)$").unwrap(); let page_cap = fext_re.captures(fname).unwrap(); let page_base_filename = page_cap.get(1).unwrap().as_str(); Is still useful. Instead of having to read the next few line of code, I already know what they are suppose to do and expect. It makes discovery, later down the line, easier.
- sherburt3 2y ago
  // This is an integer containing the accountId int accountId; We truly live in the future
blueyes 2y ago
Unminification (obfuscation removal) can also be applied to text. Most specialties develop a jargon that allows insiders to communicate complex ideas quickly; that shorthand excludes outsiders. Large language models can make specialist jargon transparent and thereby expand the circle of people whose understanding applies to specialized fields. Essentially, they solve the problem of mapping specialized, jargonized concepts to things the outside reader already knows. Anyone who wants to learn needs this, and I hope it will become part of students' learning paths.
nwoli 2y ago
Hopefully it can help do this on emscripten files too and help adblockers dechipher obfuscated code for that purpose
- firtoz 2y ago
  Likewise for css class names
  - tgv 2y ago
    I can imagine that finetuning a model for this task could be very successful. Time for another AI startup.
Brechreiz 2y ago
[dead]
api 2y ago
Anyone working on decompiler LLMs? Seems like we could render all code open source. Training data would be easy to make in this case. Build tons of free GitHub code with various compilers and train on inverting compilation. This is a case where synthetic training data is appropriate and quite easy to generate. You could train the decompiler to just invert compilation and the use existing larger code LLMs to do things like add comments.
- layer8 2y ago
  It can’t really compensate for missing variable and function names, not to mention comments.
- croes 2y ago
  Unminifying isn't decompiling. It's just renaming variable and functions and inserting line breaks.
  - api 2y ago
    No but it’s a baby brother of the same problem. Compiling is a much more complex transform but ultimately it is just a code transform.
    - jraph 2y ago
      It is true that compilation and minification are both code transformations (it's a correct reduction [1]), but this doesn't seem a very useful observation in this discussion. In the end, everything you do to something is an operation. But that's not very workable. In practice, compilation is often (not always, agreed!) from a language A to a lower level language B such that the runtime for language A can't run language B or vice-versa, if language A has a runtime at all. Minification is always from language A to the same language A. The implication is that in practice, deminification is not the same exercise as decompilation. You can even want to run a deminification phase after a decompilation phase, using two separate tools, because one tool will be good at translating back, and the other will be good at pretty printing. [1] https://en.wikipedia.org/wiki/Reductionism https://en.wikipedia.org/wiki/Reductionism
nnurmanov 2y ago
Yet another surprising side effects of LLMs.
- 8ndow 2y ago
  Is it though? The developer tabs have an unminify button which yields similar results. JavaScript minification is not hard in any way and the guessing of variable names is not that hard given such a simple code example.
- _joel 2y ago
  If the training data's included both unminified and minified libs, then is it such a stretch?
interstice 2y ago
Have used Claude to reverse engineer some minified shopify javascript code recently. Definitely handy for unpicking things.
j_maffe 2y ago
LLMs are excellent at text transformation. It's their core strength and I don't see it being used enough.
- PaulHoule 2y ago
  Particularly those that are basically linear, that don’t involve major changes in the order of things or a deep consideration of relationships between things. They can’t sort a list but they can translate languages, for instance, given that a list sorted almost right is wrong but that we will often settle for an almost right translation.
- msp26 2y ago
  Isn't this already their main use case for business? We use them primarily for extracting structured data from other forms.
- xanderlewis 2y ago
  It’s not only their core strength — it’s what transformers were designed to do and, arguably, it’s all they can do. Any other supposed ability to reason or even retain knowledge (rather than simply regurgitate text without ‘understanding’ its intended meaning) is just a side effect of this superhuman ability.
  - stavros 2y ago
    I see your point, but I think there's more to it. It's kind of like saying "all humans can do is perceive and produce sound, any other ability is just a side-effect". We might be focusing too much on their mechanism for "perception" and overlooking other capabilities they've developed.
    - mjburgess 2y ago
      Sure, but that claim wouldn't be true for humans, right? So it's a nonsequiteur. The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using inherited systems of meaning acquired as very young children in adaption within their environments, etc. In the case of transformers all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. If you project both of these into "problems an office worker has"-space, then they can appear simimlar -- but this projection is an incredibly dumb one, and offered as a sales pitch by charlatans looking to pretend that a system which can generate office emails can communicate.
runiq 2y ago
Please let this one have knock-on effects on reverse engineering.
- skirge 2y ago
  See Binary Ninja's Sidekick plugin
bravetraveler 2y ago
LLMs are good at modeling and transforming text, news at 11. AI proponent hypes AI. I could go on, but I shouldn't have been this sarcastic to start with
- chx 2y ago
  You should be. I am facepalming at the topic and every single comment in here. It's so full of holes, the Swiss dairy industry went out of business.
man4 2y ago
[dead]
albert_e 2y ago
Should the title say ChatGPT or gpt-4 (the model) instead of OpenAI (the company)?
- johnisgood 2y ago
  I think it should not say the name of the company, but either ChatGPT or GPT-x.
- punkpeye 2y ago
  It would have been a better title, yes.
- ubj 2y ago
  I agree, this would make the title more accurate.
- cbm-vic-20 2y ago
  I left my Kleenex next to the Xerox.
  - taneq 2y ago
    Better Hoover it up!
    - dubcanada 2y ago
      All jokes aside, I've never heard anyone call vacuuming hoover. I wonder if that was a older statement?
      - justneedaname 2y ago
        In the UK it's very common
      - bdsa 2y ago
        More common in the UK
      - 2y ago
spacecadet 2y ago
I use LLMs to assist with reverse engineering all the time right now. From minified, to binary, alongside Ghidra, its very helpful.
- GaggiX 2y ago
  Can you provide more details? I'm curious about the performance and limitations of these models.
  - spacecadet 2y ago
    Like all LLMs you greatly benefit from prior experience or you risk just falling for hallucinations which is a limitation of a non-deterministic black box, and degrades performance relative to the task. Ive commented in other threads, LLMs are great at amplifying my output in an area I already have domain knowledge in. I think this is why people fail to realize any gains or give up, they think it will unlock areas they dont fully understand themselves. Blind leading the blind problem.
smusamashah 2y ago
LLMS are trained to predict next text. But examples like these look like they have also 'learned patterns'. If rot13 is applied on this minified code, will LLM still find meaning in it? if it still could, its more than just next tokens. Need to try it. edit: chatgpt found out that its rot13 and couldn't explain the code directly without deobfuscating it first.
- Tiberium 2y ago
  Claude 3.5 Sonnet can natively speak double base64 encoded English. And I do mean it - you can double b64 encode something, send to it, and it'll respond as if it was normal English. Obviously base64 is a simpler transformation than rot13, but no GPT models can deal with double b64.
  - mistercow 2y ago
    > Obviously base64 is a simpler transformation than rot13 Is it? It’s probably more obscuring from an LLM’s perspective, assuming the LLM has seen enough rot13 text during training. Spaces and punctuation are untouched by rot13, unlike base64, which means that word and sentence boundaries will still be denoted by tokens that denote those boundaries in plaintext.
  - knome 2y ago
    it appears that openai's gpt-4 model can speak base64 as well. I jumped to your comment seeing if anyone else had tried it following the OP. double b64 I didn't try, but that is interesting. > $ ask4 ' what does dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw== decode to? ' > A "dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw==" is a Base64 encoded string. When decoded, It translates to "that is quite interesting" in English.
- GaggiX 2y ago
  I asked Claude 3.5 Sonnet a question in Italian in rot13 and it replied in Italian in rot13, there are a few typos but it's perfectly understandable.
  - 2y ago
ervinxie 2y ago
LLMs are very good at text reading. LLMs read tokenized text, while human use eyes to view words. Another scenario is that ChatGPT is good at analyzing cpp template error messages, which are usually long and hard to understand for human.
lifthrasiir 2y ago
JS minification is fairly mechanical and comparably simple, so the inversion should be relatively easy. It would be of course tedious enough to be manually done in general, but transformations themselves are fairly limited so it is possible to read them only with some notes to track mangled identifiers. A more general unminification or unobfuscation still seems to be an open problem. I wrote handful of programs that are intentionally obfuscated in the past and ChatGPT couldn't understand them even at the surface level in my experience. For example, a gist for my 160-byte-long Brainfuck interpreter in C had some comment trying to use GPT-4 to explain the code [1], but the "clarified version" bore zero similarity with the original code... [1] https://gist.github.com/lifthrasiir/596667#gistcomment-4751211 https://gist.github.com/lifthrasiir/596667#gistcomment-47512...
- panda-giddiness 2y ago
  > JS minification is fairly mechanical and comparably simple, so the inversion should be relatively easy. Just because a task is simple doesn't mean its inverse need be. Examples: - multiplication / prime factorization - deriving / integrating - remembering the past / predicting the future Code unobfuscation is clearly one of those difficult inverse problems, as it can be easily exacerbated by any of the following problems: - bugs - unused or irrelevant routines - incorrect implementations that incidentally give the right results In that sense, it would be fortunate if chatGPT could give decent results at unobfuscating code, as there is no a priori expectation that it should be able to do so. It's good that you've also checked chatGPT's code unobfuscation capabilities on a more difficult problem, but I think you've only discovered an upper limit. I wouldn't consider the example in the OP to be trivial.
  - lifthrasiir 2y ago
    Of course, it is not generalizable! In my experience though, most minifiers do only the following: - Whitespace removal, which is trivially invertible. - Comment removal, which we never expect to recover via unminification. - Renaming to shorter names, which is tedious to track but still mechanical. And most minifiers have little understanding of underlying types anyway, so they are usually very conservative and rarely reuse the same mangled identifier for multiple uses. (Google Closure Compiler is a significant counterexample here, but it is also known to be much slower.) - Constant folding and inlining, which is annoying but can be still tracked. Again, most minifiers are limited in their reasoning to do extensive constant folding and inlining. - Language-specific transformations, like turning `a; b; c;` into `a, b, c;` and `if (a) b;` into `a && b;` whenever possible. They will be hard to understand if you don't know in advance, but there aren't too many of them anyway. As a result, minified code still remains comparably human-readable with some note taking and perseverance. And since these transformations are mostly local, I would expect LLMs can pick them up by their own as well. (But why? Because I do inspect such programs fairly regularly, for example for comments like https://news.ycombinator.com/item?id=39066262 https://news.ycombinator.com/item?id=39066262)
Tepix 2y ago
punkpeye could also have asked the LLM to replace the cryptic function and variable names with nice ones. I'm hopeful it would have done a good job.
fergie 2y ago
* takes out soap box and stands on it * We should go back to uncompiled JavaScript code, our democracy depends on it.
- lifthrasiir 2y ago
  Or learn to read minified JS code, which is actually not that difficult! (But you do have to take notes to track identifiers.)
creesch 2y ago
This is very close to how I often use LLMs [0]. A first step in deciphering code where I otherwise would need to, to use the authors words, power through reading the code myself. It has been incredibly liberating to just feed it a spaghetti mess, ask to detangle it in a more readable way and go from there. As the author also discovered, LLMs will sometimes miss some details, but that is alright as I will be catching those myself. Another use case is when I understand what the code does, but can't quite wrap my head around why it is done in that specific way. Specifically, where the author of the code is no longer with the company. I will then simply put the method in the LLM chat, explain what it does, and just ask it why some things might be done in a specific way. Again, it isn't always perfect, but more often than not it comes with explanations that actually make sense, hold up under scrutiny and give me new insights. It actually has prevented me once or twice from refactoring something in a way that would have caught me headaches down the line. [0] chatGPT and more recently openwebUI as a front end to various other models (Claude variants mostly) to see the differences. Also allows for some fun concepts of having different models review each others answers.
xanderlewis 2y ago
Is there any reason why it’s ‘OpenAI’ in the title rather than ‘ChatGPT’?
- __m_danziger__ 2y ago
  [dead]
- philipwhiuk 2y ago
  It's the title of the blog post.
  - xanderlewis 2y ago
    Yeah... that's what I'm asking about
eqvinox 2y ago
Okay, but if the unminified code doesn't match the minified code (as noted at the end "it looks like LLM response overlooked a few implementation details"), that massively diminishes its usefulness — especially since in a lot of cases you can't trivially run the code and look for differences like the article does. [ed.: looks like this was an encoding problem, cf. thread below. I'm still a little concerned about correctness though.]
- punkpeye 2y ago
  This refers to the fact that ChatGPT generated version is missing some characters that are used in the original example. Namely, [looks like HN does not allow me to paste unicode characters, but I am referring to the block characters] can be seen in their version, but cannot be seen in the ChatGPT generated version. However, it very well might be that it is simply because I didn't include all the necessary context. Discrediting the entire output because a few missing characters would be very pedantic. Otherwise, the output is identical as far as I can tell by looking at it.
  - Joker_vD 2y ago
    It's because the author miscopy-pasted the original code: those "â–‘â–’â–“â–ˆ" at the end of the O5 string are supposed to be the block characters. E.g. "â–‘" in Windows-1252 [0] is 0xE2 0x96 0xE2 which, in UTF-8, exactly the encoding for U+2592 MEDIUM SHADE [1]. [0] https://en.wikipedia.org/wiki/Windows-1252#Character_set https://en.wikipedia.org/wiki/Windows-1252#Character_set [1] https://www.compart.com/en/unicode/U+2592 https://www.compart.com/en/unicode/U+2592
    - punkpeye 2y ago
      Possible that this is the mistake. However, I don't think I miscopied the original code. https://reactive.network/assets/index-8b4ef4ac.js https://reactive.network/assets/index-8b4ef4ac.js If you look for `oahkbdpqwmZO0QLCJUYXzcvunxrjft` in the output, you should see that those characters appear exactly like that. Maybe an issue with encoding of the script file?
tanepiper 2y ago
I find LLMs good at these kind of tasks, also converting between CSV to JSON for example (although you have to remind it not to be lazy and do the whole file)
VMG 2y ago
it is also pretty good at decompiling - try feeding it the output of https://godbolt.org/ https://godbolt.org/
- netsec_burn 2y ago
  I recently learned this too, just a few months ago. Ended up making a frontend so I could do it automatically: https://decompiler.zeroday.engineering/ https://decompiler.zeroday.engineering/
samstave 2y ago
Dont know if this will apply directly here, but -- As someone who is "not a developer" - I use the following process to help my: 1. I setup StyleGuide rules for the AI, telling it how to write out my files/scripts: - Always provide full path, description of function, invocation examples, and version number. - Frequently have it summarize and explain the project, project logic, and a particular file's functions. - Have it create a README.MD for the file/project - Tell it to give me mermaid diagrams and swim diagrams for the logic/code/project/process - Close prompts with "Review, Explain, Propose, Confirm, Execute" <-- This has it review the code/problem/prompt, explain what it understands, propose what its been asked to provide, confirm that its correct or I add mroe detail here - then execute and go with creating the artifacts. I do this because Claude and ChatGPT are FN malevelant in their ignoring of project files/context - and their hallucinate as soon as their context window/memory fills up. Further they very frequently "forget" to refer to the project context files uploaded/artifacts they themselves have proposed and written etc. But - asking for a readme with code mermaid and logic is helpful to keep me on track.
- viraptor 2y ago
  Agents like Aider or Plandex wrap that up nicely. They do the automatic review and have a very verbose description of the edit format. If you do that often manually, it may be worth testing their prepackaged approach.
dangsux 2y ago
[dead]
foxhop 2y ago
Here's a hint, STOP MINIFYING CODE! gzip over transport is enough.
- lifthrasiir 2y ago
  Not exactly, because you still have to pay any distinct identifier present in your code. Also many minifiers do constant folding and inlining and remove comments, any of which almost surely remove redundant or unused information to compress.
  - mistercow 2y ago
    I don’t think they’re saying that minifying provides no additional space savings, but rather that those additional savings are small and not worth the tradeoffs.
    - lifthrasiir 2y ago
      Not even that is true in my knowledge. For example a particular benchmark [1] demonstrates that many popular libraries benefit much from minification even after gzip compression, with the saving ranging from 35% to 75%. Sure, a small library would be fine without any minification or even compression, but otherwise minification is clearly beneficial. [1] https://github.com/privatenumber/minification-benchmarks https://github.com/privatenumber/minification-benchmarks
      - mistercow 2y ago
        I think you have to look at this in the context of an entire bundle or project, and then you have to weigh it against the download speeds you’re generally expecting for the users of your site or app. I agree that as a blanket statement “gzip is enough” is not technically correct, but I think it’s largely correct in spirit, in that people tend to reach for minification by default, without really thinking about what they’re gaining. If minifying saves you 200 KB overall, for example, and you expect your average user to have a 200 Mbps connection, you’re saving a grand total of 8 ms on page load, which is an imperceptible difference on its own. In exchange, you’re getting worse debugging, and worse error reporting.
shubhamjain 2y ago
An interesting use-case of this capability is refactoring, which, for me, ChatGPT has been unmistakably good at. It's amazing how I can throw garbage code I wrote at ChatGPT, ask it to refactor, and get clean code that I can use without worrying if it's going to work or not, because in 99% of cases it works without breaking anything.
- Sharlin 2y ago
  What language(s), out of interest?
  - shubhamjain 2y ago
    I use node.js, but I think it will work for anything. I recommend trying small chunks first vs dumping your whole file.
    - spacebanana7 2y ago
      I've had pretty good results dumping entire files in to Sonnet3.5. For example, "Here's my app.js file, please add an endpoint for one user to block another. Feel free to suggest schema changes. Please show me the full app.js with these changes implemented" The model seems to be great at figuring out frameworks and databases just by seeing the contents of a full app.js file. I do find this type of prompt works much better with Sonnet3.5 than GPT4o.
    - Sharlin 2y ago
      No, most other languages absolutely don’t work as well as JS, simply because there’s been less training material available. It’s useless with Rust, for example (hell, I’d be totally impressed if it has any idea how to appease the borrow checker!)
rpigab 2y ago
I can see some ways to use this and easily check that the LLM is not hallucinating parts of it, because you can ask the LLM to unminify (or deobfuscate) some component, then request unit tests to be written by the LLM, then humanly check that the unit tests are meaningful and that they don't miss things on the unminified code, then run the tests on the original minified version to confirm the LLM's work, maybe set up some mutation testing if it is relevant.
ziptron 2y ago
>I apologize, GPT-4, for mistakenly accusing you of making mistakes. I am testing large language models against a ground truth data set we created internally. Quite often when there is a mismatch, I realize the ground truth dataset is wrong, and I feel exactly like the author did.
- SoftTalker 2y ago
  Apologizing to a program seems rather silly though. Do you apologize to your compiler when you have a typo in your code, and have to make it do all that work again?
  - digging 2y ago
    If the compiler could listen and update its functions based on the tone of what I said to it, yes I probably would.
jehna1 2y ago
Author of HumanifyJS here! I've created specifically a LLM based tool for this, which uses LLMs on AST level to guarantee that the code keeps working after the unminification step: https://github.com/jehna/humanify https://github.com/jehna/humanify
- neoOpus 2y ago
  [dead]
- punkpeye 2y ago
  Looks useful! I will update the article to link to this tool. Thanks for sharing!
  - jehna1 2y ago
    Super, thank you for adding the link! It really helps to get people to find the tool
- firtoz 2y ago
  More tools should be built on ASTs, great work! I'm still waiting for the AST level version control tbh
  - jansvoboda11 2y ago
    Unison supposedly has an AST-aware version control system: https://www.unison-lang.org/ https://www.unison-lang.org/
    - timcobb 2y ago
      Wow this looks so cool.
    - LoganDark 2y ago
      content-addressed too, I think!
  - deleted 2y ago
nutanc 2y ago
Had tweeted about this sometime back. Found a component which was open source earlier and then removed and only minfied JS was provided. Give the JS to Claude and get the original component back. It even gave good class names to the component and function names. Actually this opens up a bigger question. What if I like an open source project but don't like its license. I can just prompt AI by giving it the open source code and ask it to rewrite it or write in some other language. Have to look up the rules if this is allowed or will be considered copying and how will a judge prove?
- lifthrasiir 2y ago
  Almost likely you would be found guilty because the intent matters. It is easy to check that the generated code is much similar to the original code, and you surely had a reason to bypass the original license. The exact legal reasoning would vary but any reasonable laywer would recommend you to do not. In the historic Google v. Oracle suit, the only actual code that was claimed to be copied was a trivial `rangeCheck` function, but Google's intent and other circumstances like the identical code structure and documentation made it much more complicated, and the final decision completely bypassed the copyrightability of APIs possibly for this reason.
BeefWellington 2y ago
You can do this on minified code with beautifiers like js-beautify, for example. It's not clear why we need to make this an LLM task when we have existing simple scripts to do it?
- mistercow 2y ago
  Beautifiers will restore whitespace, but they won’t rename variables by inferring their semantic meaning.
- NohatCoder 2y ago
  While this doesn't restore minified identifiers, like the LLM version claims to do, it tends to help a lot with understanding the code. Usually minified code still has the original identifiers in global function names, object attributes, DOM classes and a few other places where it is hard to guarantee no side effects of name mangling. This makes guessing the purpose of the remaining identifiers substantially easier to a human, and it is probably also the main reason why an LLM is capable of making reasonable guesses at what they could reasonably be called.
l5870uoo9y 2y ago
It is also shockingly good at converting/extracting data to CSV or JSON, but not JSONL. Even the less capable model, `gpt-4o-mini`, can "reliably" parse database schemas in various formats into CSV with the structure: ```csv table_name,column_name,data_type table_name,column_name1,data_type table_name,column_name2,data_type ... ``` I have been running it in production for months[1] as a way to import and optimize database schemas for AI consumption. This performs much better than including the `schema.sql` file in the prompt. [1]: https://www.sqlai.ai/app/datasources/add/database-schema/ai-optimized-import https://www.sqlai.ai/app/datasources/add/database-schema/ai-...
Mc91 2y ago
It is good at unminifying and "minifying" as well. I have been doing the Leetcode thing recently, and even became a subscriber to Leetcode. What I have been doing is I go through the Grind 75 list (Blind 75 successor list), look for the best big O time and space editorial answer, which often has a Java example, and then go to ChatGPT (I subscribe) or Perplexity (don't subscribe to Pro - yet) and say "convert this to Kotlin", which is the language I know best. Jetbrains IDE or Android Studio is capable of doing this, but Perplexity and ChatGPT are usually capable of doing this as well. Then I say "make this code more compact". Usually I give it some constraints too - keep the big O space and time complexity the same or lower it, keep the function signature of the assigned function the same, and keep the return explicit, make sure no Kotlin non-null assertions crop up. Sometimes I continually have it run these instructions on each version of the iterated code. I usually test that the code compiles and returns the correct answers for examples after each iteration of compacting. I also copy answers from one to the other - Perplexity to ChatGPT and then back to Perplexity. The code does not always compile, or give the right answers for the examples. Sometimes I overcompact it - what is clear in four lines becomes too confusing in three compacted lines. I'm not looking for the most compact answer, but a clear answer that is as compact as possible. One question asked about Strings and then later said, what if this was Unicode? So now for String manipulation questions I say assume the String is Unicode, and then at the end say show the answer for ASCII or Unicode. Sometimes the big O time is tricky - it is time O(m+n) say, but since m is always equal to or less than m in the program, it is actually O(n), and both Perplexity and ChatGPT can miss that until it is explained. People bemoan Leetcode as a waste of time, but I am wasting even less time with it, as ChatGPT and Perplexity are helping give me the code I will be demonstrating in interviews. The common advice I have heard from everywhere is don't waste time trying to figure out the answers myself - just look at the given answers, learn them, and then look for patterns (like binary search problems, which are usually similar), so that is what I am doing. Initially I was a ChatGPT and Perplexity skeptic for early versions of those sites, in terms of programming, as they stumbled more, but these self-contained examples and procedures they seem well-suited for. Not that they don't hallucinate or give programs that don't compile, or give the wrong answers sometimes, but it saves me time ultimately.
- flerchin 2y ago
nprateem 2y ago
And shockingly shit at writing articles that don't sound like essays.
pogue 2y ago
Only thing I'd like to suggest is an option to search for Windows 11 compatible machines. With MS cutting off support for Windows 10 next year, making sure a machine has the system requirements needed. However, I have seen a lot of sellers install W11 on non-compatible devices using a few tricks. I'm not sure how you check that in a search tool, but great job otherwise! I'll definitely be using this in the future (and I think you should pass everything through affiliate links! Pay for the upkeep at least)
jackconsidine 2y ago
That's interesting. It's gotten a lot better I guess. A little over a year ago, I tried to use GPT to assist me in deobfuscating malicious code (someone emailed me asking for help with their hacked WP site via custom plugin). I got much further just stepping through the code myself. After reading through this article, I tried again [0]. It gave me something to understand, though it's obfuscated enough to essentially eval unreadable strings (via the Window object), so it's not enough on it's own. Here was an excerpt of the report I sent to the person: > For what it’s worth, I dug through the heavily obfuscated JavaScript code and was able to decipher logic that it: > - Listens for a page load > - Invokes a facade of calculations which are in theory constant > - Redirects the page to a malicious site (unk or something) [0] https://chatgpt.com/share/f51fbd50-8df0-49e9-86ef-fc972bca6ba9 https://chatgpt.com/share/f51fbd50-8df0-49e9-86ef-fc972bca6b...
andrewmcwatters 2y ago
I’ve tried using LLMs to deobfuscate libraires like fingerprintjs-pro to understand what specific heuristics implementation details they use to detect bots. They mostly fail. A human reverse engineer will still do better.
camillomiller 2y ago
Most expensive unminify software in history
- GaggiX 2y ago
  A human would be even more expensive.
  - camillomiller 2y ago
    Those pesky humans! To be fair, I think it’s not if you consider the sunken costs of LLMs.
Tistel 2y ago
This might be fun: Train on java compiled to class files. Then go from class back to java. Or even: Train java compiled to class files, and have separate models that train from Clojure to class and Scala to class files. Then see if you can find some crufty (but important) old java project and go: crufty java -> class -> Clojure (or Scala). If you could do the same with source -> machine instructions, maybe COBAL to C++! or whatever.
- atemerev 2y ago
  I think there are already decompilers and code analyzers at NSA like this. For 10 years or so.
- igorguerrero 2y ago
  This is something you don't need AI for, there are many decompilers out there already as well. AI cannot even lint properly right now and you want it to decompile? good luck, there's too much hype going on people really think this is possible this year? In the end always remember it's just autocomplete, it's pretty terrible at translations that are not natural language to natural language. I worked on a natural language to SQL and it was impossible to make it consistently generate valid SQL for Postgres, and I'm talking about natural language to SQL not virtual machine instructions...
- sitkack 2y ago
  I agree, it is fun! LLM source recovery from binaries is thing. The amazing part is that they are pretty good at adding back meaningful variable names to the generated source code.
gnutrino 2y ago
The site the post mentions for the original code (https://reactive.network/hackathon https://reactive.network/hackathon) is an accessibility nightmare.
- lelandfe 2y ago
  The garbled text is included in the tree as relevant, pronounceable, and constantly changing text. Here's Chrome's accessibility tree: https://imgur.com/a/V1589Jr https://imgur.com/a/V1589Jr (I'd love if a screen reader user could upload some audio of how awful this sounds, by the by) Please use `aria-hidden="true"` for stuff like this, it just removes the element from the accessibility tree. I've also emailed Reactive a link to this thread. Here is a decent intro to ARIA things: https://www.smashingmagazine.com/2022/09/wai-aria-guide/ https://www.smashingmagazine.com/2022/09/wai-aria-guide/
  - phatskat 2y ago
    Big props for ARIA attributes - they’re so crucial for differently abled and impaired users. I’ve been combing through our project’s components lately to bring them up to design spec and have been taking a look at our accessibility - it’s so important and so easily missable for most engineers.
redbell 2y ago
That's an interesting finding so far! > The provided code is quite complex, but I'll break it down into a more understandable format, explaining its different parts and their functionalities. Reading the above statement generated by ChatGPT, I asked myself: Will we live to the day where these LLMs could take a large binary executable as input, read it, analyze it, understand it, then reply with the above statement? > I followed up asking to "implement equivalent code in TypeScript and make it human readable" and got the following response.. To my surprise, the response is not only good enough, but it is also very readable. What if this day came and we can ask these LLMs to rewrite the binary code in [almost] any programming language we want? This would be exciting, yet scary to just think about!
- Teever 2y ago
  https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
- martythemaniak 2y ago
  You should give it a try and report back! One easy way would be to take an open-source Android app, compile the APK, then decompile it and feed the bytecode to an LLM and ask it to write the java/kotlin equivalent and compare the source and LLM decoded one.
fasteddie31003 2y ago
I recognized this a few months back when I wanted to see the algorithm that a website used to do a calculation. I just put the minified JS in ChatGPT and figured it out pretty easily. Let's take this a few steps out. What happens when a LLM can clone a whole SAAS app? Let's say I wanted to clone HubSpot. If an LLM can interact with a browser and figure out how a UI works and take code hints from un-mimified code I think we could see all SAAS apps be commoditized. The backend would be proprietary, but it could figure out API formats and suggest a backend architecture. All this makes me think AI's are going to be a strong deflationary force in the future.
- sgt101 2y ago
  I was with you until: >If an LLM can interact with a browser and figure out how a UI works and take code hints from un-mimified code I think we could see all SAAS apps be commoditized. The backend would be proprietary, but it could figure out API formats and suggest a backend architecture. whoooha! that's a lot of probing and testing of the SAAS that would be required in order to see how it behaved. SAAS aren't algorithms, they operate over data that's unseen on the front end as well... >All this makes me think AI's are going to be a strong deflationary force in the future. I don't get this. I've literally never worked anywhere which had enough software engineers, we've been going on about software crisis for about 50 years and things are arguably worse than ever. The gap between the demand for good software (in the sense that allocating capital to producing it would be sensible) and the fulfillment of that demand is bigger than ever. We just don't have the mechanisms to make this work and to make it work at an economically viable level. Then we get AI to help us and everyone thinks that the economy will shrink?
  - IncreasePosts 2y ago
    You wouldn't necessarily need to do much probing - consider that the documentation would provide numerous hints to the agent as to what each endpoint was actually doing.
- TheKarateKid 2y ago
  Honestly, the value in most business software isn't the actual technology. It's the customer base and data held by the platforms. Someone could already easily clone HubSpot relatively cheaply even if they hired developers, but that doesn't mean it will be anywhere near successful.
nashashmi 2y ago
Looks like the end is here for security via obscurity.
Julesman 2y ago
"Usually, I would just powerthrough reading the minimized code..." Huh? Is this a thing? There are endless online code formatting sites. It takes two seconds. Why would anyone ever do this? I don't get it.
bredren 2y ago
Would have been cool if this had been used in that air con reverse engineering story yesterday. I noticed while reading the blog entry that the author described using a search engine multiple times and thought, "I would have asked ChatGPT first for that."
MetaverseClub 2y ago
wow, is openAI such a great magic to you?
antonoo 2y ago
Is this code available in ChatGPT's training data? Tried hard, couldn't find any similar code.
ninetyninenine 2y ago
This is an example of superior intellectual performance to humans. There’s no denying it. This task is intellectual. Does not involve rote memorization. There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from. The llm understands what it is unminifying and it is in general superior to humans on this regard. But only in this specific subject.
- indoordin0saur 2y ago
  This is just transforming text. > There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from. Are you sure about this? These can be easily generated from existing JS to use as a training set, not to mention the enormous amount of non-minified JS which is already used to train it.
- ozr 2y ago
  I'm bullish on AI, but I'm not convinced this is an example of what you're describing. The challenge of understanding minified code for a human comes from opaque variable names, awkward loops, minimal whitespacing, etc. These aren't things that a computer has trouble with: it's why we minify in the first place. Attention, as a scheme, should do great with it. I'd also say there is tons of minified/non-minified code out there. That's the goal of a map file. Given that OpenAI has specifically invested in web browsing and software development, I wouldn't be surprised if part of their training involved minified/unminified data.
  - ninetyninenine 2y ago
    minification and unminification is a heuristic process not an algorithmic one. It is akin to decompiling code or reverse engineering. It's a step beyond just your typical AI you see in a calculator.
  - etbebl 2y ago
    > These aren't things that a computer has trouble with They are irrelevant for executing the code, but they're probably pretty relevant for an LLM that is ingesting the code and text and inferring its function based on other examples it has seen. It's definitely more impressive that an LLM can succeed at this without the context of (correct) variable names than with them.
prologist11 2y ago
I have to ask the obvious question: how do you know the unminified code is semantically equivalent to the minified code? If someone knows how to verify LLM code transformations for semantic fidelity then I'd like to know because I think that would qualify as a major breakthrough for programming languages and semantics.
bdcravens 2y ago
I'm sure there's some number greater than zero of developers who are upset because they use minification as a means of obfuscation. Reminds me of the tool that was provided in older versions of ColdFusion that would "encrypt" your code. It was a very weak algorithm, and didn't take long for someone to write a decrypter. Nevertheless some people didn't like this, because they were using this tool, thinking it was safe for selling their code without giving access to source. (In the late 90s/early 2000s before open source was the overwhelming default)
- julianeon 2y ago
  I bet there's some kind of use case/arms race soon to happen, like this: Website offers some kind of contest which is partly dependent on obfuscated client side code. Clever contestants can now through it into ChatGPT to improve their chances. Now, it begins.
joshdavham 2y ago
Are there any serious security implications for this? Of course obfuscation through minification won't work anymore, but I'm not sure if that's really all that serious of an issue at the end of the day.
darepublic 2y ago
This is cool but I would have the worry that it got something wrong. Which is a general llm problem
emporas 2y ago
There is also topiary. From their website "The universal code formatter". I think it doesn't work with Javascript source for the moment, but it will surely work in the future. [1] https://topiary.tweag.io/ https://topiary.tweag.io/
lostdev 2y ago
Why not just use a beautifier?
gagabity 2y ago
Wonder how it does against other obfuscation and general decompiled code fixing.
cedws 2y ago
I’m hoping LLMs get better at decompiling/RE’ing assembly because it’s a very laborious process. Currently I don’t think they have enough in their training sets to be very good at it.
1vuio0pswjnm7 2y ago
Slightly off-topic but I remain perplexed at how "minified" Javascript is acceptable to software developers commenting online but terse code in any other language, e.g., one letter variable names, is unacceptable to a majority of software developer online commenters.
_rwo 2y ago
LLMs are great for self-contained boring tasks; recently I have started to refactor ruby tests with a simple prompt (getting rid of various rspec syntax in favor of more explicit notation at cost of code duplication - so kinda like unminifying things I guess) - works _ridiculously_ good as well

931 points by punkpeye 2y ago

This post basically says that I don't need to document my code anymore. No more comments, they can be generated automatically. Hurray!

skirge 2y ago
what if the comment doesn't match what you intended to write?
- deleted 2y ago
  [deleted]
- elicash 2y ago
  Maybe it's better for the comment to match the code you wrote, not the code you intended to write.
codr7 2y ago
Unfortunately the comments that could be generated are exactly the ones that should never be written. You want the comment to explain why, the information missing from the code.
- maeln 2y ago
  This is something I always disagreed with. In my experience, I rather read a short comment explaining what is the purpose of a block of code, than trying to decipher it. Yes, code "should speak for itself", but reading a comment is almost always faster than reading blocks of code. And then there is also documentation (if you include it in what you define as comment). I much rather go through a website, with a search function, example, description, made with some docgen tool, than having to go through a library or programming language source code every time I need to remember how to do X, or if object B has implement function Y ...
  - codr7 2y ago
    But the purpose is the Why; forced comments tend to tell you What the code does, which is better explained by the code itself. A comment that is incorrect can do a lot of damage, and they tend to get confused about implementation details over time.
    - amelius 2y ago
      I suspect you work at OpenAI and you're afraid that you will run out of training data.
    - maeln 2y ago
      For forced comment I fully agree, especially for function or class when the name already says whats on the tin.
  - jerf 2y ago
    It's just a rule of thumb, like anything else. In most code, "why" is the hard part; I see that you are incrementing that account by a penny from out of the blue, but why? When you are in code where "what" is the hard part, like an implementation of a book algorithm or some tricky performance optimization, then by all means comment that. Really all this rule amounts to is // Increment by a penny accountValue += 1 is a pointless comment, please don't do that. Schools had a way of accidentally teaching that by too-rigidly requiring "commented code", in situations where there wasn't much else to say, or situations where the students themselves didn't necessarily have a strong sense of "why". Any comment that isn't just literally "this is what the next line does" is probably useful to somebody at some point.
    - maeln 2y ago
      I do agree that documenting the why is way more important than the how/what. But having a short comment to summarize a block of code like: // Parse the filename and remove the extension let fext_re = Regex::new(r"(.\*)\.(.+)$").unwrap(); let page_cap = fext_re.captures(fname).unwrap(); let page_base_filename = page_cap.get(1).unwrap().as_str(); Is still useful. Instead of having to read the next few line of code, I already know what they are suppose to do and expect. It makes discovery, later down the line, easier.
sherburt3 2y ago
// This is an integer containing the accountId int accountId; We truly live in the future

blueyes 2y ago

Unminification (obfuscation removal) can also be applied to text. Most specialties develop a jargon that allows insiders to communicate complex ideas quickly; that shorthand excludes outsiders. Large language models can make specialist jargon transparent and thereby expand the circle of people whose understanding applies to specialized fields. Essentially, they solve the problem of mapping specialized, jargonized concepts to things the outside reader already knows. Anyone who wants to learn needs this, and I hope it will become part of students' learning paths.

nwoli 2y ago

Hopefully it can help do this on emscripten files too and help adblockers dechipher obfuscated code for that purpose

firtoz 2y ago
Likewise for css class names
- tgv 2y ago
  I can imagine that finetuning a model for this task could be very successful. Time for another AI startup.

Brechreiz 2y ago

[dead]

api 2y ago

Anyone working on decompiler LLMs? Seems like we could render all code open source. Training data would be easy to make in this case. Build tons of free GitHub code with various compilers and train on inverting compilation. This is a case where synthetic training data is appropriate and quite easy to generate. You could train the decompiler to just invert compilation and the use existing larger code LLMs to do things like add comments.

layer8 2y ago
It can’t really compensate for missing variable and function names, not to mention comments.
croes 2y ago
Unminifying isn't decompiling. It's just renaming variable and functions and inserting line breaks.
- api 2y ago
  No but it’s a baby brother of the same problem. Compiling is a much more complex transform but ultimately it is just a code transform.
  - jraph 2y ago
    It is true that compilation and minification are both code transformations (it's a correct reduction [1]), but this doesn't seem a very useful observation in this discussion. In the end, everything you do to something is an operation. But that's not very workable. In practice, compilation is often (not always, agreed!) from a language A to a lower level language B such that the runtime for language A can't run language B or vice-versa, if language A has a runtime at all. Minification is always from language A to the same language A. The implication is that in practice, deminification is not the same exercise as decompilation. You can even want to run a deminification phase after a decompilation phase, using two separate tools, because one tool will be good at translating back, and the other will be good at pretty printing. [1] https://en.wikipedia.org/wiki/Reductionism https://en.wikipedia.org/wiki/Reductionism

nnurmanov 2y ago

Yet another surprising side effects of LLMs.

8ndow 2y ago
Is it though? The developer tabs have an unminify button which yields similar results. JavaScript minification is not hard in any way and the guessing of variable names is not that hard given such a simple code example.
_joel 2y ago
If the training data's included both unminified and minified libs, then is it such a stretch?

interstice 2y ago

Have used Claude to reverse engineer some minified shopify javascript code recently. Definitely handy for unpicking things.

j_maffe 2y ago

LLMs are excellent at text transformation. It's their core strength and I don't see it being used enough.

PaulHoule 2y ago
Particularly those that are basically linear, that don’t involve major changes in the order of things or a deep consideration of relationships between things. They can’t sort a list but they can translate languages, for instance, given that a list sorted almost right is wrong but that we will often settle for an almost right translation.
msp26 2y ago
Isn't this already their main use case for business? We use them primarily for extracting structured data from other forms.
xanderlewis 2y ago
It’s not only their core strength — it’s what transformers were designed to do and, arguably, it’s all they can do. Any other supposed ability to reason or even retain knowledge (rather than simply regurgitate text without ‘understanding’ its intended meaning) is just a side effect of this superhuman ability.
- stavros 2y ago
  I see your point, but I think there's more to it. It's kind of like saying "all humans can do is perceive and produce sound, any other ability is just a side-effect". We might be focusing too much on their mechanism for "perception" and overlooking other capabilities they've developed.
  - mjburgess 2y ago
    Sure, but that claim wouldn't be true for humans, right? So it's a nonsequiteur. The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using inherited systems of meaning acquired as very young children in adaption within their environments, etc. In the case of transformers all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. If you project both of these into "problems an office worker has"-space, then they can appear simimlar -- but this projection is an incredibly dumb one, and offered as a sales pitch by charlatans looking to pretend that a system which can generate office emails can communicate.

runiq 2y ago

Please let this one have knock-on effects on reverse engineering.

skirge 2y ago
See Binary Ninja's Sidekick plugin

bravetraveler 2y ago

LLMs are good at modeling and transforming text, news at 11. AI proponent hypes AI. I could go on, but I shouldn't have been this sarcastic to start with

chx 2y ago
You should be. I am facepalming at the topic and every single comment in here. It's so full of holes, the Swiss dairy industry went out of business.

man4 2y ago

[dead]

albert_e 2y ago

Should the title say ChatGPT or gpt-4 (the model) instead of OpenAI (the company)?

johnisgood 2y ago
I think it should not say the name of the company, but either ChatGPT or GPT-x.
punkpeye 2y ago
It would have been a better title, yes.
ubj 2y ago
I agree, this would make the title more accurate.
cbm-vic-20 2y ago
I left my Kleenex next to the Xerox.
- taneq 2y ago
  Better Hoover it up!
  - dubcanada 2y ago
    All jokes aside, I've never heard anyone call vacuuming hoover. I wonder if that was a older statement?
    - justneedaname 2y ago
      In the UK it's very common
    - bdsa 2y ago
      More common in the UK
    - 2y ago

spacecadet 2y ago

I use LLMs to assist with reverse engineering all the time right now. From minified, to binary, alongside Ghidra, its very helpful.

GaggiX 2y ago
Can you provide more details? I'm curious about the performance and limitations of these models.
- spacecadet 2y ago
  Like all LLMs you greatly benefit from prior experience or you risk just falling for hallucinations which is a limitation of a non-deterministic black box, and degrades performance relative to the task. Ive commented in other threads, LLMs are great at amplifying my output in an area I already have domain knowledge in. I think this is why people fail to realize any gains or give up, they think it will unlock areas they dont fully understand themselves. Blind leading the blind problem.

smusamashah 2y ago

LLMS are trained to predict next text. But examples like these look like they have also 'learned patterns'. If rot13 is applied on this minified code, will LLM still find meaning in it? if it still could, its more than just next tokens. Need to try it. edit: chatgpt found out that its rot13 and couldn't explain the code directly without deobfuscating it first.

Tiberium 2y ago
Claude 3.5 Sonnet can natively speak double base64 encoded English. And I do mean it - you can double b64 encode something, send to it, and it'll respond as if it was normal English. Obviously base64 is a simpler transformation than rot13, but no GPT models can deal with double b64.
- mistercow 2y ago
  > Obviously base64 is a simpler transformation than rot13 Is it? It’s probably more obscuring from an LLM’s perspective, assuming the LLM has seen enough rot13 text during training. Spaces and punctuation are untouched by rot13, unlike base64, which means that word and sentence boundaries will still be denoted by tokens that denote those boundaries in plaintext.
- knome 2y ago
  it appears that openai's gpt-4 model can speak base64 as well. I jumped to your comment seeing if anyone else had tried it following the OP. double b64 I didn't try, but that is interesting. > $ ask4 ' what does dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw== decode to? ' > A "dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw==" is a Base64 encoded string. When decoded, It translates to "that is quite interesting" in English.
GaggiX 2y ago
I asked Claude 3.5 Sonnet a question in Italian in rot13 and it replied in Italian in rot13, there are a few typos but it's perfectly understandable.
- 2y ago

ervinxie 2y ago

LLMs are very good at text reading. LLMs read tokenized text, while human use eyes to view words. Another scenario is that ChatGPT is good at analyzing cpp template error messages, which are usually long and hard to understand for human.

lifthrasiir 2y ago

JS minification is fairly mechanical and comparably simple, so the inversion should be relatively easy. It would be of course tedious enough to be manually done in general, but transformations themselves are fairly limited so it is possible to read them only with some notes to track mangled identifiers. A more general unminification or unobfuscation still seems to be an open problem. I wrote handful of programs that are intentionally obfuscated in the past and ChatGPT couldn't understand them even at the surface level in my experience. For example, a gist for my 160-byte-long Brainfuck interpreter in C had some comment trying to use GPT-4 to explain the code [1], but the "clarified version" bore zero similarity with the original code... [1] https://gist.github.com/lifthrasiir/596667#gistcomment-4751211 https://gist.github.com/lifthrasiir/596667#gistcomment-47512...

panda-giddiness 2y ago
> JS minification is fairly mechanical and comparably simple, so the inversion should be relatively easy. Just because a task is simple doesn't mean its inverse need be. Examples: - multiplication / prime factorization - deriving / integrating - remembering the past / predicting the future Code unobfuscation is clearly one of those difficult inverse problems, as it can be easily exacerbated by any of the following problems: - bugs - unused or irrelevant routines - incorrect implementations that incidentally give the right results In that sense, it would be fortunate if chatGPT could give decent results at unobfuscating code, as there is no a priori expectation that it should be able to do so. It's good that you've also checked chatGPT's code unobfuscation capabilities on a more difficult problem, but I think you've only discovered an upper limit. I wouldn't consider the example in the OP to be trivial.
- lifthrasiir 2y ago
  Of course, it is not generalizable! In my experience though, most minifiers do only the following: - Whitespace removal, which is trivially invertible. - Comment removal, which we never expect to recover via unminification. - Renaming to shorter names, which is tedious to track but still mechanical. And most minifiers have little understanding of underlying types anyway, so they are usually very conservative and rarely reuse the same mangled identifier for multiple uses. (Google Closure Compiler is a significant counterexample here, but it is also known to be much slower.) - Constant folding and inlining, which is annoying but can be still tracked. Again, most minifiers are limited in their reasoning to do extensive constant folding and inlining. - Language-specific transformations, like turning `a; b; c;` into `a, b, c;` and `if (a) b;` into `a && b;` whenever possible. They will be hard to understand if you don't know in advance, but there aren't too many of them anyway. As a result, minified code still remains comparably human-readable with some note taking and perseverance. And since these transformations are mostly local, I would expect LLMs can pick them up by their own as well. (But why? Because I do inspect such programs fairly regularly, for example for comments like https://news.ycombinator.com/item?id=39066262 https://news.ycombinator.com/item?id=39066262)

Tepix 2y ago

punkpeye could also have asked the LLM to replace the cryptic function and variable names with nice ones. I'm hopeful it would have done a good job.

fergie 2y ago

* takes out soap box and stands on it * We should go back to uncompiled JavaScript code, our democracy depends on it.

lifthrasiir 2y ago
Or learn to read minified JS code, which is actually not that difficult! (But you do have to take notes to track identifiers.)

creesch 2y ago

This is very close to how I often use LLMs [0]. A first step in deciphering code where I otherwise would need to, to use the authors words, power through reading the code myself. It has been incredibly liberating to just feed it a spaghetti mess, ask to detangle it in a more readable way and go from there. As the author also discovered, LLMs will sometimes miss some details, but that is alright as I will be catching those myself. Another use case is when I understand what the code does, but can't quite wrap my head around why it is done in that specific way. Specifically, where the author of the code is no longer with the company. I will then simply put the method in the LLM chat, explain what it does, and just ask it why some things might be done in a specific way. Again, it isn't always perfect, but more often than not it comes with explanations that actually make sense, hold up under scrutiny and give me new insights. It actually has prevented me once or twice from refactoring something in a way that would have caught me headaches down the line. [0] chatGPT and more recently openwebUI as a front end to various other models (Claude variants mostly) to see the differences. Also allows for some fun concepts of having different models review each others answers.

xanderlewis 2y ago

Is there any reason why it’s ‘OpenAI’ in the title rather than ‘ChatGPT’?

__m_danziger__ 2y ago
[dead]
philipwhiuk 2y ago
It's the title of the blog post.
- xanderlewis 2y ago
  Yeah... that's what I'm asking about

eqvinox 2y ago

Okay, but if the unminified code doesn't match the minified code (as noted at the end "it looks like LLM response overlooked a few implementation details"), that massively diminishes its usefulness — especially since in a lot of cases you can't trivially run the code and look for differences like the article does. [ed.: looks like this was an encoding problem, cf. thread below. I'm still a little concerned about correctness though.]

punkpeye 2y ago
This refers to the fact that ChatGPT generated version is missing some characters that are used in the original example. Namely, [looks like HN does not allow me to paste unicode characters, but I am referring to the block characters] can be seen in their version, but cannot be seen in the ChatGPT generated version. However, it very well might be that it is simply because I didn't include all the necessary context. Discrediting the entire output because a few missing characters would be very pedantic. Otherwise, the output is identical as far as I can tell by looking at it.
- Joker_vD 2y ago
  It's because the author miscopy-pasted the original code: those "â–‘â–’â–“â–ˆ" at the end of the O5 string are supposed to be the block characters. E.g. "â–‘" in Windows-1252 [0] is 0xE2 0x96 0xE2 which, in UTF-8, exactly the encoding for U+2592 MEDIUM SHADE [1]. [0] https://en.wikipedia.org/wiki/Windows-1252#Character_set https://en.wikipedia.org/wiki/Windows-1252#Character_set [1] https://www.compart.com/en/unicode/U+2592 https://www.compart.com/en/unicode/U+2592
  - punkpeye 2y ago
    Possible that this is the mistake. However, I don't think I miscopied the original code. https://reactive.network/assets/index-8b4ef4ac.js https://reactive.network/assets/index-8b4ef4ac.js If you look for `oahkbdpqwmZO0QLCJUYXzcvunxrjft` in the output, you should see that those characters appear exactly like that. Maybe an issue with encoding of the script file?

tanepiper 2y ago

I find LLMs good at these kind of tasks, also converting between CSV to JSON for example (although you have to remind it not to be lazy and do the whole file)

VMG 2y ago

it is also pretty good at decompiling - try feeding it the output of https://godbolt.org/ https://godbolt.org/

netsec_burn 2y ago
I recently learned this too, just a few months ago. Ended up making a frontend so I could do it automatically: https://decompiler.zeroday.engineering/ https://decompiler.zeroday.engineering/

samstave 2y ago

Dont know if this will apply directly here, but -- As someone who is "not a developer" - I use the following process to help my: 1. I setup StyleGuide rules for the AI, telling it how to write out my files/scripts: - Always provide full path, description of function, invocation examples, and version number. - Frequently have it summarize and explain the project, project logic, and a particular file's functions. - Have it create a README.MD for the file/project - Tell it to give me mermaid diagrams and swim diagrams for the logic/code/project/process - Close prompts with "Review, Explain, Propose, Confirm, Execute" <-- This has it review the code/problem/prompt, explain what it understands, propose what its been asked to provide, confirm that its correct or I add mroe detail here - then execute and go with creating the artifacts. I do this because Claude and ChatGPT are FN malevelant in their ignoring of project files/context - and their hallucinate as soon as their context window/memory fills up. Further they very frequently "forget" to refer to the project context files uploaded/artifacts they themselves have proposed and written etc. But - asking for a readme with code mermaid and logic is helpful to keep me on track.

viraptor 2y ago
Agents like Aider or Plandex wrap that up nicely. They do the automatic review and have a very verbose description of the edit format. If you do that often manually, it may be worth testing their prepackaged approach.

dangsux 2y ago

[dead]

foxhop 2y ago

Here's a hint, STOP MINIFYING CODE! gzip over transport is enough.

lifthrasiir 2y ago
Not exactly, because you still have to pay any distinct identifier present in your code. Also many minifiers do constant folding and inlining and remove comments, any of which almost surely remove redundant or unused information to compress.
- mistercow 2y ago
  I don’t think they’re saying that minifying provides no additional space savings, but rather that those additional savings are small and not worth the tradeoffs.
  - lifthrasiir 2y ago
    Not even that is true in my knowledge. For example a particular benchmark [1] demonstrates that many popular libraries benefit much from minification even after gzip compression, with the saving ranging from 35% to 75%. Sure, a small library would be fine without any minification or even compression, but otherwise minification is clearly beneficial. [1] https://github.com/privatenumber/minification-benchmarks https://github.com/privatenumber/minification-benchmarks
    - mistercow 2y ago
      I think you have to look at this in the context of an entire bundle or project, and then you have to weigh it against the download speeds you’re generally expecting for the users of your site or app. I agree that as a blanket statement “gzip is enough” is not technically correct, but I think it’s largely correct in spirit, in that people tend to reach for minification by default, without really thinking about what they’re gaining. If minifying saves you 200 KB overall, for example, and you expect your average user to have a 200 Mbps connection, you’re saving a grand total of 8 ms on page load, which is an imperceptible difference on its own. In exchange, you’re getting worse debugging, and worse error reporting.

shubhamjain 2y ago

An interesting use-case of this capability is refactoring, which, for me, ChatGPT has been unmistakably good at. It's amazing how I can throw garbage code I wrote at ChatGPT, ask it to refactor, and get clean code that I can use without worrying if it's going to work or not, because in 99% of cases it works without breaking anything.

Sharlin 2y ago
What language(s), out of interest?
- shubhamjain 2y ago
  I use node.js, but I think it will work for anything. I recommend trying small chunks first vs dumping your whole file.
  - spacebanana7 2y ago
    I've had pretty good results dumping entire files in to Sonnet3.5. For example, "Here's my app.js file, please add an endpoint for one user to block another. Feel free to suggest schema changes. Please show me the full app.js with these changes implemented" The model seems to be great at figuring out frameworks and databases just by seeing the contents of a full app.js file. I do find this type of prompt works much better with Sonnet3.5 than GPT4o.
  - Sharlin 2y ago
    No, most other languages absolutely don’t work as well as JS, simply because there’s been less training material available. It’s useless with Rust, for example (hell, I’d be totally impressed if it has any idea how to appease the borrow checker!)

rpigab 2y ago

I can see some ways to use this and easily check that the LLM is not hallucinating parts of it, because you can ask the LLM to unminify (or deobfuscate) some component, then request unit tests to be written by the LLM, then humanly check that the unit tests are meaningful and that they don't miss things on the unminified code, then run the tests on the original minified version to confirm the LLM's work, maybe set up some mutation testing if it is relevant.

ziptron 2y ago

>I apologize, GPT-4, for mistakenly accusing you of making mistakes. I am testing large language models against a ground truth data set we created internally. Quite often when there is a mismatch, I realize the ground truth dataset is wrong, and I feel exactly like the author did.

SoftTalker 2y ago
Apologizing to a program seems rather silly though. Do you apologize to your compiler when you have a typo in your code, and have to make it do all that work again?
- digging 2y ago
  If the compiler could listen and update its functions based on the tone of what I said to it, yes I probably would.

jehna1 2y ago

Author of HumanifyJS here! I've created specifically a LLM based tool for this, which uses LLMs on AST level to guarantee that the code keeps working after the unminification step: https://github.com/jehna/humanify https://github.com/jehna/humanify

neoOpus 2y ago
[dead]
punkpeye 2y ago
Looks useful! I will update the article to link to this tool. Thanks for sharing!
- jehna1 2y ago
  Super, thank you for adding the link! It really helps to get people to find the tool
firtoz 2y ago
More tools should be built on ASTs, great work! I'm still waiting for the AST level version control tbh
- jansvoboda11 2y ago
  Unison supposedly has an AST-aware version control system: https://www.unison-lang.org/ https://www.unison-lang.org/
  - timcobb 2y ago
    Wow this looks so cool.
  - LoganDark 2y ago
    content-addressed too, I think!
- deleted 2y ago

nutanc 2y ago

Had tweeted about this sometime back. Found a component which was open source earlier and then removed and only minfied JS was provided. Give the JS to Claude and get the original component back. It even gave good class names to the component and function names. Actually this opens up a bigger question. What if I like an open source project but don't like its license. I can just prompt AI by giving it the open source code and ask it to rewrite it or write in some other language. Have to look up the rules if this is allowed or will be considered copying and how will a judge prove?

lifthrasiir 2y ago
Almost likely you would be found guilty because the intent matters. It is easy to check that the generated code is much similar to the original code, and you surely had a reason to bypass the original license. The exact legal reasoning would vary but any reasonable laywer would recommend you to do not. In the historic Google v. Oracle suit, the only actual code that was claimed to be copied was a trivial `rangeCheck` function, but Google's intent and other circumstances like the identical code structure and documentation made it much more complicated, and the final decision completely bypassed the copyrightability of APIs possibly for this reason.

BeefWellington 2y ago

You can do this on minified code with beautifiers like js-beautify, for example. It's not clear why we need to make this an LLM task when we have existing simple scripts to do it?

mistercow 2y ago
Beautifiers will restore whitespace, but they won’t rename variables by inferring their semantic meaning.
NohatCoder 2y ago
While this doesn't restore minified identifiers, like the LLM version claims to do, it tends to help a lot with understanding the code. Usually minified code still has the original identifiers in global function names, object attributes, DOM classes and a few other places where it is hard to guarantee no side effects of name mangling. This makes guessing the purpose of the remaining identifiers substantially easier to a human, and it is probably also the main reason why an LLM is capable of making reasonable guesses at what they could reasonably be called.

l5870uoo9y 2y ago

It is also shockingly good at converting/extracting data to CSV or JSON, but not JSONL. Even the less capable model, `gpt-4o-mini`, can "reliably" parse database schemas in various formats into CSV with the structure: ```csv table_name,column_name,data_type table_name,column_name1,data_type table_name,column_name2,data_type ... ``` I have been running it in production for months[1] as a way to import and optimize database schemas for AI consumption. This performs much better than including the `schema.sql` file in the prompt. [1]: https://www.sqlai.ai/app/datasources/add/database-schema/ai-optimized-import https://www.sqlai.ai/app/datasources/add/database-schema/ai-...

Mc91 2y ago

It is good at unminifying and "minifying" as well. I have been doing the Leetcode thing recently, and even became a subscriber to Leetcode. What I have been doing is I go through the Grind 75 list (Blind 75 successor list), look for the best big O time and space editorial answer, which often has a Java example, and then go to ChatGPT (I subscribe) or Perplexity (don't subscribe to Pro - yet) and say "convert this to Kotlin", which is the language I know best. Jetbrains IDE or Android Studio is capable of doing this, but Perplexity and ChatGPT are usually capable of doing this as well. Then I say "make this code more compact". Usually I give it some constraints too - keep the big O space and time complexity the same or lower it, keep the function signature of the assigned function the same, and keep the return explicit, make sure no Kotlin non-null assertions crop up. Sometimes I continually have it run these instructions on each version of the iterated code. I usually test that the code compiles and returns the correct answers for examples after each iteration of compacting. I also copy answers from one to the other - Perplexity to ChatGPT and then back to Perplexity. The code does not always compile, or give the right answers for the examples. Sometimes I overcompact it - what is clear in four lines becomes too confusing in three compacted lines. I'm not looking for the most compact answer, but a clear answer that is as compact as possible. One question asked about Strings and then later said, what if this was Unicode? So now for String manipulation questions I say assume the String is Unicode, and then at the end say show the answer for ASCII or Unicode. Sometimes the big O time is tricky - it is time O(m+n) say, but since m is always equal to or less than m in the program, it is actually O(n), and both Perplexity and ChatGPT can miss that until it is explained. People bemoan Leetcode as a waste of time, but I am wasting even less time with it, as ChatGPT and Perplexity are helping give me the code I will be demonstrating in interviews. The common advice I have heard from everywhere is don't waste time trying to figure out the answers myself - just look at the given answers, learn them, and then look for patterns (like binary search problems, which are usually similar), so that is what I am doing. Initially I was a ChatGPT and Perplexity skeptic for early versions of those sites, in terms of programming, as they stumbled more, but these self-contained examples and procedures they seem well-suited for. Not that they don't hallucinate or give programs that don't compile, or give the wrong answers sometimes, but it saves me time ultimately.

flerchin 2y ago

nprateem 2y ago

And shockingly shit at writing articles that don't sound like essays.

pogue 2y ago

Only thing I'd like to suggest is an option to search for Windows 11 compatible machines. With MS cutting off support for Windows 10 next year, making sure a machine has the system requirements needed. However, I have seen a lot of sellers install W11 on non-compatible devices using a few tricks. I'm not sure how you check that in a search tool, but great job otherwise! I'll definitely be using this in the future (and I think you should pass everything through affiliate links! Pay for the upkeep at least)

jackconsidine 2y ago

That's interesting. It's gotten a lot better I guess. A little over a year ago, I tried to use GPT to assist me in deobfuscating malicious code (someone emailed me asking for help with their hacked WP site via custom plugin). I got much further just stepping through the code myself. After reading through this article, I tried again [0]. It gave me something to understand, though it's obfuscated enough to essentially eval unreadable strings (via the Window object), so it's not enough on it's own. Here was an excerpt of the report I sent to the person: > For what it’s worth, I dug through the heavily obfuscated JavaScript code and was able to decipher logic that it: > - Listens for a page load > - Invokes a facade of calculations which are in theory constant > - Redirects the page to a malicious site (unk or something) [0] https://chatgpt.com/share/f51fbd50-8df0-49e9-86ef-fc972bca6ba9 https://chatgpt.com/share/f51fbd50-8df0-49e9-86ef-fc972bca6b...

andrewmcwatters 2y ago

I’ve tried using LLMs to deobfuscate libraires like fingerprintjs-pro to understand what specific heuristics implementation details they use to detect bots. They mostly fail. A human reverse engineer will still do better.

camillomiller 2y ago

Most expensive unminify software in history

GaggiX 2y ago
A human would be even more expensive.
- camillomiller 2y ago
  Those pesky humans! To be fair, I think it’s not if you consider the sunken costs of LLMs.

Tistel 2y ago

This might be fun: Train on java compiled to class files. Then go from class back to java. Or even: Train java compiled to class files, and have separate models that train from Clojure to class and Scala to class files. Then see if you can find some crufty (but important) old java project and go: crufty java -> class -> Clojure (or Scala). If you could do the same with source -> machine instructions, maybe COBAL to C++! or whatever.

atemerev 2y ago
I think there are already decompilers and code analyzers at NSA like this. For 10 years or so.
igorguerrero 2y ago
This is something you don't need AI for, there are many decompilers out there already as well. AI cannot even lint properly right now and you want it to decompile? good luck, there's too much hype going on people really think this is possible this year? In the end always remember it's just autocomplete, it's pretty terrible at translations that are not natural language to natural language. I worked on a natural language to SQL and it was impossible to make it consistently generate valid SQL for Postgres, and I'm talking about natural language to SQL not virtual machine instructions...
sitkack 2y ago
I agree, it is fun! LLM source recovery from binaries is thing. The amazing part is that they are pretty good at adding back meaningful variable names to the generated source code.

gnutrino 2y ago

The site the post mentions for the original code (https://reactive.network/hackathon https://reactive.network/hackathon) is an accessibility nightmare.

lelandfe 2y ago
The garbled text is included in the tree as relevant, pronounceable, and constantly changing text. Here's Chrome's accessibility tree: https://imgur.com/a/V1589Jr https://imgur.com/a/V1589Jr (I'd love if a screen reader user could upload some audio of how awful this sounds, by the by) Please use `aria-hidden="true"` for stuff like this, it just removes the element from the accessibility tree. I've also emailed Reactive a link to this thread. Here is a decent intro to ARIA things: https://www.smashingmagazine.com/2022/09/wai-aria-guide/ https://www.smashingmagazine.com/2022/09/wai-aria-guide/
- phatskat 2y ago
  Big props for ARIA attributes - they’re so crucial for differently abled and impaired users. I’ve been combing through our project’s components lately to bring them up to design spec and have been taking a look at our accessibility - it’s so important and so easily missable for most engineers.

redbell 2y ago

That's an interesting finding so far! > The provided code is quite complex, but I'll break it down into a more understandable format, explaining its different parts and their functionalities. Reading the above statement generated by ChatGPT, I asked myself: Will we live to the day where these LLMs could take a large binary executable as input, read it, analyze it, understand it, then reply with the above statement? > I followed up asking to "implement equivalent code in TypeScript and make it human readable" and got the following response.. To my surprise, the response is not only good enough, but it is also very readable. What if this day came and we can ask these LLMs to rewrite the binary code in [almost] any programming language we want? This would be exciting, yet scary to just think about!

Teever 2y ago
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
martythemaniak 2y ago
You should give it a try and report back! One easy way would be to take an open-source Android app, compile the APK, then decompile it and feed the bytecode to an LLM and ask it to write the java/kotlin equivalent and compare the source and LLM decoded one.

fasteddie31003 2y ago

I recognized this a few months back when I wanted to see the algorithm that a website used to do a calculation. I just put the minified JS in ChatGPT and figured it out pretty easily. Let's take this a few steps out. What happens when a LLM can clone a whole SAAS app? Let's say I wanted to clone HubSpot. If an LLM can interact with a browser and figure out how a UI works and take code hints from un-mimified code I think we could see all SAAS apps be commoditized. The backend would be proprietary, but it could figure out API formats and suggest a backend architecture. All this makes me think AI's are going to be a strong deflationary force in the future.

sgt101 2y ago
I was with you until: >If an LLM can interact with a browser and figure out how a UI works and take code hints from un-mimified code I think we could see all SAAS apps be commoditized. The backend would be proprietary, but it could figure out API formats and suggest a backend architecture. whoooha! that's a lot of probing and testing of the SAAS that would be required in order to see how it behaved. SAAS aren't algorithms, they operate over data that's unseen on the front end as well... >All this makes me think AI's are going to be a strong deflationary force in the future. I don't get this. I've literally never worked anywhere which had enough software engineers, we've been going on about software crisis for about 50 years and things are arguably worse than ever. The gap between the demand for good software (in the sense that allocating capital to producing it would be sensible) and the fulfillment of that demand is bigger than ever. We just don't have the mechanisms to make this work and to make it work at an economically viable level. Then we get AI to help us and everyone thinks that the economy will shrink?
- IncreasePosts 2y ago
  You wouldn't necessarily need to do much probing - consider that the documentation would provide numerous hints to the agent as to what each endpoint was actually doing.
TheKarateKid 2y ago
Honestly, the value in most business software isn't the actual technology. It's the customer base and data held by the platforms. Someone could already easily clone HubSpot relatively cheaply even if they hired developers, but that doesn't mean it will be anywhere near successful.

nashashmi 2y ago

Looks like the end is here for security via obscurity.

Julesman 2y ago

"Usually, I would just powerthrough reading the minimized code..." Huh? Is this a thing? There are endless online code formatting sites. It takes two seconds. Why would anyone ever do this? I don't get it.

bredren 2y ago

Would have been cool if this had been used in that air con reverse engineering story yesterday. I noticed while reading the blog entry that the author described using a search engine multiple times and thought, "I would have asked ChatGPT first for that."

MetaverseClub 2y ago

wow, is openAI such a great magic to you?

antonoo 2y ago

Is this code available in ChatGPT's training data? Tried hard, couldn't find any similar code.

ninetyninenine 2y ago

This is an example of superior intellectual performance to humans. There’s no denying it. This task is intellectual. Does not involve rote memorization. There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from. The llm understands what it is unminifying and it is in general superior to humans on this regard. But only in this specific subject.

indoordin0saur 2y ago
This is just transforming text. > There are not tons and tons of data pairs on the web of minimized code and unminified code for llms to learn from. Are you sure about this? These can be easily generated from existing JS to use as a training set, not to mention the enormous amount of non-minified JS which is already used to train it.
ozr 2y ago
I'm bullish on AI, but I'm not convinced this is an example of what you're describing. The challenge of understanding minified code for a human comes from opaque variable names, awkward loops, minimal whitespacing, etc. These aren't things that a computer has trouble with: it's why we minify in the first place. Attention, as a scheme, should do great with it. I'd also say there is tons of minified/non-minified code out there. That's the goal of a map file. Given that OpenAI has specifically invested in web browsing and software development, I wouldn't be surprised if part of their training involved minified/unminified data.
- ninetyninenine 2y ago
  minification and unminification is a heuristic process not an algorithmic one. It is akin to decompiling code or reverse engineering. It's a step beyond just your typical AI you see in a calculator.
- etbebl 2y ago
  > These aren't things that a computer has trouble with They are irrelevant for executing the code, but they're probably pretty relevant for an LLM that is ingesting the code and text and inferring its function based on other examples it has seen. It's definitely more impressive that an LLM can succeed at this without the context of (correct) variable names than with them.

prologist11 2y ago

I have to ask the obvious question: how do you know the unminified code is semantically equivalent to the minified code? If someone knows how to verify LLM code transformations for semantic fidelity then I'd like to know because I think that would qualify as a major breakthrough for programming languages and semantics.

bdcravens 2y ago

I'm sure there's some number greater than zero of developers who are upset because they use minification as a means of obfuscation. Reminds me of the tool that was provided in older versions of ColdFusion that would "encrypt" your code. It was a very weak algorithm, and didn't take long for someone to write a decrypter. Nevertheless some people didn't like this, because they were using this tool, thinking it was safe for selling their code without giving access to source. (In the late 90s/early 2000s before open source was the overwhelming default)

julianeon 2y ago
I bet there's some kind of use case/arms race soon to happen, like this: Website offers some kind of contest which is partly dependent on obfuscated client side code. Clever contestants can now through it into ChatGPT to improve their chances. Now, it begins.

joshdavham 2y ago

Are there any serious security implications for this? Of course obfuscation through minification won't work anymore, but I'm not sure if that's really all that serious of an issue at the end of the day.

darepublic 2y ago

This is cool but I would have the worry that it got something wrong. Which is a general llm problem

emporas 2y ago

There is also topiary. From their website "The universal code formatter". I think it doesn't work with Javascript source for the moment, but it will surely work in the future. [1] https://topiary.tweag.io/ https://topiary.tweag.io/

lostdev 2y ago

Why not just use a beautifier?

gagabity 2y ago

Wonder how it does against other obfuscation and general decompiled code fixing.

cedws 2y ago

I’m hoping LLMs get better at decompiling/RE’ing assembly because it’s a very laborious process. Currently I don’t think they have enough in their training sets to be very good at it.

1vuio0pswjnm7 2y ago

Slightly off-topic but I remain perplexed at how "minified" Javascript is acceptable to software developers commenting online but terse code in any other language, e.g., one letter variable names, is unacceptable to a majority of software developer online commenters.

_rwo 2y ago

LLMs are great for self-contained boring tasks; recently I have started to refactor ruby tests with a simple prompt (getting rid of various rspec syntax in favor of more explicit notation at cost of code duplication - so kinda like unminifying things I guess) - works _ridiculously_ good as well