4 ms·
Beware of Base64 Encoded Strings
- behroozam 3y agoSpecially useful when you encode ssh key to base64 in github to use it in github actions. I spent hour to figure it why the key is not working and the answer is -w0
- woliveirajr 3y agoTL;DR: the base64 util breaks lines after a certain number of columns. Use a parameter to specify "don't break". [0] [0] https://www.man7.org/linux/man-pages/man1/base64.1.html https://www.man7.org/linux/man-pages/man1/base64.1.html
- loloquwowndueo 3y agoTitle sounds like click bait :) . There is nothing wrong with base64 strings. The base64 command, on the other hand, will line-split its output if it’s very long. Using the -w0 param as the article discovered and shared is the way to go - but this is dependent on the specific b64 tooling that was used.
- MatthiasPortzel 3y agoI’ve never encountered this issue because macOS/BSD `base64` doesn’t have this default. So I agree, a bad title for an issue with one tool.
- rob74 3y agoYeah, the title should be "Beware of strings encoded with base64" (as in the specific utility, not the general encoding).
- taftster 3y agoBeware of long strings encoded with certain base64 tools.
- efrecon 3y agoOr better : use the --wrap option so it gets clearer that no wrapping on long lines happens .
- garritfra 3y agoThis is exactly what the article suggests.
- milliams 3y agoI think they mean use "--wrap=0" instead of "-w0" as it's a little better self-documenting for those who come across your script. In general I always try to use the long-form options in scripts as readability is more important that conciseness.
- deciduously 3y agoThe article notes this was not possible in their specific context, as only the short form is implemented in busybox.
- nebulous1 3y agoThe commit that they have a screenshot of notes this, but it isn't in the article text. FWIW I can't read that screenshot with zoom level 100% on my desktop, but I can if I zoom further or open it in a new window (it isn't clickable though).
- efrecon 3y agoThat's what I meant indeed. The article never mentions busybox in the text, does it?
- bravetraveler 3y agoI regularly use base64 to {en,de}code binary files that I want to copy and paste - laziest rsync imaginable Surprised I haven't run into this wrapping or worse, lol
- timvisee 3y agoDoes it still do this when piping into things?
- BlueTemplar 3y agoAny good reason to still use base64 compared to base32 ? Since base64 seems to be not much more compact in practice (?), but also fails at transmission through the lossy channel that is human communication. (Arguably, there's also hexadecimal, but compactness starts becoming an issue there ?)
- deleted 3y ago[deleted]
- blueflow 3y agoRTFM, from `man base64`: -w, --wrap=COLS wrap encoded lines after COLS character (default 76). Use 0 to disable line wrapping The manpage is about one screen size large. 1 minute read.
- echoangle 3y agoSure, but it’s still a bad default IMO, the expected behavior would be an unformatted string without any linebreaks
- deleted 3y ago[deleted]
- oniony 3y agoSo you pass in a 1gig binary you want a very, very long line of text that many text editors would choke on? The purpose of base64 is to turn binary into text that can be embedded in normal text messages such as email and Usenet. A right margin of about 80 characters is very much what one should expect such a tool to default to.
- scrollaway 3y agoIf you're passing a 1GB binary to base64 you're ... incredibly unlikely to be dealing with the resulting base64 manually (copypaste, text editor, whatever). So, yes, the expected default would still be no linebreaks. The primary purpose of these tools have changed over time as the environment has changed. You know nobody uses usenet anymore, and nobody copypastes base64 into emails anymore, right?
- blueflow 3y agoThe default will not be changed because it breaks the original use cases. Which some people still have even if you do not.
- PinguTS 3y agoFor old farts like me who used to use News groups where binary data was first encoded in uuencode and later with the innovative base64, it is totally expected behavior. Also look into the text files für the public/private keys. It is always this, because the screen has only 80 characters in width.
- jacksonhacker 3y ago[dead]
- nickcw 3y agoThe base64 command wraps its output every 76 characters by default. This is because of its original use which was to encode binary files to include in emails. Very long lines in emails don't work well. See the precursor to base64 which is the uuencode command which you probably have installed too if you have the base64 command. base64 is a better, standardised version of uuencode. Use --wrap=0 to disable.
- silvestrov 3y ago> $(echo -n $BASIC_AUTH_USERNAME:$BASIC_AUTH_PASSWORD | base64) I would really like some quoting of that argument. Otherwise a weird password can really break things.
- Quimoniz 3y agoAnother prime example on the typical arrogance of this trade, of computer scientists: My brain immediately goes like: You nitwit, base64 comes back from E-Mails, where you would work with Mail-Server's conception that your mails were all written, legible characters. Of course you should expect newlines, potentially whitespaces and tabs in it. But that would be stupid of me. There's so much old knowledge and know-how we carry along for many decades - around 50 years in case of mail and SMTP. Unix and Linux are a specially egregious example, with so many conventions.... just ran into a long standing bug a few days ago, with some GECOS implementation (that's 62 years old stuff). We do a lot of good in throwing out the now unnecessary old stuff and separating the wheat from the chaff. Again and again. In so far: Thanks for the pointer on that Base64 appears strange (also please do not let us talk about groff).
- deleted 3y ago[deleted]
- ColinWright 3y agoIt's "discussions" like this that ensure I will never write up the small mistakes I occasionally make, even if people might benefit from knowing about them. If ever you make a mistake, never, ever write it up. It might get submitted here for everyone to tell you how stupid you were.
- mapreduce 3y ago> If ever you make a mistake, never, ever write it up. Sure you can write it up! But post it to maybe, Reddit? The HN audience expects certain standard of quality and insight in the article if the article is on the front page. Here the community has voted that this article does not meet the bar. I can't speak for others but for me this article is too obvious and uninteresting. Instead of complaining about something that didn't happen here (nobody said the OP was stupid), maybe learn from the community feedback and refrain from posting articles that don't meet the community expectations?
- ColinWright 3y ago> The HN audience expects certain standard of quality and insight in the article if the article is on the front page. It was on the front page ... it hit the top spot. Various people in the community clearly did think it met the bar. > ... nobody said the OP was stupid ... Reading between the lines, several people effectively said exactly that. I know, I'm in "Old man yells at cloud" territory here, but I was kinda hoping people would find a more positive way of interpreting the situation, rather than "Can't the author just RTFM?!?" types of responses. My agèd and fading memory of what HN was like when I first joined suggests that people used to be more interested in being constructive and learning from other people's mistakes, even the mistakes that, when they didn't make them, seemed obvious in retrospect. Perhaps especially those ones. Since this didn't meet your bar, since it's all too obvious for you, I guess there's very little for you to learn. I'm sorry you (and others) felt you had to dump on it, instead of leaving it for people who can still learn from things as "obvious" as this. After all, the votes that got it to the front page show that not everyone knows as much as you do.
- 3y ago
- mapreduce 3y agoDon't people RTFM anymore? Is this another "changeme is valid base64" moment? Come on! Base64 has been around for 30 years!
- ColinWright 3y agoIf you've used a command for the past 10[0] years, and it's always worked, and you go to use it again, and you're using it in a similar way to how you always have, do you read the man page? No, of course you don't. You expect things to behave the way they always have. Then when something goes wrong you don't immediately suspect the command you've successfully been using for the past 10 years and read the man page for all the commands you've been using for the past 10[1] years. No, you suspect the new code. [0] For some value of 10[1]. [1] In my case significantly longer than 10.
- thrill 3y ago"If you've used a command for the past 10[0] years, and it's always worked, and you go to use it again, and you're using it in a similar way to how you always have, do you read the man page?" Yes, that's the first thing I do.
- ColinWright 3y agoI'm trying to find where the cut-off point might be. Do you read the man page every time you use "ls"? Do you read the man page every time you invoke the compiler for your favourite language? If you use someone else's machine, do you read the man page for every command you type? The distro could be different, so you can't be sure it will be the same. I suspect the answer here in each case is "no", so I'm curious as to where you draw the line. If you used a command yesterday, and the day before and the day before that, do you read the man page today? Again, I suspect the answer is "no", but again, I'm curious as to where that line really is.
- NuSkooler 3y agoBeware of !rtfm