Beware when base64-encoding on the CLI using echo

Recently I needed to base64-encode a string. To do this I turned to the CLI and piped the output of echo to base64.

# Don't use! Correct command further down this post.
$ echo 'test' | base64
dGVzdAo=

Although the output looks quite good, things didn’t quite work out when using the string: it failed when used in an implementation.

Tracking down the issue I worked my way back and decided to double check the base64-encoded string (dGVzdAo=) for its correctness. Using PHP I re-encoded the input and voila:

$ php -r "echo base64_encode('test');"
dGVzdA==

That’s different output … and it’s also to correct result. But how come? Is base64 on the CLI wrong?

~

Don’t worry, base64 on the CLI works fine. The problem we’re having here is further upstream, namely with the use of echo. By default echo will append a newline to its output, and that’s why base64 is returning an unexpected result.

Thankfully echo has a -n switch to suppress the trailing newline. By using that, things work as expected:

# Add -n to echo to prevent it from adding a newline
$ echo -n 'test' | base64
dGVzdA==

Let this post act as a note to my future self … πŸ˜…

~

Did this help you out? Like what you see?
Thank me with a coffee.

I don't do this for profit but a small one-time donation would surely put a smile on my face. Thanks!

β˜•οΈ Buy me a Coffee (€3)

To stay in the loop you can follow @bramus or follow @bramusblog on Twitter.

AV1, the video codec of the future

Next to praising the AV1 Codec and providing conversion examples (using ffmpeg), Andrey Sitnik also gives a good overview on containers and codecs – concepts every web developer who embed video should know imho – in his post on the subject:

File extensions for video (.mp4, .wmv, .webm or .mov) barely represent containers. When you see a .mp4 extension, the only thing you can be sure about is that the MP4 container had been used to package a file. The choice of codec depends entirely on a creator: it can be H.264/AAC, or AV1/Opus, or something else.

Better web video with AV1 codec →

Text Escaping and Unescaping in JavaScript with strutil

Today the package strutil came in handy, after receiving (*) this UTF-8 Quoted Printable text in my hands:

=?UTF-8?Q?Test_=F0=9F=91=BB?=

Thanks to strutil#unescapeFromMime() I was able to regain the original string:

Test πŸ‘»

Upon further investigation of strutil it turns out that this package is really powerful as it can do a truckload of things when it comes to encoding:

Text Escaping and Unescaping in JavaScript (UTF-8, UTF-16, UTF-32, dec, hex, punycode, mime, base64, and more)

strutil
jsescape demo →

(*) The encoding in the first place itself happened automatically by Amazon, whilst storing User-Defined Metadata on an S3 Object:

User-defined metadata is a set of key-value pairs. Amazon S3 stores user-defined metadata keys in lowercase. Each key-value pair must conform to US-ASCII when using REST and UTF-8 when using SOAP or browser-based uploads via POST.

Now you know πŸ˜‰