openssl has no memory

Graybeard Unix Curmudgeon

Using openssl

As part of the work I have been doing with One Time Pads, I have been switching from using the standalone digest commands to using only the openssl implementations since they will have more active maintenance for a longer period of time. To that end, I have been doing a project that was originally written as shell scripts to implementing a One Time Pad application as a C program for simple performance reasons.

In the process of doing so I have been struggling with the complexity of the OpenSSL documentation. It is only after a great deal of reading that I learned that most of the complexity of OpenSSL is unnecessary for most simple use cases. However, in the process I have been stumbling across a realization that the implementors of OpenSSL are not familiar with the “small is beautiful” paradigm of Unix/Linux systems.

Small is Beautiful

I offer the following case in point. When I want to know the list of cryptographic digests that are supported by the installed version of openssl, I can execute a simple command:

$ openssl list -digest-commands        

I found that this was problematic when I was doing the shell scripts that invoked this command line. The implementer of the command chose to take the list of supported digest commands and output them in their own columnized output.

$ openssl list -digest-commands
blake2b512        blake2s256        md5               sha1              
sha224            sha256            sha3-224          sha3-256          
sha3-384          sha3-512          sha384            sha512            
sha512-224        sha512-256        shake128          shake256          
sm3               

        

The output looks OK but is unable to be parsed by a simple bash script.

I had to unwind the output with a simple loop to be able to check for the existence of any one digest (e.g. sha256). Although it is possible to perform:

$ if [[ "$(openssl list -digest-commands | grep sha256)" == "sha256" ]] ; then echo "sha256 found"; else echo "sha256 not found";fi
sha256 not found        

As you can see, I get the wrong result. I had to resort to the following:

$ if [[ "$(for i in $(openssl list -digest-commands); do echo $i;done | grep sha256)" == "sha256" ]] ; then echo "sha256 found" ; else echo "sha256 not found"; fi
sha256 found        

A standard Linux/Unix command would have output the simple list and left it up to the user to compose the output that they wanted. Using the above "for" loop to produce the list of digest commands, the proper composition for an end user would be:

$ for i in $(openssl list -digest-commands); do echo $i;done | column -x
blake2b512	blake2s256	md5		sha1
sha224		sha256		sha3-224	sha3-256
sha3-384	     sha3-512	     sha384	sha512
sha512-224	sha512-256	shake128	shake256
sm3        

Eating Crow (sort of)

It turns out that I missed a command line option for openssl. It turns out that the single column output is available (but is not the default). In the above command if I had specified a "-1" for one column output, I would have gotten the desired result:

$ openssl list -1 -digest-commands | column -x
blake2b512	blake2s256	md5		sha1
sha224		sha256		sha3-224	sha3-256
sha3-384	sha3-512	sha384		sha512
sha512-224	sha512-256	shake128	shake256
sm3
        

However note that there is no extraneous blank line at the end of the multi-column output of the column command and that the column command adjusts its output automatically to the width of the current terminal window. I still contend that the single column output should be the default (in the way of classic Unix/Linux.

Things get worse

I had forgotten the syntax of the openssl list command so I ran (extra output deleted):

$ openssl list -help
 -digest-commands          List of message digest commands (deprecated)
 -digest-algorithms        List of message digest algorithms        

and discovered that horror of horrors that the -digest-commands has been deprecated and is presumably replaced by the more meaningful -digest-algorithms. However when I run that command I get a whole bunch of presumably more accurate, but UGLY and unparseable output for a bash script:

$ openssl list -digest-algorithms | grep -i sha256
  RSA-SHA256 => SHA256
  SHA256
  sha256WithRSAEncryption => SHA256
  { 2.16.840.1.101.3.4.2.1, SHA-256, SHA2-256, SHA256 } @ default        

Then I ran a different list generation:

$ openssl dgst -1 -list
Supported digests:
-blake2b512                -blake2s256                -md4                      
-md5                       -md5-sha1                  -ripemd                   
-ripemd160                 -rmd160                    -sha1                     
-sha224                    -sha256                    -sha3-224                 
-sha3-256                  -sha3-384                  -sha3-512                 
-sha384                    -sha512                    -sha512-224               
-sha512-256                -shake128                  -shake256                 
-sm3                       -ssl3-md5                  -ssl3-sha1                
-whirlpool                         

This list is at least human readable without confusing output but there are two problems with this list:

1. The list is automatically multi-column and the -1 option does not work with this list.
2. The digest names are preceded by a "-" character, presumably to remind the user that if you want a sha512 digest, that you would need to precede the digest name by a "-" in the command line as shown here.

$ echo "Hello World" | openssl dgst -sha512
SHA2-512(stdin)= e1c112ff908febc3b98b1693a6cd3564eaf8e5e6ca629d084d9f0eba99247cacdd72e369ff8941397c2807409ff66be64be908da17ad7b8a49a2a26c0e8086aa        

Yet there is no simple way to unwind this list into a single column and just get the names of the digests. In order to get a simple single column list I had to do the following:

$ for i in $(openssl dgst -list) ; do echo $i; done | grep -v "Supported" | grep -v "digests:"
-blake2b512
-blake2s256
-md4
-md5
-md5-sha1
-ripemd
-ripemd160
-rmd160
-sha1
-sha224
-sha256
-sha3-224
-sha3-256
-sha3-384
-sha3-512
-sha384
-sha512
-sha512-224
-sha512-256
-shake128
-shake256
-sm3
-ssl3-md5
-ssl3-sha1
-whirlpool        

I did try other sequences, but the "-" before the name of each digest made it challenging to use in Bash "if" statements.

My bottom line is that the authors of openssl may indeed be wizards at cryptography and yes, I am still learning, but they seem to be woefully lacking in the fundamentals of Unix/Linux and the basics of producing output that is useful in shell scripting, like Bash. I can only speculate that they have done all of their scripting in a language like python where none of the issues I note would arise. But in the case of the openssl command outputting messages like "Supported digests:" is something that in a classic Unix/Linux command would only be produced if you had specified a "-v" (verbose) argument and if it is a default output, could be suppressed with a "-q" (quiet) argument.

Is it any wonder why graybeards have so little hair?