This more accurately reflects that "non-zero exit status indicates
failure"; while > 0 will no doubt work as well, != 0 avoids the question
of whether $? is signed or unsigned in bash and more accurately
represents the documentation ("non-zero", != 0).
This patch implements two structural changes.
First, OS-level detection routines are broken out into a case statement.
Darwin doesn't need to test for readlink/timeout nor Busybox, so this
noticeably improves performance over multiple runs on Darwin.
Linux suffers no additional penalty, since we already ran if $(uname)
every time anyways, and continues to use the more complex
timeout/gtimeout/busybox logic at the (preexisting, unaffected) cost to
performance over multiple runs.
Second, if NOAUTODETECT is set, then the script assumes (and verifies)
that you're providing TIMEOUTBIN and OPENSSLBIN values. If both of those
values are executable files, then the script will proceed, else it will
abort. In this scenario, readlink is unnecessary and is thus unused.
The combination of these two changes will improve performance over
multiple runs both on Darwin and when NOAUTODETECT is set for top1m.
This takes advantage of the new --cafile logic to avoid running CACERTS
autodetection when a file is provided on the command line.
It then ensures the readability of that file, whether provided or
autodetected.
This also adds an undocumented CAPATH environment variable alternative
to --capath, to go along with the existing undocumented CACERTS
environment variable alternative to --cafile, to provide legacy support
for preexisting users.
Prior to this commit, the code accepts both the --cafile and the
--capath options, as that's how it's always behaved. This patch corrects
that, refusing to proceed if the options are provided.
Technically, openssl permits the use of both the -CAfile and -CApath
options. However, cipherscan itself can only make use of one of the two
options, and does not currently support "one or both" scenarios.
So this patch ensures that users are not caught unaware when they
specify --capath and --cafile and the script refuses to honor the
latter.
The HOST[:PORT] extraction routine was written using several calls to
sed and a bunch of regex post-processing of the bash $@ array.
This replaces that with bash-native array commands, copying $@ into
a $PARAMS array, removing the last element into $TARGET, and then
passing the remainder to openssl s_client.
This adds validation of the TARGET to ensure that it matches what we
expect for a HOST[:PORT]; if a ':' is present, it must be preceded by a
hostname and followed by a port number, otherwise :443 is appended.
The check to ensure that HOST is not an -option is merged into this as
well, since we already test for : at the beginning of the HOST
(indicating that only a port was provided).
Additionally, this now defends against an empty string "" being passed
as the final option, which could occur if a script calling cipherscan
goes awry and starts passing empty values as the target.
top1m may see a slight speed improvement from this commit, as 4 calls to
sed are replaced with native bash functions.
Fixes one "SC2086: Double quote to prevent globbing and word splitting.":
In cipherscan line 1402:
SCLIENTARGS=$(sed -e s,${TEMPTARGET},,<<<"${@}")
^-- SC2086: Double quote to prevent globbing and
word splitting.
Prior to this patch, if the user fails to provide a host:port after
specifying cipherscan options, the script runs sed on an empty variable
(failing with a syntax error) and then asttempts to cipherscan the
target ':443'.
This adds a simple test to ensure that a target was actually provided.
This detects and prevents a specific category of user error, where an
incomplete cipherscan command line ending in an OpenSSL -option results
in cipherscan attempting to scan the target '-option:443'.
For unknown reasons, while we previously supported --capath we did not
support --cafile. This forces the --cafile autodetection logic to run
every time, unnecessarily, when we have a specific file in mind to use.
This patch relocates the -CAfile autodetection logic to run *only if*
the --cafile parameter is not provided. If it is not provided, the
autodetection logic occurs precisely as before.
This patch declines to address what happens if both --capath and
--cafile are passed. The previous logic already ensured that the CA file
was *always* set, and then only sometimes was the CA path set. The new
logic maintains that behavior precisely, reserving logic flow changes
for a separate commit.
This catches instances where the wrong openssl binary is selected (for
instance, if uname -s is neither Darwin nor Linux) and serves as a
simple up-front test to make sure that openssl is working before we
proceed further into the script.
The crude_grep function served only to perform a simple substring check
against the output of openssl -help. So, instead of running the command
each time, iterating its output line by line, and checking for the
substring within it, this simply caches the -help output at startup and
uses $help =~ substring to produce the same result in a single pass.
In cipherscan line 451:
for ((i=0; i<$certificate_count; i=i+1 )); do
^-- SC2004: $/${} is unnecessary on arithmetic variables.
In cipherscan line 603:
cipherbenchms="$((t/1000/$BENCHMARKITER))"
^-- SC2004: $/${} is unnecessary on arithmetic variables.
In cipherscan line 13:
REALPATH=$(dirname $0)
^-- SC2086: Double quote to prevent globbing and word splitting.
In cipherscan line 15:
readlink -f $0 &>/dev/null && REALPATH=$(dirname $(readlink -f $0))
^-- SC2086: Double quote to prevent globbing and word splitting.
^-- SC2046: Quote this to prevent word splitting.
^-- SC2086: Double quote to prevent globbing and word splitting.
In cipherscan line 46:
if [[ -e $(dirname $0)/openssl.cnf ]]; then
^-- SC2086: Double quote to prevent globbing and word splitting.
In cipherscan line 47:
export OPENSSL_CONF="$(dirname $0)/openssl.cnf"
^-- SC2155: Declare and assign separately to avoid masking return values.
^-- SC2086: Double quote to prevent globbing and word splitting.
In cipherscan line 60:
CACERTS="$(dirname $0)/ca-bundle.crt"
^-- SC2086: Double quote to prevent globbing and word splitting.
In cipherscan line 941:
verbose "Server supported curves: ${tmp_curves[@]}"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
In cipherscan line 968:
verbose "ephem_data: ${ephem_data[@]}"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
Bash implements a backwards-compatible sh syntax for [ .. ], which
handles undef variables poorly. Use [[ .. ]] instead, to take full
advantage of the Bash improvements to the comparison brackets.
buggy servers may choke on large ClientHello's, TLSv1.2 ClientHello's,
etc. try to detect such failures and report them
among tried connections are TLS1.2, TLS1.1, TLS1.0 and SSLv3 with
ability to downgrade to lower protocol versions as well as a size
limited client hello, both TLS1.2 and TLS1.0 version
It's unlikely that there are SSLv2 only servers on the 'net, all
that were detected as such and I've checked actually are intolerant
to low placement of RC4 in cipher order or intolerant to large client
hello in general. In case we detect issues with the server, switch to
reduced cipher set and run the test again that should give better results
for about 3% of hosts
because to advertise curves to server we need extensions and
extensions are only available in TLSv1.0 or later, we need to force
OpenSSL not to send SSLv2 compatible hello if it thinks it's ok to
do (when there are SSLv2 ciphers present in cipherstring it will try to)
since early versions of 1.0.2 openssl supports -curves command line
option, it allows us to set the curves advertised as supported
use the same approach to testing: advertise all, check what server
accepts, remove the accepted from list, repeat. When server aborts
connection or selects non ECC cipher, we know that we've tested all.
bash has a built in regular expression processor, we can match
lines using =~
moreover, stuff that will match while being inside parentheses is
later available in the BASH_REMATCH array
the IFS (Internal Field Separator) by default includes space, tab and
new line, as such we can use it to split longer lines to separate
words, just as awk '{print $1}' can, just need to put the value to
an array for that
we also don't have to use $(echo $var) when assigning variables, $var
is enough
bash has also built in substitution engine, so we can do ${var/,/ & }
to switch all commas to ampersands when using the variable
openssl sometimes will print the filename, then the error, and finish
with OK, matching the colon and space prevents from considering such
certs to be valid
the certificate extracted in the above way will contain some junk
from openssl s_client output we don't want like verification status
we can remove it ro reduce disk usage for saved certificates
awk has an inbuilt version of grep, also truncate processing as soon
as we find what we're looking for
This version uses slightly different syntax that is compatible with old
awk
firstly, test_cipher_on_target() will try at least 4 connections before
incurring the sleep, for aggressive rate limiter on server side it may be
too much, so sleep before every connection
secondly, because running external commands like sleep incurs a fork
penalty, we first check if it is necessary
when openssl is run in -ssl2 mode, it doesn't accept -servername
option and just aborts operation, it doesn't consider -status
to be special though.
Remove this option when running the SSLv2 portion of the test.
