Visualising curl downloads
Latest update:
If downloading from a server is slow, how would you prove to a devops
guy that you're experiencing a slowdown? There are a couple of
possibilities, the worst of which would be sending a video. What if
you send a speed graph using data from curl?
Every second curl prints to stderr the following:
fprintf(tool_stderr,
"\r"
"%-3s " /* percent downloaded */
"%-3s " /* percent uploaded */
"%s " /* Dled */
"%s " /* Uled */
"%5" CURL_FORMAT_CURL_OFF_T " " /* Xfers */
"%5" CURL_FORMAT_CURL_OFF_T " " /* Live */
" %s " /* Total time */
"%s " /* Current time */
"%s " /* Time left */
"%s " /* Speed */
"%5s" /* final newline */,
…
Therefore, by replacing \r with \n we can send a download log:
$ curl http://example.com/1.zip -o 1.zip 2>&1 | tr \\r \\n
To draw a graph with gnuplot, we can use Current time and Speed
columns. Gnuplot understands time as input data, but I don't know how
to persuade it to interpret values like 100k or 200M, thus we need to
convert them into 'bytes'. This is a cute little problem for code
golf, but amusingly, it was already solved in coreutils > 12 years ago
via numfmt(1).
$ echo 1M and 10M | numfmt --from iec --field 3
1M and 10485760
(macOS & FreeBSD both have coreutils package, where the utility
executable is prefixed with 'g'.)
#!/usr/bin/env -S stdbuf -o0 bash
set -e -o pipefail
numfmt=`type -p gnumfmt numfmt;:`; test "${numfmt:?}"
cat <<E
set xdata time
set timefmt "%H:%M:%S"
set xlabel "Time, MM:SS or HH:MM:SS"
set format y "%.0s%cB"
set ylabel "Speed, Unit/second" offset -1,0
set grid
plot "-" using 1:2 with lines title ""
E
curl "$@" -fL -o /dev/null 2>&1 | tr \\r \\n | awk '
/[0-9.][kMGTP]?$/ {
time = index($10, ":") == 0 ? $11 : $10
if (time != "--:--:--") print time, $NF
}' | tr k K | $numfmt --from iec --field 2
Usage:
$ ./curlbench http://example.com/1.zip | gnuplot -p
Tags: ойті
Authors: ag