Grandfather, Father, Son Rotation Script

Here’s a simple script I put together to rotate backup files in a given directory with decreasing resolution the further away we get from the current time.

#!/bin/bash set -euo pipefail DRYRUN=${DRYRUN:-1} DIR=${DIR:-'/backups/mysql'} HOURS=${HOURS:-48} DAYS=${DAYS:-35} MONTHS=${MONTHS:-24} usage() { echo "usage: DRYRUN=0 DIR=/backups HOURS=48 DAYS=30 MONTHS=24 $0" exit 1 } if [ $DRYRUN -eq 1 ]; then echo "DRYRUN: No changes will be made" fi if [ "$HOURS" -lt 0 ] || [ "$DAYS" -lt 1 ] || [ "$MONTHS" -lt 1 ]; then echo "ERROR: Cannot have negative rotation" usage fi ## Bail and cleanup if we exit early cleanup() { find $DIR -type f -execdir rename 's/.process$//g' '{}' \; } trap cleanup EXIT trap cleanup SIGINT ## Flag all the files for processing find $DIR -type f | grep -v '.process$' | xargs -I{} mv {}{,.process} while [ $MONTHS -gt 0 ]; do MONTHS=$(($MONTHS-1)) days=$(( ( $(date '+%s') - $(date -d "$MONTHS months ago" '+%s') ) / 86400 )) file=$(find $DIR -type f -mtime "+$days" | sort -n | tail -n1) if [ "$DRYRUN" -eq 0 ]; then rename 's/.process$//g' "$file" else echo "MONTH: $MONTHS $file" | sed 's/.process$//' fi done while [ $DAYS -gt 0 ]; do DAYS=$(($DAYS-1)) file=$(find $DIR -type f -mtime "+$DAYS" | sort -n | tail -n1) if [ "$DRYRUN" -eq 0 ]; then rename 's/.process$//g' "$file" else echo "DAY: $DAYS $file" | sed 's/.process$//' fi done while [ $HOURS -gt 0 ]; do HOURS=$(($HOURS-1)) minutes=$(($HOURS*60)) file=$(find $DIR -type f -mmin "+$minutes" | sort -n | tail -n1) if [ "$DRYRUN" -eq 0 ]; then rename 's/.process$//g' "$file" else echo "HOUR: $HOURS $file" | sed 's/.process$//' fi done ## Delete all remaining files [ "$DRYRUN" -eq 0 ] && rm "$DIR"/*.process 2> /dev/null

Grafana Overview Dashboard

There should be an “Overview” dashboard for your infrastructure that gives you a quick, high level view of how everything is working. The idea isn’t to necessarily use this for diagnosing problems, but for knowing where to look next. Here’s a quick one I put together for fun.