Automatic indi-allsky backup on external drive (Fritzbox NAS)

Posted on : Posted in : Hardware (EN)|Software (EN)

In this article, I describe how I back up my indi-allsky installation on my Raspberry Pi fully automatically to an external hard disk.

The official indi-allsky documentation describes very well what should be backed up – but leaves open how to turn it into a reliable, automated backup.

In my setup, indi-allsky runs permanently on a Raspberry Pi. The backup is not done locally, but on an external SSD on a Fritzbox, connected via SMB.

The aim was to consistently implement the recommendations from the official documentation: Backup and Recovery – indi-allsky and to expand it to production readiness:

  • external target
  • automatic backups
  • integrity checks
  • time-based retention
  • Mail notification in case of errors

…and this is how I proceeded!

What needs to be backed up according to the official indi-allsky documentary

Based on the official documentation are relevant:

  • SQLite database – /var/lib/indi-allsky/indi-allsky.sqlite
  • Flask configuration – /etc/indi-allsky/flask.json
  • Service environment (optional) – /etc/indi-allsky/indi-allsky.env
  • Database Migrations – /var/lib/indi-allsky/migrations/versions/

Videos, photos & thumbnails are not part of this guide for now. Phase 2 will follow soon!

Prepare external drive

The SSD is formatted with ext4 under Linux:

sudo parted /dev/sda mklabel gpt mkpart BACKUP ext4 0% 100% quit sudo mkfs.ext4 -L backup_allsky /dev/sda1

The SSD is then connected to the Fritzbox and made available via FRITZ!NAS (SMB). Important: A user with access to the NAS is required!

SMB mount on the Raspberry Pi

Create a mount point:

sudo mkdir -p /mnt/backup_allsky

Store user data (root-only) of the user who has access to the NAS:

sudo nano /root/.smbcredentials
username=YOUR_USER
password=YOUR_PASSWORD

Store /etc/fstab entry – Attention: One line:

//192.168.178.1/FRITZ.NAS /mnt/backup_allsky cifs credentials=/root/.smbcredentials,vers=3.1.1,iocharset=utf8,noforceuid,noforcegid,nounix,soft,_netdev,noserverino 0 0

noserverino is deliberately set against “stale file handle” on Fritzbox!
Test mount:

sudo mount /mnt/backup_allsky mountpoint /mnt/backup_allsky

Output should be “is mountpoint”!

Backup script for indi-allsky (complete)

Create file:

sudo nano /usr/local/bin/backup_allsky.sh

Complete, productive script – this one

  • performs a backup,
  • checks this for consistency
  • and deletes all backups older than 6 weeks.

If there are errors, a mail is sent out.

.sh
#!/bin/bash
set -euo pipefail

# ======================================================
# Configuration
# ======================================================
BACKUP_ROOT="/mnt/backup_allsky/backup_allsky"

DB_SRC="/var/lib/indi-allsky/indi-allsky.sqlite"
CONFIG_DIR="/etc/indi-allsky"
MIGRATIONS_DIR="/var/lib/indi-allsky/migrations/versions"

LOG="/var/log/backup_allsky.log"
MAIL_TO="mail@stefanwatzinger.de"
HOST="$(hostname)"

RETENTION_DAYS=42   # 6 weeks

# ======================================================
# Logging
# ======================================================
exec >>"$LOG" 2>&1

# ======================================================
# Error handling (MAIL ONLY ON FAILURE)
# ======================================================
error_handler() {
  RC=$?
  {
    echo "Allsky Phase-1 Backup FEHLGESCHLAGEN"
    echo "Host: $HOST"
    echo "Zeit: $(date)"
    echo "Exit-Code: $RC"
    echo
    echo "Letzte Logzeilen:"
    tail -n 50 "$LOG"
  } | mail -s "Allsky Backup FEHLER ($HOST)" "$MAIL_TO"
  exit $RC
}
trap error_handler ERR

# ======================================================
# Start
# ======================================================
echo "=== Backup Start: $(date) ==="

# ======================================================
# Defensive Re-Mount (prevents stale handles)
# ======================================================
umount /mnt/backup_allsky || true
mount /mnt/backup_allsky
mountpoint -q /mnt/backup_allsky

# ======================================================
# Prepare directories
# ======================================================
mkdir -p \
  "$BACKUP_ROOT/db" \
  "$BACKUP_ROOT/config" \
  "$BACKUP_ROOT/migrations"

# ======================================================
# Database backup (SQLite dump + gzip)
# ======================================================
echo "--- DB Backup ---"
DB_BACKUP="$BACKUP_ROOT/db/backup_indi-allsky_sqlite_$(date +%Y%m%d_%H%M%S).sql.gz"

sqlite3 "$DB_SRC" .dump \
| gzip -9 \
> "$DB_BACKUP"

echo "--- DB Integrity Check ---"
gunzip -t "$DB_BACKUP"

# ======================================================
# Config backup
# ======================================================
echo "--- Config Backup ---"
cp -v "$CONFIG_DIR/flask.json" "$BACKUP_ROOT/config/"

if [ -f "$CONFIG_DIR/indi-allsky.env" ]; then
  cp -v "$CONFIG_DIR/indi-allsky.env" "$BACKUP_ROOT/config/"
fi

echo "--- Config Integrity Check ---"
test -s "$BACKUP_ROOT/config/flask.json"

# ======================================================
# Database migrations backup
# ======================================================
echo "--- Migrations Backup ---"
MIG_BACKUP="$BACKUP_ROOT/migrations/backup_migrations_$(date +%Y%m%d_%H%M%S).tgz"

tar -C "$MIGRATIONS_DIR" -czf "$MIG_BACKUP" .

echo "--- Migrations Integrity Check ---"
tar -tzf "$MIG_BACKUP" > /dev/null

# ======================================================
# Retention cleanup (AFTER successful backup)
# ======================================================
echo "--- Retention Cleanup (> ${RETENTION_DAYS} days) ---"

find "$BACKUP_ROOT/db" \
  -type f -name "backup_indi-allsky_sqlite_*.sql.gz" \
  -mtime +"$RETENTION_DAYS" -print -delete

find "$BACKUP_ROOT/migrations" \
  -type f -name "backup_migrations_*.tgz" \
  -mtime +"$RETENTION_DAYS" -print -delete

# ======================================================
# Finish
# ======================================================
echo "=== Backup Ende: $(date) ==="

Set permissions:

sudo chmod 700 /usr/local/bin/backup_allsky.sh

Cronjob (daily, during the day)

sudo crontab -e
0 10 * * * /usr/local/bin/backup_allsky.sh

Check-ups & verification

For security reasons, it makes sense to check the consistency of the data once after the initial backup:

Database:

gunzip -t backup_indi-allsky_sqlite_*.sql.gz

Migrations:

tar -tzf backup_migrations_*.tgz | head

Config:

python3 -m json.tool flask.json > /dev/null

Troubleshooting: Mail dispatch

Technically, sending mail is based on msmtp, which is installed on the Raspberry Pi as a lightweight SMTP client and acts as a replacement for sendmail.

Installation:

sudo apt install msmtp msmtp-mta bsd-mailx

msmtp is configured to send via the existing SMTP account of your own mail provider. The configuration is located system-wide in /etc/msmtprc and contains the host, port, user and password of the SMTP server.

No separate mail logic is implemented in the backup script itself, but only with :

set -euo pipefail
trap error_handler ERR

is used.

This means: Any error (mount problems, defective archives, failed integrity checks)

  • leads to an immediate termination of the script
  • triggers the error_handler function
  • and sends a specific error mail with a log extract

The content of the mail contains:

  • Hostname
  • time
  • Exit code
  • the last log lines of the backup run

Successful backups do not generate mails!

Enjoyed this post?

You can support allsky-rodgau.de with a small coffee on BuyMeACoffee.

Buy me a coffee!