"""Verified SQLite online backups and bounded tiered retention.""" from __future__ import annotations import hashlib import hmac import os import shutil import sqlite3 import uuid from contextlib import closing from dataclasses import dataclass from datetime import datetime, timezone from pathlib import Path from archive_clients.config import BackupConfig from archive_clients.locking import DatabaseLease from archive_clients.state import SCHEMA_VERSION class BackupError(RuntimeError): pass @dataclass(frozen=True) class BackupRecord: database: Path checksum: Path sha256: str class SQLiteBackupManager: def __init__( self, database: Path, backup_dir: Path, retention: BackupConfig, busy_timeout_ms: int = 5000, ): self.database = database self.backup_dir = backup_dir self.retention = retention self.busy_timeout_ms = busy_timeout_ms def create(self, label: str = "scheduled") -> BackupRecord: if not self.database.is_file(): raise BackupError("source database does not exist") if not label or not all( character.isalnum() or character in "-_" for character in label ): raise BackupError("backup label contains unsafe characters") self.backup_dir.mkdir(parents=True, exist_ok=True) timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S.%fZ") final = self.backup_dir / f"archive-client-{timestamp}-{label}.db" checksum = final.with_suffix(final.suffix + ".sha256") temporary = self.backup_dir / f".{final.name}.{uuid.uuid4().hex}.tmp" checksum_temporary = self.backup_dir / ( f".{checksum.name}.{uuid.uuid4().hex}.tmp" ) if final.exists() or checksum.exists(): raise BackupError("backup destination already exists") try: with closing(sqlite3.connect(self.database)) as source, closing( sqlite3.connect(temporary) ) as target: source.execute(f"PRAGMA busy_timeout = {self.busy_timeout_ms}") source.backup(target) os.chmod(temporary, 0o600) self._verify_sqlite(temporary) _fsync_file(temporary) digest = _sha256(temporary) os.replace(temporary, final) with checksum_temporary.open("x", encoding="ascii") as sidecar: sidecar.write(f"{digest} {final.name}\n") sidecar.flush() os.fsync(sidecar.fileno()) os.chmod(checksum_temporary, 0o600) os.replace(checksum_temporary, checksum) _fsync_directory(self.backup_dir) record = BackupRecord(final, checksum, digest) self.verify(final) self.prune() return record except Exception: temporary.unlink(missing_ok=True) checksum_temporary.unlink(missing_ok=True) final.unlink(missing_ok=True) checksum.unlink(missing_ok=True) raise def verify(self, backup: Path) -> BackupRecord: backup = Path(backup) checksum = backup.with_suffix(backup.suffix + ".sha256") if not backup.is_file() or not checksum.is_file(): raise BackupError("backup database or checksum sidecar is missing") parts = checksum.read_text(encoding="ascii").strip().split() if len(parts) != 2 or parts[1] != backup.name: raise BackupError("backup checksum sidecar is malformed") actual = _sha256(backup) if not hmac.compare_digest(parts[0], actual): raise BackupError("backup checksum does not match") self._verify_sqlite(backup) return BackupRecord(backup, checksum, actual) def list(self) -> list[BackupRecord]: if not self.backup_dir.exists(): return [] return [ self.verify(path) for path in sorted( self.backup_dir.glob("archive-client-*.db"), reverse=True ) ] def restore(self, backup: Path) -> Path | None: verified = self.verify(Path(backup)) self.database.parent.mkdir(parents=True, exist_ok=True) timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S.%fZ") temporary = self.database.parent / ( f".{self.database.name}.restore-{uuid.uuid4().hex}.tmp" ) preserved = self.database.with_name( f"{self.database.name}.suspect-{timestamp}" ) moved: list[tuple[Path, Path]] = [] installed = False with DatabaseLease(self.database): try: shutil.copyfile(verified.database, temporary) os.chmod(temporary, 0o600) _fsync_file(temporary) self._verify_sqlite(temporary) for source, destination in ( (self.database, preserved), (Path(f"{self.database}-wal"), Path(f"{preserved}-wal")), (Path(f"{self.database}-shm"), Path(f"{preserved}-shm")), ): if source.exists(): os.replace(source, destination) moved.append((source, destination)) os.replace(temporary, self.database) installed = True _fsync_directory(self.database.parent) self._verify_sqlite(self.database) return preserved if moved else None except Exception: temporary.unlink(missing_ok=True) if installed: self.database.unlink(missing_ok=True) for source, destination in reversed(moved): os.replace(destination, source) _fsync_directory(self.database.parent) raise def prune(self) -> list[Path]: candidates = sorted( self.backup_dir.glob("archive-client-*.db"), key=lambda path: (path.stat().st_mtime_ns, path.name), reverse=True, ) keep = set(candidates[: self.retention.recent]) keep.update(_newest_per_period(candidates, self.retention.daily, "day")) keep.update(_newest_per_period(candidates, self.retention.weekly, "week")) removed = [] for backup in candidates: if backup in keep: continue backup.unlink(missing_ok=True) backup.with_suffix(backup.suffix + ".sha256").unlink(missing_ok=True) removed.append(backup) if removed: _fsync_directory(self.backup_dir) return removed @staticmethod def _verify_sqlite(database: Path) -> None: uri = f"{database.resolve().as_uri()}?mode=ro" try: with closing(sqlite3.connect(uri, uri=True)) as connection: if connection.execute("PRAGMA integrity_check").fetchall() != [ ("ok",) ]: raise BackupError("SQLite integrity check failed") if connection.execute("PRAGMA foreign_key_check").fetchall(): raise BackupError("SQLite foreign-key check failed") version = connection.execute("PRAGMA user_version").fetchone()[0] if version != SCHEMA_VERSION: raise BackupError(f"unsupported backup schema version {version}") except sqlite3.Error as exc: raise BackupError("backup is not a readable SQLite database") from exc def _sha256(path: Path) -> str: digest = hashlib.sha256() with path.open("rb") as source: for chunk in iter(lambda: source.read(1024 * 1024), b""): digest.update(chunk) return digest.hexdigest() def _fsync_file(path: Path) -> None: with path.open("rb") as source: os.fsync(source.fileno()) def _fsync_directory(path: Path) -> None: descriptor = os.open(path, os.O_RDONLY) try: os.fsync(descriptor) finally: os.close(descriptor) def _newest_per_period( candidates: list[Path], limit: int, period: str ) -> list[Path]: selected = [] seen: set[object] = set() for candidate in candidates: instant = datetime.fromtimestamp(candidate.stat().st_mtime, tz=timezone.utc) key: object = instant.date() if period == "day" else instant.isocalendar()[:2] if key in seen: continue seen.add(key) selected.append(candidate) if len(selected) == limit: break return selected