PR#773: keep weekly weblate backups on backup01 - fedora-infra/ansible

fedora-infra / ansible

#773 keep weekly weblate backups on backup01

Merged 2 years ago by kevin. Opened 2 years ago by darknao.

fedora-infra/ darknao/ansible weblate_backup into main

keep weekly weblate backups on backup01

Francois Andrieu • 2 years ago

e0941ac

inventory/host_vars/backup01.iad2.fedoraproject.org

file modified

		`@@ -7,6 +7,7 @@`
		`ansible_ifcfg_blocklist: true`

		`grokmirror_topdir: /fedora_backups/grokmirror`
		`+ weblate_backup_topdir: /fedora_backups/misc/weblate`

		`dns1: 10.3.163.33`
		`dns2: 10.3.163.34`

playbooks/groups/backup-server.yml

file modified

		`@@ -27,6 +27,7 @@`
		`nfs_mount_opts: "rw,hard,bg,intr,noatime,nodev,nosuid,sec=sys,nfsvers=3",`
		`nfs_src_dir: 'fedora_backups' }`
		`- grokmirror_mirror`
		`+ - weblate_backup`

		`pre_tasks:`
		`- import_tasks: "{{ tasks_path }}/yumrepos.yml"`

roles/weblate-backup/tasks/main.yml

file added

+62

		`@@ -0,0 +1,62 @@`
		`+ - name: Install packages`
		`+ package:`
		`+ name:`
		`+ - borgbackup`
		`+ state: present`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Create a non root user to run the code`
		`+ user:`
		`+ name: _backup_weblate`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Copy private key`
		`+ copy:`
		`+ src: "{{ private }}/files/weblate/id_rsa"`
		`+ dest: ~_backup_weblate/.ssh/id_rsa`
		`+ owner: _backup_weblate`
		`+ group: _backup_weblate`
		`+ mode: 0600`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Copy passphrase file`
		`+ copy:`
		`+ src: "{{ private }}/files/weblate/borg_passphrase"`
		`+ dest: ~_backup_weblate/.borgpass`
		`+ owner: _backup_weblate`
		`+ group: _backup_weblate`
		`+ mode: 0600`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Install backup script`
		`+ template:`
		`+ src: weblate-backup`
		`+ dest: /usr/local/bin/weblate-backup`
		`+ owner: root`
		`+ group: root`
		`+ mode: 0755`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Setup cron job`
		`+ template:`
		`+ src: cron-weblate-backup`
		`+ dest: /etc/cron.d/cron-weblate-backup`
		`+ owner: root`
		`+ group: root`
		`+ mode: 0644`
		`+ tags:`
		`+ - weblate_backup`
		`+`
		`+ - name: Create directory to mirror backups to`
		`+ file:`
		`+ dest: "{{ weblate_backup_topdir }}"`
		`+ mode: 0755`
		`+ state: directory`
		`+ owner: _backup_weblate`
		`+ tags:`
		`+ - weblate_backup`

roles/weblate-backup/templates/cron-weblate-backup

file added

		`@@ -0,0 +1,1 @@`
		`+ 0 2 * * 0 _weblate_backup /usr/local/bin/weblate-backup`

roles/weblate-backup/templates/weblate-backup

file added

+35

		`@@ -0,0 +1,35 @@`
		`+ #!/bin/bash`
		`+`
		`+ pass_file=~/.borgpass`
		`+ backup_prefix=weblate_backup`
		`+ dest_backup={{ weblate_backup_topdir }}`
		`+`
		`+ dest_backup_file=${dest_backup}/${backup_prefix}_$(date +%Y%m%d%H%M%S).tar.gz`
		`+ export BORG_RSH="ssh -o StrictHostKeyChecking=no"`
		`+ export BORG_REPO="ssh://u164666-sub8@u164666-sub8.your-storagebox.de:23/./backups"`
		`+`
		`+ #############################`
		`+`
		`+ if [ ! -f "$pass_file" ]; then`
		`+ echo "Passphrase file not found!"`
		`+ exit 1`
		`+ fi`
		`+ export BORG_PASSCOMMAND="cat $pass_file"`
		`+`
		`+ echo "* listing backup ..."`
		`+ last_backup=$(borg list --last 1 \| awk '{print $1}')`
		`+ if [ -z "$last_backup" ]; then`
		`+ echo "* no backup found"`
		`+ exit 1`
		`+ fi`
		`+ echo "* last backup is $last_backup"`
		`+`
		`+`
		`+ echo "* extracting backup to $dest_backup ..."`
		`+ borg export-tar ::$last_backup $dest_backup_file`
		`+ echo "* extraction complete"`
		`+`
		`+ echo "* purging old backups ..."`
		`+ # 56 days = 8 weeks`
		`+ find ${dest_backup} -name ${backup_prefix}\*.gz -ctime +56 -exec rm -v {} \;`
		`+ echo "* purge complete"`

darknao commented 2 years ago

A couple of files is needed for this:
/srv/private/ansible/files/weblate/id_rsa : ssh private key to access weblate backup host
/srv/private/ansible/files/weblate/borg_passphrase : passphrase used to decrypt the borgbackup repository

Backups will go in /fedora_backups/misc/weblate on backup01, fetched weekly, with 8 weeks retention.

zuul commented 2 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 2m 14s

rebased onto 81dc0135a538c9e00a3465cc88954278fb552631

2 years ago

zuul commented 2 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 3m 20s

kevin commented 2 years ago

Looks pretty good to me, but lets wait until after freeze.

kevin commented 2 years ago

This has conflicts now, can you rebase?

rebased onto a60f8aded3a3fb4ce2efa70af96e72c160546bff

2 years ago

zuul commented 2 years ago

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.

rebased onto d08cad7267a8a0ec4f062a2c66e9e9d336ae0b73

2 years ago

zuul commented 2 years ago

Build succeeded.