Replace custom/matrix-postgres-backup role with galaxy/com.devture.ansible.role.postgres_backup

This role is usable on its own and it's not tied to Matrix, so
extracting it out into an independent role that we install via
ansible-galaxy makes sense.

This also fixes the confusion from the other day, where
`matrix_postgres_*` had to be renamed to `devture_postgres_*`
(unless it was about `matrix_postgres_backup_*`).
We now can safely say that ALL `matrix_postgres_*` variables need to be
renamed.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2305

CHANGELOG.md

@@ -1,3 +1,12 @@
+# 2022-11-30
+
+## matrix-postgres-backup has been replaced by the com.devture.ansible.role.postgres_backup external role
+
+Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
+
+You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix.
+
+
 # 2022-11-28
 
 ## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role

docs/configuring-playbook-postgres-backup.md

@@ -1,6 +1,6 @@
 # Setting up postgres backup (optional)
 
-The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
+The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you via the [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup) Ansible role.
 
 For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
 
@@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
 Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
 
 ```yaml
-matrix_postgres_backup_enabled: true
+devture_postgres_backup_enabled: true
 ```
 
 Refer to the table below for additional configuration variables and their default values.
@@ -18,12 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
 
 | Name                              | Default value                | Description                                                      |
 | :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
-|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
+|`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
-|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
+|`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
-|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
+|`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
-|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
+|`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
-|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
+|`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
-|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
+|`devture_postgres_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_data_path` |
+|`devture_postgres_data_path` | `"{{ devture_postgres_base_path }}/data"` | Storage path for postgres-backup database backups |
 
 
 ## Installing

group_vars/matrix_servers

@@ -289,7 +289,7 @@ devture_systemd_service_manager_services_list_auto: |
     +
     ([{'name': (devture_postgres_identifier + '.service'), 'priority': 500, 'groups': ['matrix', 'postgres']}] if devture_postgres_enabled else [])
     +
-    ([{'name': 'matrix-postgres-backup.service', 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if matrix_postgres_backup_enabled else [])
+    ([{'name': (devture_postgres_backup_identifier + '.service'), 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if devture_postgres_backup_enabled else [])
     +
     ([{'name': 'matrix-prometheus.service', 'priority': 4000, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-core']}] if matrix_prometheus_enabled else [])
     +
@@ -2241,6 +2241,7 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
 #
 ######################################################################
 
+
 ########################################################################
 #                                                                      #
 # com.devture.ansible.role.postgres                                    #
@@ -2504,6 +2505,42 @@ devture_postgres_managed_databases_auto: |
 #                                                                      #
 ########################################################################
 
+
+########################################################################
+#                                                                      #
+# com.devture.ansible.role.postgres_backup                             #
+#                                                                      #
+########################################################################
+
+devture_postgres_backup_enabled: false
+
+devture_postgres_backup_identifier: matrix-postgres-backup
+
+devture_postgres_backup_architecture: "{{ matrix_architecture }}"
+
+devture_postgres_backup_base_path: "{{ matrix_base_data_path }}/postgres-backup"
+
+devture_postgres_backup_container_network: "{{ matrix_docker_network }}"
+
+devture_postgres_backup_uid: "{{ matrix_user_uid }}"
+devture_postgres_backup_gid: "{{ matrix_user_gid }}"
+
+devture_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
+devture_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
+devture_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
+devture_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
+
+devture_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
+
+devture_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
+
+########################################################################
+#                                                                      #
+# /com.devture.ansible.role.postgres_backup                            #
+#                                                                      #
+########################################################################
+
+
 ######################################################################
 #
 # matrix-sygnal
@@ -2965,26 +3002,6 @@ matrix_registration_database_password: "{{ '%s' | format(matrix_homeserver_gener
 #
 ######################################################################
 
-######################################################################
-#
-# matrix-postgres-backup
-#
-######################################################################
-
-matrix_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
-matrix_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
-matrix_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
-matrix_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
-
-matrix_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
-
-matrix_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
-
-######################################################################
-#
-# /matrix-postgres-backup
-#
-######################################################################
 
 ######################################################################
 #

playbooks/matrix.yml

@@ -97,8 +97,11 @@
     - custom/matrix-nginx-proxy
     - custom/matrix-coturn
     - custom/matrix-aux
-    - custom/matrix-postgres-backup
+
+    - role: galaxy/com.devture.ansible.role.postgres_backup
+
     - custom/matrix-backup-borg
+
     - custom/matrix-user-creator
     - custom/matrix-common-after
 

requirements.yml

@@ -21,6 +21,9 @@
 - src: git+https://github.com/devture/com.devture.ansible.role.postgres.git
   version: 381feb7f671c60e8875298102a7c9b41242979ee
 
+- src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git
+  version: 77b1f9ae1aafa31c9078178c1036bf744c99d08b
+
 - src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git
   version: 6ccb88ac5fc27e1e70afcd48278ade4b564a9096
 

roles/custom/matrix-postgres-backup/defaults/main.yml

@@ -1,45 +0,0 @@
----
-# Project source code URL: https://github.com/prodrigestivill/docker-postgres-backup-local
-
-matrix_postgres_backup_enabled: false
-
-matrix_postgres_backup_connection_hostname: ''
-matrix_postgres_backup_connection_port: 5432
-matrix_postgres_backup_connection_username: "matrix"
-matrix_postgres_backup_connection_password: ""
-
-matrix_postgres_backup_extra_opts: "-Z9 --schema=public --blobs"
-matrix_postgres_backup_schedule: "@daily"
-matrix_postgres_backup_keep_days: 7
-matrix_postgres_backup_keep_weeks: 4
-matrix_postgres_backup_keep_months: 12
-matrix_postgres_backup_healthcheck_port: "8080"
-matrix_postgres_backup_databases: []
-matrix_postgres_backup_path: "{{ matrix_base_data_path }}/postgres-backup"
-
-# Specifies where the Postgres data is.
-# We use this to autodetect the Postgres version during playbook runtime (by parsing the `PG_VERSION` file contained there).
-# You can leave this empty to prevent auto-detection.
-matrix_postgres_backup_postgres_data_path: ""
-
-matrix_postgres_backup_architecture: amd64
-
-# matrix_postgres_backup_docker_image_distro controls whether we use Alpine-based images (`-alpine`) or the normal Debian-based images.
-# Alpine-based Postgres images are smaller and we usually prefer them, but they don't work on ARM32 (tested on a Raspberry Pi 3 running Raspbian 10.7).
-# On ARM32, `-alpine` images fail with the following error:
-# > LOG:  startup process (PID 37) was terminated by signal 11: Segmentation fault
-matrix_postgres_backup_docker_image_distro: "{{ 'alpine' if matrix_postgres_backup_architecture in ['amd64', 'arm64'] else 'debian' }}"
-
-matrix_postgres_backup_docker_image_v9: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:9.6-{{ matrix_postgres_backup_docker_image_distro }}-2aa03d1"
-matrix_postgres_backup_docker_image_v10: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:10-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_v11: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:11-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_v12: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:12-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_v13: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:13-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_v14: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:14-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_v15: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:15-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
-matrix_postgres_backup_docker_image_latest: "{{ matrix_postgres_backup_docker_image_v15 }}"
-
-# This variable is assigned at runtime. Overriding its value has no effect.
-matrix_postgres_backup_docker_image_to_use: '{{ matrix_postgres_backup_docker_image_latest }}'
-
-matrix_postgres_backup_docker_image_force_pull: "{{ matrix_postgres_backup_docker_image_to_use.endswith(':latest') }}"

roles/custom/matrix-postgres-backup/tasks/main.yml

@@ -1,20 +0,0 @@
----
-
-- block:
-    - when: matrix_postgres_backup_enabled | bool
-      ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml"
-
-    - when: matrix_postgres_backup_enabled | bool
-      ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_install.yml"
-  tags:
-    - setup-all
-    - setup-postgres-backup
-    - install-all
-    - install-postgres-backup
-
-- block:
-    - when: not matrix_postgres_backup_enabled | bool
-      ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
-  tags:
-    - setup-all
-    - setup-postgres-backup

roles/custom/matrix-postgres-backup/tasks/setup_install.yml

@@ -1,51 +0,0 @@
----
-
-
-- ansible.builtin.include_role:
-    name: galaxy/com.devture.ansible.role.postgres
-    tasks_from: detect_existing_postgres_version
-  when: 'matrix_postgres_backup_postgres_data_path != ""'
-
-# If we have found an existing version (installed from before), we use its corresponding Docker image.
-# If not, we install using the latest Postgres.
-#
-# Upgrading is supposed to be performed separately and explicitly (see `upgrade_postgres.yml`).
-- ansible.builtin.set_fact:
-    matrix_postgres_backup_docker_image_to_use: "{{ matrix_postgres_backup_docker_image_latest if matrix_postgres_backup_detected_version_corresponding_docker_image | default('') == '' else matrix_postgres_backup_detected_version_corresponding_docker_image }}"
-
-- name: Ensure postgres backup Docker image is pulled
-  community.docker.docker_image:
-    name: "{{ matrix_postgres_backup_docker_image_to_use }}"
-    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
-    force_source: "{{ matrix_postgres_backup_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
-    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_backup_docker_image_force_pull }}"
-  register: result
-  retries: "{{ devture_playbook_help_container_retries_count }}"
-  delay: "{{ devture_playbook_help_container_retries_delay }}"
-  until: result is not failed
-
-- name: Ensure Postgres backup paths exist
-  ansible.builtin.file:
-    path: "{{ item }}"
-    state: directory
-    mode: 0700
-    owner: "{{ matrix_user_username }}"
-    group: "{{ matrix_user_groupname }}"
-  with_items:
-    - "{{ matrix_postgres_backup_path }}"
-
-- name: Ensure Postgres environment variables file created
-  ansible.builtin.template:
-    src: "{{ role_path }}/templates/{{ item }}.j2"
-    dest: "{{ matrix_postgres_backup_path }}/{{ item }}"
-    owner: "{{ matrix_user_username }}"
-    group: "{{ matrix_user_groupname }}"
-    mode: 0640
-  with_items:
-    - "env-postgres-backup"
-
-- name: Ensure matrix-postgres-backup.service installed
-  ansible.builtin.template:
-    src: "{{ role_path }}/templates/systemd/matrix-postgres-backup.service.j2"
-    dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
-    mode: 0644

roles/custom/matrix-postgres-backup/tasks/setup_uninstall.yml

@@ -1,39 +0,0 @@
----
-
-- name: Check existence of matrix-postgres-backup service
-  ansible.builtin.stat:
-    path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
-  register: matrix_postgres_backup_service_stat
-
-
-- when: matrix_postgres_backup_service_stat.stat.exists | bool
-  block:
-    - name: Ensure matrix-postgres-backup is stopped
-      ansible.builtin.service:
-        name: matrix-postgres-backup
-        state: stopped
-        enabled: false
-        daemon_reload: true
-
-    - name: Ensure matrix-postgres-backup.service doesn't exist
-      ansible.builtin.file:
-        path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
-        state: absent
-
-- name: Check existence of matrix-postgres-backup backup path
-  ansible.builtin.stat:
-    path: "{{ matrix_postgres_backup_path }}"
-  register: matrix_postgres_backup_path_stat
-
-# We just want to notify the user. Deleting data is too destructive.
-- name: Inject warning if matrix-postgres backup data remains
-  ansible.builtin.set_fact:
-    devture_playbook_runtime_messages_list: |
-      {{
-        devture_playbook_runtime_messages_list | default([])
-        +
-        [
-          "NOTE: You are not using the local backup service to backup the PostgreSQL database, but some old data remains from before in `{{ matrix_postgres_backup_path }}`. Feel free to delete it."
-        ]
-      }}
-  when: matrix_postgres_backup_path_stat.stat.exists | bool

roles/custom/matrix-postgres-backup/tasks/validate_config.yml

@@ -1,27 +0,0 @@
----
-
-- name: Fail if required postgres-backup settings not defined
-  ansible.builtin.fail:
-    msg: >-
-      You need to define a required configuration setting (`{{ item }}`).
-  when: "vars[item] == ''"
-  with_items:
-    - "matrix_postgres_backup_connection_hostname"
-    - "matrix_postgres_backup_connection_username"
-    - "matrix_postgres_backup_connection_password"
-    - "matrix_postgres_backup_connection_port"
-    - "matrix_postgres_backup_schedule"
-    - "matrix_postgres_backup_keep_days"
-    - "matrix_postgres_backup_keep_weeks"
-    - "matrix_postgres_backup_keep_months"
-    - "matrix_postgres_backup_path"
-    - "matrix_postgres_backup_databases"
-
-- name: (Deprecation) Catch and report renamed settings
-  ansible.builtin.fail:
-    msg: >-
-      Your configuration contains a variable, which now has a different name.
-      Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`).
-  when: "item.old in vars"
-  with_items:
-    - {'old': 'matrix_postgres_backup_docker_image_suffix', 'new': 'matrix_postgres_backup_docker_image_distro'}

roles/custom/matrix-postgres-backup/templates/env-postgres-backup.j2

@@ -1,12 +0,0 @@
-#jinja2: lstrip_blocks: "True"
-POSTGRES_USER={{ matrix_postgres_backup_connection_username }}
-POSTGRES_PASSWORD={{ matrix_postgres_backup_connection_password }}
-POSTGRES_HOST={{ matrix_postgres_backup_connection_hostname }}
-POSTGRES_DB={{ matrix_postgres_backup_databases|join(', ') }}
-POSTGRES_EXTRA_OPTS={{ matrix_postgres_backup_extra_opts }}
-SCHEDULE={{ matrix_postgres_backup_schedule }}
-BACKUP_KEEP_DAYS={{ matrix_postgres_backup_keep_days }}
-BACKUP_KEEP_WEEKS={{ matrix_postgres_backup_keep_weeks }}
-BACKUP_KEEP_MONTHS={{ matrix_postgres_backup_keep_months }}
-HEALTHCHECK_PORT={{ matrix_postgres_backup_healthcheck_port }}
-POSTGRES_PORT={{ matrix_postgres_backup_connection_port }}

roles/custom/matrix-postgres-backup/templates/systemd/matrix-postgres-backup.service.j2

@@ -1,31 +0,0 @@
-#jinja2: lstrip_blocks: "True"
-[Unit]
-Description=Automatic Backup of  Matrix Postgres server
-After=docker.service
-Requires=docker.service
-DefaultDependencies=no
-
-[Service]
-Type=simple
-Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
-ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
-ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
-
-ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-postgres-backup \
-			--log-driver=none \
-			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
-			--cap-drop=ALL \
-			--read-only \
-			--network={{ matrix_docker_network }} \
-			--env-file={{ matrix_postgres_backup_path }}/env-postgres-backup \
-			--mount type=bind,src={{ matrix_postgres_backup_path }},dst=/backups \
-			{{ matrix_postgres_backup_docker_image_to_use }}
-
-ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
-ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
-Restart=always
-RestartSec=30
-SyslogIdentifier=matrix-postgres-backup
-
-[Install]
-WantedBy=multi-user.target

roles/custom/matrix_playbook_migration/tasks/validate_config.yml

@@ -41,8 +41,10 @@
       The matrix-postgres role in the playbook has been replaced with the com.devture.ansible.role.postgres role (https://github.com/devture/com.devture.ansible.role.postgres).
       The new role is pretty much the same, but uses differently named variables.
 
-      Please change your configuration (vars.yml) to rename all matrix-postgres variables (`matrix_postgres_*` -> `devture_postgres_*`).
+      The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup).
-      Note that `matrix_postgres_backup_*` variables (used by the `matrix-postgres-backup` role) need to remain as they are for now. Do not rename those!
+      The new role is pretty much the same, but uses differently named variables.
 
-      The following variables in your configuration need to be renamed: {{ vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
+      Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `devture_postgres_*`).
-  when: "vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"
+
+      The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
+  when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"