GMH v0.4.2 update

CHANGELOG.md

@@ -1,3 +1,13 @@
+# 2021-04-05
+
+## Automated local Postgres backup support
+
+Thanks to [foxcris](https://github.com/foxcris), the playbook can now make automated local Postgres backups on a fixed schedule using [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local).
+
+Additional details are available in [Setting up postgres backup](docs/configuring-playbook-postgres-backup.md).
+
+
+
 # 2021-04-03
 
 ## Mjolnir moderation tool (bot) support

docs/configuring-playbook-bot-mjolnir.md

@@ -46,7 +46,7 @@ You will need to prevent Synapse from rate limiting the bot's account. This is n
 1. Copy the statement below into a text editor. 
 
 	```
-	INSERT INTO ratelimit_override VALUES ("@bot.mjolnir:DOMAIN", 0, 0);
+	INSERT INTO ratelimit_override VALUES ('@bot.mjolnir:DOMAIN', 0, 0);
 	```
 
 1. Change the username (`@bot.mjolnir:DOMAIN`) to the username you used when you registered the bot's account. You must change `DOMAIN` to your server's domain.

docs/configuring-playbook-postgres-backup.md

@@ -0,0 +1,32 @@
+# Setting up postgres backup (optional)
+
+The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
+
+## Adjusting the playbook configuration
+
+Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
+
+```yaml
+matrix_postgres_backup_enabled: true
+```
+
+Refer to the table below for additional configuration variables and their default values.
+
+
+| Name                              | Default value                | Description                                                      |
+| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
+|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
+|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
+|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
+|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
+|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
+|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
+
+
+## Installing
+
+After configuring the playbook, run the [installation](installing.md) command again:
+
+```
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+```

docs/configuring-playbook-prometheus-grafana.md

@@ -17,11 +17,10 @@ matrix_grafana_anonymous_access: false
 
 # This has no relation to your Matrix user id. It can be any username you'd like.
 # Changing the username subsequently won't work.
-matrix_grafana_default_admin_user: some_username_chosen_by_you
+matrix_grafana_default_admin_user: "some_username_chosen_by_you"
 
-# Passwords containing special characters may be troublesome.
 # Changing the password subsequently won't work.
-matrix_grafana_default_admin_password: some_strong_password_chosen_by_you
+matrix_grafana_default_admin_password: "some_strong_password_chosen_by_you"
 ```
 
 By default, a [Grafana](https://grafana.com/) web user-interface will be available at `https://stats.<your-domain>`.

docs/faq.md

@@ -458,3 +458,18 @@ If your server's IP address has changed, you may need to [set up DNS](configurin
 When you [perform a major Postgres upgrade](maintenance-postgres.md#upgrading-postgresql), we save the the old data files in `/matrix/postgres/data-auto-upgrade-backup`, just so you could easily restore them should something have gone wrong.
 
 After verifying that everything still works after the Postgres upgrade, you can safely delete `/matrix/postgres/data-auto-upgrade-backup`
+
+### How do I debug or force SSL certificate renewal?
+
+SSL certificate renewal normally happens automatically via [systemd timers](https://wiki.archlinux.org/index.php/Systemd/Timers).
+
+If you're having trouble with SSL certificate renewal, you can inspect the renewal logs using:
+
+- `journalctl -fu matrix-ssl-lets-encrypt-certificates-renew.service`
+- *or* by looking at the log files in `/matrix/ssl/log/`
+
+To trigger renewal, run: `systemctl start matrix-ssl-lets-encrypt-certificates-renew.service`. You can then take a look at the logs again.
+
+If you're using the integrated webserver (`matrix-nginx-proxy`), you can reload it manually like this: `systemctl reload matrix-nginx-proxy`. Reloading also happens periodically via a systemd timer.
+
+If you're [using your own webserver](configuring-playbook-own-webserver.md) instead of the integrated one (`matrix-nginx-proxy`) you may also need to reload/restart it, to make it pick up the renewed SSL certificate files.

docs/maintenance-postgres.md

@@ -49,7 +49,9 @@ ansible-playbook -i inventory/hosts setup.yml --tags=run-postgres-vacuum,start
 
 ## Backing up PostgreSQL
 
-To make a back up of the current PostgreSQL database, make sure it's running and then execute a command like this on the server:
+To automatically make Postgres database backups on a fixed schedule, see [Setting up postgres backup](configuring-playbook-postgres-backup.md).
+
+To make a one off back up of the current PostgreSQL database, make sure it's running and then execute a command like this on the server:
 
 ```bash
 /usr/bin/docker exec \

group_vars/matrix_servers

@@ -9,6 +9,7 @@
 # You can also override ANY variable (seen here or in any given role),
 # by re-defining it in your own configuration file (`inventory/host_vars/matrix.<your-domain>`).
 
+matrix_container_global_registry_prefix: "docker.io/"
 
 ######################################################################
 #
@@ -1677,7 +1678,7 @@ matrix_prometheus_node_exporter_enabled: false
 # Normally, matrix-nginx-proxy is enabled and nginx can reach Prometheus Node Exporter over the container network.
 # If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose
 # Prometheus' HTTP port to the local host.
-matrix_prometheus_node_exporter_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9100' }}"
+matrix_prometheus_node_exporter_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9200' }}"
 
 ######################################################################
 #
@@ -1774,3 +1775,32 @@ matrix_registration_database_password: "{{ matrix_synapse_macaroon_secret_key |
 # /matrix-registration
 #
 ######################################################################
+
+######################################################################
+#
+# matrix-postgres-backup
+#
+######################################################################
+
+matrix_postgres_backup_connection_hostname: "{{ matrix_postgres_connection_hostname }}"
+matrix_postgres_backup_connection_port: "{{ matrix_postgres_connection_port }}"
+matrix_postgres_backup_connection_username: "{{ matrix_postgres_connection_username }}"
+matrix_postgres_backup_connection_password: "{{ matrix_postgres_connection_password }}"
+
+matrix_postgres_backup_postgres_data_path: "{{ matrix_postgres_data_path if matrix_postgres_enabled else '' }}"
+
+# the default matrix synapse databse is not always part of the matrix_postgres_additional_databases variable thus we have to add it if the default database is used
+matrix_postgres_backup_databases: |
+  {{
+    (([{
+        'name': matrix_synapse_database_database
+    }] if (matrix_synapse_enabled and matrix_synapse_database_database == matrix_postgres_db_name and matrix_synapse_database_host == 'matrix-postgres') else [])
+    +
+    matrix_postgres_additional_databases)|map(attribute='name')|list
+  }}
+
+######################################################################
+#
+# /matrix-postgres-backup
+#
+######################################################################

roles/matrix-awx/tasks/main.yml

@@ -26,6 +26,15 @@
   tags:
     - create-user
 
+# Purge local/remote media if called
+- include_tasks: 
+    file: "purge_media_main.yml"
+    apply:
+      tags: purge-media
+  when: run_setup|bool and matrix_awx_enabled|bool
+  tags:
+    - purge-media
+
 # Import configs, media repo from /chroot/backup import
 - include_tasks: 
     file: "import_awx.yml"

roles/matrix-awx/tasks/purge_media_local.yml

@@ -0,0 +1,13 @@
+
+- name: Collect epoche time from date
+  shell: |
+    date -d '{{ item }}' +"%s"
+  register: epoche_time
+
+- name: Purge local media to specific date
+  shell: |
+    curl -X POST --header "Authorization: Bearer {{ janitors_token.stdout }}" 'https://matrix.{{ matrix_domain }}/_synapse/admin/v1/media/matrix.{{ matrix_domain }}/delete?before_ts={{ epoche_time.stdout }}'
+
+- name: Pause for 5 seconds to let Synapse breathe
+  pause:
+    seconds: 5

roles/matrix-awx/tasks/purge_media_main.yml

@@ -0,0 +1,94 @@
+
+- name: Ensure dateutils and curl is installed in AWX
+  delegate_to: 127.0.0.1
+  yum:
+    name: dateutils
+    state: latest
+
+- name: Include vars in matrix_vars.yml
+  include_vars:
+    file: '/var/lib/awx/projects/clients/{{ member_id }}/{{ subscription_id }}/matrix_vars.yml'
+  no_log: True
+  
+- name: Ensure curl and jq intalled on target machine
+  apt:
+    pkg:
+    - curl
+    - jq
+    state: present
+
+- name: Collect access token for janitor user
+  shell: |
+    curl -XPOST -d '{"type":"m.login.password", "user":"janitor", "password":"{{ matrix_awx_janitor_user_password }}"}' "https://matrix.{{ matrix_domain }}/_matrix/client/r0/login" | jq '.access_token'
+  register: janitors_token
+    
+- name: Generate list of dates to purge to
+  delegate_to: 127.0.0.1
+  shell: "dateseq {{ matrix_purge_from_date }} {{ matrix_purge_to_date }}"
+  register: purge_dates
+    
+- name: Calculate initial size of local media repository
+  shell: du -sh /matrix/synapse/storage/media-store/local*
+  register: local_media_size_before
+  when: matrix_purge_media_type == "Local Media"
+  ignore_errors: yes
+  no_log: True
+
+- name: Calculate initial size of remote media repository
+  shell: du -sh /matrix/synapse/storage/media-store/remote*
+  register: remote_media_size_before
+  when: matrix_purge_media_type == "Remote Media"
+  ignore_errors: yes
+  no_log: True
+
+- name: Purge local media with loop
+  include_tasks: purge_media_local.yml 
+  loop: "{{ purge_dates.stdout_lines | flatten(levels=1) }}"
+  when: matrix_purge_media_type == "Local Media"
+
+- name: Purge remote media with loop
+  include_tasks: purge_media_remote.yml 
+  loop: "{{ purge_dates.stdout_lines | flatten(levels=1) }}"
+  when: matrix_purge_media_type == "Remote Media"
+
+- name: Calculate final size of local media repository
+  shell: du -sh /matrix/synapse/storage/media-store/local*
+  register: local_media_size_after
+  when: matrix_purge_media_type == "Local Media"
+  ignore_errors: yes
+  no_log: True
+
+- name: Calculate final size of remote media repository
+  shell: du -sh /matrix/synapse/storage/media-store/remote*
+  register: remote_media_size_after
+  when: matrix_purge_media_type == "Remote Media"
+  ignore_errors: yes
+  no_log: True
+
+- name: Print size of local media repository before purge
+  debug:
+    msg: "{{ local_media_size_before.stdout.split('\n') }}"
+  when: matrix_purge_media_type == "Local Media"
+
+- name: Print size of local media repository after purge
+  debug:
+    msg: "{{ local_media_size_after.stdout.split('\n') }}"
+  when: matrix_purge_media_type == "Local Media"
+
+- name: Print size of remote media repository before purge
+  debug:
+    msg: "{{ remote_media_size_before.stdout.split('\n') }}"
+  when: matrix_purge_media_type == "Remote Media"
+
+- name: Print size of remote media repository after purge
+  debug:
+    msg: "{{ remote_media_size_after.stdout.split('\n') }}"
+  when: matrix_purge_media_type == "Remote Media"
+
+- name: Set boolean value to exit playbook
+  set_fact:
+    end_playbook: true
+
+- name: End playbook early if this task is called.
+  meta: end_play
+  when: end_playbook is defined and end_playbook|bool

roles/matrix-awx/tasks/purge_media_remote.yml

@@ -0,0 +1,13 @@
+
+- name: Collect epoche time from date
+  shell: |
+    date -d '{{ item }}' +"%s"
+  register: epoche_time
+
+- name: Purge local media to specific date
+  shell: |
+    curl -X POST --header "Authorization: Bearer {{ janitors_token.stdout }}" 'https://matrix.{{ matrix_domain }}/_synapse/admin/v1/purge_media_cache?before_ts={{ epoche_time.stdout }}'
+
+- name: Pause for 5 seconds to let Synapse breathe
+  pause:
+    seconds: 5

roles/matrix-base/tasks/sanity_check.yml

@@ -52,3 +52,10 @@
   when:
     - ansible_distribution == 'Archlinux'
     - ansible_python.version.major != 3
+
+- name: Fail if architecture is set incorrectly
+  fail:
+    msg: "Detected that variable matrix_architecture {{ matrix_architecture }} appears to be set incorrectly. See docs/alternative-architectures.md. Server appears to be {{ ansible_architecture }}."
+  when: (ansible_architecture == "x86_64" and matrix_architecture != "amd64") or
+        (ansible_architecture == "aarch64" and matrix_architecture != "arm64") or
+        (ansible_architecture.startswith("armv") and matrix_architecture != "arm32")

roles/matrix-bot-matrix-reminder-bot/defaults/main.yml

@@ -3,7 +3,7 @@
 
 matrix_bot_matrix_reminder_bot_enabled: true
 matrix_bot_matrix_reminder_bot_version: release-v0.2.0
-matrix_bot_matrix_reminder_bot_docker_image: "docker.io/anoa/matrix-reminder-bot:{{ matrix_bot_matrix_reminder_bot_version }}"
+matrix_bot_matrix_reminder_bot_docker_image: "{{ matrix_container_global_registry_prefix }}anoa/matrix-reminder-bot:{{ matrix_bot_matrix_reminder_bot_version }}"
 matrix_bot_matrix_reminder_bot_docker_image_force_pull: "{{ matrix_bot_matrix_reminder_bot_docker_image.endswith(':latest') }}"
 
 matrix_bot_matrix_reminder_bot_base_path: "{{ matrix_base_data_path }}/matrix-reminder-bot"

roles/matrix-bot-mjolnir/defaults/main.yml

@@ -3,7 +3,7 @@
 
 matrix_bot_mjolnir_enabled: true
 matrix_bot_mjolnir_version: "v0.1.17"
-matrix_bot_mjolnir_docker_image: "docker.io/matrixdotorg/mjolnir:{{ matrix_bot_mjolnir_version }}"
+matrix_bot_mjolnir_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/mjolnir:{{ matrix_bot_mjolnir_version }}"
 matrix_bot_mjolnir_docker_image_force_pull: "{{ matrix_bot_mjolnir_docker_image.endswith(':latest') }}"
 
 matrix_bot_mjolnir_base_path: "{{ matrix_base_data_path }}/mjolnir"

roles/matrix-bridge-appservice-discord/defaults/main.yml

@@ -4,7 +4,7 @@
 matrix_appservice_discord_enabled: true
 
 matrix_appservice_discord_version: v1.0.0
-matrix_appservice_discord_docker_image: "docker.io/halfshot/matrix-appservice-discord:{{ matrix_appservice_discord_version }}"
+matrix_appservice_discord_docker_image: "{{ matrix_container_global_registry_prefix }}halfshot/matrix-appservice-discord:{{ matrix_appservice_discord_version }}"
 matrix_appservice_discord_docker_image_force_pull: "{{ matrix_appservice_discord_docker_image.endswith(':latest') }}"
 
 matrix_appservice_discord_base_path: "{{ matrix_base_data_path }}/appservice-discord"

roles/matrix-bridge-appservice-irc/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_appservice_irc_docker_repo: "https://github.com/matrix-org/matrix-appserv
 matrix_appservice_irc_docker_src_files_path: "{{ matrix_base_data_path }}/appservice-irc/docker-src"
 
 matrix_appservice_irc_version: release-0.25.0
-matrix_appservice_irc_docker_image: "docker.io/matrixdotorg/matrix-appservice-irc:{{ matrix_appservice_irc_version }}"
+matrix_appservice_irc_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/matrix-appservice-irc:{{ matrix_appservice_irc_version }}"
 matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_image.endswith(':latest') }}"
 
 matrix_appservice_irc_base_path: "{{ matrix_base_data_path }}/appservice-irc"

roles/matrix-bridge-appservice-slack/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_appservice_slack_docker_repo: "https://github.com/matrix-org/matrix-appse
 matrix_appservice_slack_docker_src_files_path: "{{ matrix_base_data_path }}/appservice-slack/docker-src"
 
 matrix_appservice_slack_version: release-1.5.0
-matrix_appservice_slack_docker_image: "docker.io/matrixdotorg/matrix-appservice-slack:{{ matrix_appservice_slack_version }}"
+matrix_appservice_slack_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/matrix-appservice-slack:{{ matrix_appservice_slack_version }}"
 matrix_appservice_slack_docker_image_force_pull: "{{ matrix_appservice_slack_docker_image.endswith(':latest') }}"
 
 matrix_appservice_slack_base_path: "{{ matrix_base_data_path }}/appservice-slack"

roles/matrix-bridge-appservice-webhooks/defaults/main.yml

@@ -4,7 +4,7 @@
 matrix_appservice_webhooks_enabled: true
 
 matrix_appservice_webhooks_version: latest
-matrix_appservice_webhooks_docker_image: "docker.io/turt2live/matrix-appservice-webhooks:{{ matrix_appservice_webhooks_version }}"
+matrix_appservice_webhooks_docker_image: "{{ matrix_container_global_registry_prefix }}turt2live/matrix-appservice-webhooks:{{ matrix_appservice_webhooks_version }}"
 matrix_appservice_webhooks_docker_image_force_pull: "{{ matrix_appservice_webhooks_docker_image.endswith(':latest') }}"
 
 matrix_appservice_webhooks_base_path: "{{ matrix_base_data_path }}/appservice-webhooks"

roles/matrix-bridge-appservice-webhooks/tasks/init.yml

@@ -44,16 +44,19 @@
   - name: Generate Matrix Appservice webhooks proxying configuration for matrix-nginx-proxy
     set_fact:
       matrix_appservice_webhooks_matrix_nginx_proxy_configuration: |
-        location {{ matrix_appservice_webhooks_public_endpoint }}/ {
         {% if matrix_nginx_proxy_enabled|default(False) %}
         {# Use the embedded DNS resolver in Docker containers to discover the service #}
+        location ~ ^{{ matrix_appservice_webhooks_public_endpoint }}/(.*)$ {
           resolver 127.0.0.11 valid=5s;
-          proxy_pass {{ matrix_appservice_webhooks_appservice_url }}:{{ matrix_appservice_webhooks_matrix_port }}/;
-        {% else %}
-          {# Generic configuration for use outside of our container setup #}
-          proxy_pass http://127.0.0.1:{{ matrix_appservice_webhooks_matrix_port }}/;
-        {% endif %}
+          set $backend "matrix-appservice-webhooks:{{ matrix_appservice_webhooks_matrix_port }}";
+          proxy_pass http://$backend/$1;
         }
+        {% else %}
+        {# Generic configuration for use outside of our container setup #}
+        location {{ matrix_appservice_webhooks_public_endpoint }}/ {
+          proxy_pass http://127.0.0.1:{{ matrix_appservice_webhooks_matrix_port }}/;
+        }
+        {% endif %}
 
   - name: Register webhooks Appservice proxying configuration with matrix-nginx-proxy
     set_fact:

roles/matrix-bridge-mx-puppet-discord/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_mx_puppet_discord_container_http_host_bind_port: ''
 
 matrix_mx_puppet_discord_version: latest
 matrix_mx_puppet_discord_docker_image: "{{ matrix_mx_puppet_discord_docker_image_name_prefix }}sorunome/mx-puppet-discord:{{ matrix_mx_puppet_discord_version }}"
-matrix_mx_puppet_discord_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_discord_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_discord_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_discord_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_discord_docker_image_force_pull: "{{ matrix_mx_puppet_discord_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_discord_base_path: "{{ matrix_base_data_path }}/mx-puppet-discord"

roles/matrix-bridge-mx-puppet-groupme/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_mx_puppet_groupme_container_http_host_bind_port: ''
 
 matrix_mx_puppet_groupme_version: latest
 matrix_mx_puppet_groupme_docker_image: "{{ matrix_mx_puppet_groupme_docker_image_name_prefix }}xangelix/mx-puppet-groupme:{{ matrix_mx_puppet_groupme_version }}"
-matrix_mx_puppet_groupme_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_groupme_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_groupme_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_groupme_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_groupme_docker_image_force_pull: "{{ matrix_mx_puppet_groupme_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_groupme_base_path: "{{ matrix_base_data_path }}/mx-puppet-groupme"

roles/matrix-bridge-mx-puppet-instagram/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_mx_puppet_instagram_container_image_self_build_repo: "https://github.com/
 
 matrix_mx_puppet_instagram_version: latest
 matrix_mx_puppet_instagram_docker_image: "{{ matrix_mx_puppet_instagram_docker_image_name_prefix }}sorunome/mx-puppet-instagram:{{ matrix_mx_puppet_instagram_version }}"
-matrix_mx_puppet_instagram_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_instagram_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_instagram_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_instagram_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_instagram_docker_image_force_pull: "{{ matrix_mx_puppet_instagram_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_instagram_base_path: "{{ matrix_base_data_path }}/mx-puppet-instagram"

roles/matrix-bridge-mx-puppet-skype/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_mx_puppet_skype_container_image_self_build_repo: "https://github.com/Soru
 
 matrix_mx_puppet_skype_version: latest
 matrix_mx_puppet_skype_docker_image: "{{ matrix_mx_puppet_skype_docker_image_name_prefix }}sorunome/mx-puppet-skype:{{ matrix_mx_puppet_skype_version }}"
-matrix_mx_puppet_skype_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_skype_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_skype_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_skype_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_skype_docker_image_force_pull: "{{ matrix_mx_puppet_skype_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_skype_base_path: "{{ matrix_base_data_path }}/mx-puppet-skype"

roles/matrix-bridge-mx-puppet-slack/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_mx_puppet_slack_container_http_host_bind_port: ''
 
 matrix_mx_puppet_slack_version: latest
 matrix_mx_puppet_slack_docker_image: "{{ matrix_mx_puppet_slack_docker_image_name_prefix }}sorunome/mx-puppet-slack:{{ matrix_mx_puppet_slack_version }}"
-matrix_mx_puppet_slack_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_slack_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_slack_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_slack_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_slack_docker_image_force_pull: "{{ matrix_mx_puppet_slack_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_slack_base_path: "{{ matrix_base_data_path }}/mx-puppet-slack"

roles/matrix-bridge-mx-puppet-steam/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_mx_puppet_steam_container_http_host_bind_port: ''
 
 matrix_mx_puppet_steam_version: latest
 matrix_mx_puppet_steam_docker_image: "{{ matrix_mx_puppet_steam_docker_image_name_prefix }}icewind1991/mx-puppet-steam:{{ matrix_mx_puppet_steam_version }}"
-matrix_mx_puppet_steam_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_steam_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_steam_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_steam_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_steam_docker_image_force_pull: "{{ matrix_mx_puppet_steam_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_steam_base_path: "{{ matrix_base_data_path }}/mx-puppet-steam"

roles/matrix-bridge-mx-puppet-twitter/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_mx_puppet_twitter_container_http_host_bind_port: ''
 
 matrix_mx_puppet_twitter_version: latest
 matrix_mx_puppet_twitter_docker_image: "{{ matrix_mx_puppet_twitter_docker_image_name_prefix }}sorunome/mx-puppet-twitter:{{ matrix_mx_puppet_twitter_version }}"
-matrix_mx_puppet_twitter_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_twitter_container_image_self_build else 'docker.io/' }}"
+matrix_mx_puppet_twitter_docker_image_name_prefix: "{{ 'localhost/' if matrix_mx_puppet_twitter_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mx_puppet_twitter_docker_image_force_pull: "{{ matrix_mx_puppet_twitter_docker_image.endswith(':latest') }}"
 
 matrix_mx_puppet_twitter_base_path: "{{ matrix_base_data_path }}/mx-puppet-twitter"

roles/matrix-bridge-sms/defaults/main.yml

@@ -4,7 +4,7 @@
 matrix_sms_bridge_enabled: true
 
 matrix_sms_bridge_version: 0.5.5
-matrix_sms_bridge_docker_image: "docker.io/folivonet/matrix-sms-bridge:{{ matrix_sms_bridge_version }}"
+matrix_sms_bridge_docker_image: "{{ matrix_container_global_registry_prefix }}folivonet/matrix-sms-bridge:{{ matrix_sms_bridge_version }}"
 
 matrix_sms_bridge_base_path: "{{ matrix_base_data_path }}/matrix-sms-bridge"
 matrix_sms_bridge_config_path: "{{ matrix_base_data_path }}/matrix-sms-bridge/config"

roles/matrix-client-element/defaults/main.yml

@@ -3,9 +3,9 @@ matrix_client_element_enabled: true
 matrix_client_element_container_image_self_build: false
 matrix_client_element_container_image_self_build_repo: "https://github.com/vector-im/riot-web.git"
 
-matrix_client_element_version: v1.7.24.1
+matrix_client_element_version: v1.7.25
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
-matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else 'docker.io/' }}"
+matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"
 
 matrix_client_element_data_path: "{{ matrix_base_data_path }}/client-element"

roles/matrix-corporal/defaults/main.yml

@@ -24,7 +24,7 @@ matrix_corporal_systemd_required_services_list: ['docker.service']
 
 matrix_corporal_version: 2.1.0
 matrix_corporal_docker_image: "{{ matrix_corporal_docker_image_name_prefix }}devture/matrix-corporal:{{ matrix_corporal_docker_image_tag }}"
-matrix_corporal_docker_image_name_prefix: "{{ 'localhost/' if matrix_corporal_container_image_self_build else 'docker.io/' }}"
+matrix_corporal_docker_image_name_prefix: "{{ 'localhost/' if matrix_corporal_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_corporal_docker_image_tag: "{{ matrix_corporal_version }}" # for backward-compatibility
 matrix_corporal_docker_image_force_pull: "{{ matrix_corporal_docker_image.endswith(':latest') }}"
 

roles/matrix-coturn/defaults/main.yml

@@ -5,7 +5,7 @@ matrix_coturn_container_image_self_build_repo: "https://github.com/instrumentist
 
 matrix_coturn_version: 4.5.2
 matrix_coturn_docker_image: "{{ matrix_coturn_docker_image_name_prefix }}instrumentisto/coturn:{{ matrix_coturn_version }}"
-matrix_coturn_docker_image_name_prefix: "{{ 'localhost/' if matrix_coturn_container_image_self_build else 'docker.io/' }}"
+matrix_coturn_docker_image_name_prefix: "{{ 'localhost/' if matrix_coturn_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_coturn_docker_image_force_pull: "{{ matrix_coturn_docker_image.endswith(':latest') }}"
 
 # The Docker network that Coturn would be put into.

roles/matrix-dimension/defaults/main.yml

@@ -13,7 +13,7 @@ matrix_dimension_widgets_allow_self_signed_ssl_certificates: false
 matrix_dimension_base_path: "{{ matrix_base_data_path }}/dimension"
 
 matrix_dimension_version: latest
-matrix_dimension_docker_image: "docker.io/turt2live/matrix-dimension:{{ matrix_dimension_version }}"
+matrix_dimension_docker_image: "{{ matrix_container_global_registry_prefix }}turt2live/matrix-dimension:{{ matrix_dimension_version }}"
 matrix_dimension_docker_image_force_pull: "{{ matrix_dimension_docker_image.endswith(':latest') }}"
 
 # List of systemd services that matrix-dimension.service depends on.

roles/matrix-dynamic-dns/defaults/main.yml

@@ -9,7 +9,7 @@ matrix_dynamic_dns_version: v3.9.1-ls45
 # The docker container to use when in mode
 matrix_dynamic_dns_docker_image: "{{ matrix_dynamic_dns_docker_image_name_prefix }}linuxserver/ddclient:{{ matrix_dynamic_dns_version }}"
 
-matrix_dynamic_dns_docker_image_name_prefix: "{{ 'localhost/' if matrix_dynamic_dns_container_image_self_build else 'docker.io/' }}"
+matrix_dynamic_dns_docker_image_name_prefix: "{{ 'localhost/' if matrix_dynamic_dns_container_image_self_build else matrix_container_global_registry_prefix }}"
 
 # The image to force pull
 matrix_dynamic_dns_docker_image_force_pull: "{{ matrix_dynamic_dns_docker_image.endswith(':latest') }}"

roles/matrix-email2matrix/defaults/main.yml

@@ -4,7 +4,7 @@ matrix_email2matrix_base_path: "{{ matrix_base_data_path }}/email2matrix"
 matrix_email2matrix_config_dir_path: "{{ matrix_email2matrix_base_path }}/config"
 
 matrix_email2matrix_version: 1.0.1
-matrix_email2matrix_docker_image: "docker.io/devture/email2matrix:{{ matrix_email2matrix_version }}"
+matrix_email2matrix_docker_image: "{{ matrix_container_global_registry_prefix }}devture/email2matrix:{{ matrix_email2matrix_version }}"
 matrix_email2matrix_docker_image_force_pull: "{{ matrix_email2matrix_docker_image.endswith(':latest') }}"
 
 # A list of extra arguments to pass to the container

roles/matrix-etherpad/defaults/main.yml

@@ -3,7 +3,7 @@ matrix_etherpad_enabled: false
 matrix_etherpad_base_path: "{{ matrix_base_data_path }}/etherpad"
 
 matrix_etherpad_version: 1.8.12
-matrix_etherpad_docker_image: "docker.io/etherpad/etherpad:{{ matrix_etherpad_version }}"
+matrix_etherpad_docker_image: "{{ matrix_container_global_registry_prefix }}etherpad/etherpad:{{ matrix_etherpad_version }}"
 matrix_etherpad_docker_image_force_pull: "{{ matrix_etherpad_docker_image.endswith(':latest') }}"
 
 # List of systemd services that matrix-etherpad.service depends on.

roles/matrix-grafana/defaults/main.yml

@@ -3,8 +3,8 @@
 
 matrix_grafana_enabled: false
 
-matrix_grafana_version: 7.4.0
-matrix_grafana_docker_image: "docker.io/grafana/grafana:{{ matrix_grafana_version }}"
+matrix_grafana_version: 7.5.2
+matrix_grafana_docker_image: "{{ matrix_container_global_registry_prefix }}grafana/grafana:{{ matrix_grafana_version }}"
 matrix_grafana_docker_image_force_pull: "{{ matrix_grafana_docker_image.endswith(':latest') }}"
 
 # Not conditional, because when someone disables metrics

roles/matrix-grafana/templates/grafana.ini.j2

@@ -1,16 +1,16 @@
 [security]
 # default admin user, created on startup
-admin_user = {{ matrix_grafana_default_admin_user }}
+admin_user = "{{ matrix_grafana_default_admin_user }}"
 
-# default admin password, can be changed before first start of grafana,  or in profile settings
-admin_password = {{ matrix_grafana_default_admin_password }}
+# default admin password, can be changed before first start of grafana, or in profile settings
+admin_password = """{{ matrix_grafana_default_admin_password }}"""
 
 [auth.anonymous]
 # enable anonymous access
 enabled = {{ matrix_grafana_anonymous_access }}
 
 # specify organization name that should be used for unauthenticated users
-org_name = {{ matrix_grafana_anonymous_access_org_name }}
+org_name = "{{ matrix_grafana_anonymous_access_org_name }}"
 
 [dashboards]
 {% if matrix_synapse_metrics_enabled %}

roles/matrix-jitsi/defaults/main.yml

@@ -55,7 +55,7 @@ matrix_jitsi_enable_lobby: false
 matrix_jitsi_version: stable-5142
 matrix_jitsi_container_image_tag: "{{ matrix_jitsi_version }}" # for backward-compatibility
 
-matrix_jitsi_web_docker_image: "docker.io/jitsi/web:{{ matrix_jitsi_container_image_tag }}"
+matrix_jitsi_web_docker_image: "{{ matrix_container_global_registry_prefix }}jitsi/web:{{ matrix_jitsi_container_image_tag }}"
 matrix_jitsi_web_docker_image_force_pull: "{{ matrix_jitsi_web_docker_image.endswith(':latest') }}"
 
 matrix_jitsi_web_base_path: "{{ matrix_base_data_path }}/jitsi/web"
@@ -163,7 +163,7 @@ matrix_jitsi_web_custom_config_extension: ''
 matrix_jitsi_web_environment_variables_extension: ''
 
 
-matrix_jitsi_prosody_docker_image: "docker.io/jitsi/prosody:{{ matrix_jitsi_container_image_tag }}"
+matrix_jitsi_prosody_docker_image: "{{ matrix_container_global_registry_prefix }}jitsi/prosody:{{ matrix_jitsi_container_image_tag }}"
 matrix_jitsi_prosody_docker_image_force_pull: "{{ matrix_jitsi_prosody_docker_image.endswith(':latest') }}"
 
 matrix_jitsi_prosody_base_path: "{{ matrix_base_data_path }}/jitsi/prosody"
@@ -177,7 +177,7 @@ matrix_jitsi_prosody_container_extra_arguments: []
 matrix_jitsi_prosody_systemd_required_services_list: ['docker.service']
 
 
-matrix_jitsi_jicofo_docker_image: "docker.io/jitsi/jicofo:{{ matrix_jitsi_container_image_tag }}"
+matrix_jitsi_jicofo_docker_image: "{{ matrix_container_global_registry_prefix }}jitsi/jicofo:{{ matrix_jitsi_container_image_tag }}"
 matrix_jitsi_jicofo_docker_image_force_pull: "{{ matrix_jitsi_jicofo_docker_image.endswith(':latest') }}"
 
 matrix_jitsi_jicofo_base_path: "{{ matrix_base_data_path }}/jitsi/jicofo"
@@ -194,7 +194,7 @@ matrix_jitsi_jicofo_auth_user: focus
 matrix_jitsi_jicofo_auth_password: ''
 
 
-matrix_jitsi_jvb_docker_image: "docker.io/jitsi/jvb:{{ matrix_jitsi_container_image_tag }}"
+matrix_jitsi_jvb_docker_image: "{{ matrix_container_global_registry_prefix }}jitsi/jvb:{{ matrix_jitsi_container_image_tag }}"
 matrix_jitsi_jvb_docker_image_force_pull: "{{ matrix_jitsi_jvb_docker_image.endswith(':latest') }}"
 
 matrix_jitsi_jvb_base_path: "{{ matrix_base_data_path }}/jitsi/jvb"

roles/matrix-ma1sd/defaults/main.yml

@@ -12,7 +12,7 @@ matrix_ma1sd_architecture: "amd64"
 matrix_ma1sd_version: "2.4.0"
 
 matrix_ma1sd_docker_image: "{{ matrix_ma1sd_docker_image_name_prefix }}ma1uta/ma1sd:{{ matrix_ma1sd_version }}-{{ matrix_ma1sd_architecture }}"
-matrix_ma1sd_docker_image_name_prefix: "{{ 'localhost/' if matrix_ma1sd_container_image_self_build else 'docker.io/' }}"
+matrix_ma1sd_docker_image_name_prefix: "{{ 'localhost/' if matrix_ma1sd_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_ma1sd_docker_image_force_pull: "{{ matrix_ma1sd_docker_image.endswith(':latest') }}"
 
 matrix_ma1sd_base_path: "{{ matrix_base_data_path }}/ma1sd"

roles/matrix-mailer/defaults/main.yml

@@ -7,9 +7,9 @@ matrix_mailer_container_image_self_build_repository_url: "https://github.com/dev
 matrix_mailer_container_image_self_build_src_files_path: "{{ matrix_mailer_base_path }}/docker-src"
 matrix_mailer_container_image_self_build_version: "{{ matrix_mailer_docker_image.split(':')[1] }}"
 
-matrix_mailer_version: 4.93-r1
+matrix_mailer_version: 4.94-r0
 matrix_mailer_docker_image: "{{ matrix_mailer_docker_image_name_prefix }}devture/exim-relay:{{ matrix_mailer_version }}"
-matrix_mailer_docker_image_name_prefix: "{{ 'localhost/' if matrix_mailer_container_image_self_build else 'docker.io/' }}"
+matrix_mailer_docker_image_name_prefix: "{{ 'localhost/' if matrix_mailer_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_mailer_docker_image_force_pull: "{{ matrix_mailer_docker_image.endswith(':latest') }}"
 
 # The user/group that the container runs with.

roles/matrix-nginx-proxy/defaults/main.yml

@@ -1,10 +1,10 @@
 matrix_nginx_proxy_enabled: true
-matrix_nginx_proxy_version: 1.19.8-alpine
+matrix_nginx_proxy_version: 1.19.10-alpine
 
 # We use an official nginx image, which we fix-up to run unprivileged.
 # An alternative would be an `nginxinc/nginx-unprivileged` image, but
 # that is frequently out of date.
-matrix_nginx_proxy_docker_image: "docker.io/nginx:{{ matrix_nginx_proxy_version }}"
+matrix_nginx_proxy_docker_image: "{{ matrix_container_global_registry_prefix }}nginx:{{ matrix_nginx_proxy_version }}"
 matrix_nginx_proxy_docker_image_force_pull: "{{ matrix_nginx_proxy_docker_image.endswith(':latest') }}"
 
 matrix_nginx_proxy_base_path: "{{ matrix_base_data_path }}/nginx-proxy"
@@ -354,7 +354,7 @@ matrix_ssl_additional_domains_to_obtain_certificates_for: []
 
 # Controls whether to obtain production or staging certificates from Let's Encrypt.
 matrix_ssl_lets_encrypt_staging: false
-matrix_ssl_lets_encrypt_certbot_docker_image: "docker.io/certbot/certbot:{{ matrix_ssl_architecture }}-v1.11.0"
+matrix_ssl_lets_encrypt_certbot_docker_image: "{{ matrix_container_global_registry_prefix }}certbot/certbot:{{ matrix_ssl_architecture }}-v1.14.0"
 matrix_ssl_lets_encrypt_certbot_docker_image_force_pull: "{{ matrix_ssl_lets_encrypt_certbot_docker_image.endswith(':latest') }}"
 matrix_ssl_lets_encrypt_certbot_standalone_http_port: 2402
 matrix_ssl_lets_encrypt_support_email: ~

roles/matrix-postgres-backup/defaults/main.yml

@@ -0,0 +1,40 @@
+matrix_postgres_backup_enabled: false
+
+matrix_postgres_backup_connection_hostname: "matrix-postgres"
+matrix_postgres_backup_connection_port: 5432
+matrix_postgres_backup_connection_username: "matrix"
+matrix_postgres_backup_connection_password: ""
+
+matrix_postgres_backup_extra_opts: "-Z9 --schema=public --blobs"
+matrix_postgres_backup_schedule: "@daily"
+matrix_postgres_backup_keep_days: 7
+matrix_postgres_backup_keep_weeks: 4
+matrix_postgres_backup_keep_months: 12
+matrix_postgres_backup_healthcheck_port: "8080"
+matrix_postgres_backup_databases: []
+matrix_postgres_backup_path: "{{ matrix_base_data_path }}/postgres-backup"
+
+# Specifies where the Postgres data is.
+# We use this to autodetect the Postgres version during playbook runtime (by parsing the `PG_VERSION` file contained there).
+# You can leave this empty to prevent auto-detection.
+matrix_postgres_backup_postgres_data_path: ""
+
+matrix_postgres_backup_architecture: amd64
+
+# matrix_postgres_docker_image_suffix controls whether we use Alpine-based images (`-alpine`) or the normal Debian-based images.
+# Alpine-based Postgres images are smaller and we usually prefer them, but they don't work on ARM32 (tested on a Raspberry Pi 3 running Raspbian 10.7).
+# On ARM32, `-alpine` images fail with the following error:
+# > LOG:  startup process (PID 37) was terminated by signal 11: Segmentation fault
+matrix_postgres_backup_docker_image_suffix: "{{ '-alpine' if matrix_postgres_backup_architecture in ['amd64', 'arm64'] else '' }}"
+
+matrix_postgres_backup_docker_image_v9: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:9.6{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v10: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:10{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v11: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:11{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v12: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:12{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_v13: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:13{{ matrix_postgres_backup_docker_image_suffix }}"
+matrix_postgres_backup_docker_image_latest: "{{ matrix_postgres_backup_docker_image_v13 }}"
+
+# This variable is assigned at runtime. Overriding its value has no effect.
+matrix_postgres_backup_docker_image_to_use: '{{ matrix_postgres_backup_docker_image_latest }}'
+
+matrix_postgres_backup_docker_image_force_pull: "{{ matrix_postgres_backup_docker_image_to_use.endswith(':latest') }}"

roles/matrix-postgres-backup/tasks/init.yml

@@ -0,0 +1,3 @@
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-postgres-backup.service'] }}"
+  when: matrix_postgres_backup_enabled|bool

roles/matrix-postgres-backup/tasks/main.yml

@@ -0,0 +1,17 @@
+---
+
+- import_tasks: "{{ role_path }}/tasks/init.yml"
+  tags:
+    - always
+
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: "run_setup|bool and matrix_postgres_backup_enabled|bool"
+  tags:
+    - setup-all
+    - setup-postgres-backup
+
+- import_tasks: "{{ role_path }}/tasks/setup_postgres_backup.yml"
+  when: run_setup|bool
+  tags:
+    - setup-all
+    - setup-postgres-backup

roles/matrix-postgres-backup/tasks/setup_postgres_backup.yml

@@ -0,0 +1,103 @@
+---
+
+#
+# Tasks related to setting up an internal postgres server
+#
+- import_tasks: "{{ role_path }}/tasks/util/detect_existing_postgres_version.yml"
+  when: 'matrix_postgres_backup_enabled|bool and matrix_postgres_backup_postgres_data_path != ""'
+
+# If we have found an existing version (installed from before), we use its corresponding Docker image.
+# If not, we install using the latest Postgres.
+#
+# Upgrading is supposed to be performed separately and explicitly (see `upgrade_postgres.yml`).
+- set_fact:
+    matrix_postgres_backup_docker_image_to_use: "{{ matrix_postgres_backup_docker_image_latest if matrix_postgres_backup_detected_version_corresponding_docker_image|default('') == '' else matrix_postgres_backup_detected_version_corresponding_docker_image }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure postgres backup Docker image is pulled
+  docker_image:
+    name: "{{ matrix_postgres_backup_docker_image_to_use }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_postgres_backup_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_backup_docker_image_force_pull }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure Postgres backup paths exist
+  file:
+    path: "{{ item }}"
+    state: directory
+    mode: 0700
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - "{{ matrix_postgres_backup_path }}"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure Postgres environment variables file created
+  template:
+    src: "{{ role_path }}/templates/{{ item }}.j2"
+    dest: "{{ matrix_postgres_backup_path }}/{{ item }}"
+    mode: 0640
+  with_items:
+    - "env-postgres-backup"
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure matrix-postgres-backup.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-postgres-backup.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+    mode: 0644
+  register: matrix_postgres_backup_systemd_service_result
+  when: matrix_postgres_backup_enabled|bool
+
+- name: Ensure systemd reloaded after matrix-postgres-backup.service installation
+  service:
+    daemon_reload: yes
+  when: "matrix_postgres_backup_enabled|bool and matrix_postgres_backup_systemd_service_result.changed"
+
+#
+# Tasks related to getting rid of the internal postgres backup server (if it was previously enabled)
+#
+
+- name: Check existence of matrix-postgres-backup service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+  register: matrix_postgres_backup_service_stat
+  when: "not matrix_postgres_backup_enabled|bool"
+
+- name: Ensure matrix-postgres-backup is stopped
+  service:
+    name: matrix-postgres-backup
+    state: stopped
+    daemon_reload: yes
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Ensure matrix-postgres-backup.service doesn't exist
+  file:
+    path: "{{ matrix_systemd_path }}/matrix-postgres-backup.service"
+    state: absent
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Ensure systemd reloaded after matrix-postgres-backup.service removal
+  service:
+    daemon_reload: yes
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_service_stat.stat.exists"
+
+- name: Check existence of matrix-postgres-backup backup path
+  stat:
+    path: "{{ matrix_postgres_backup_path }}"
+  register: matrix_postgres_backup_path_stat
+  when: "not matrix_postgres_backup_enabled|bool"
+
+# We just want to notify the user. Deleting data is too destructive.
+- name: Inject warning if matrix-postgres backup data remains
+  set_fact:
+    matrix_playbook_runtime_results: |
+      {{
+        matrix_playbook_runtime_results|default([])
+        +
+        [
+          "NOTE: You are not using the local backup service to backup the PostgreSQL database, but some old data remains from before in `{{ matrix_postgres_backup_path }}`. Feel free to delete it."
+        ]
+      }}
+  when: "not matrix_postgres_backup_enabled|bool and matrix_postgres_backup_path_stat.stat.exists"

roles/matrix-postgres-backup/tasks/util/detect_existing_postgres_version.yml

@@ -0,0 +1,56 @@
+---
+
+# This utility aims to determine if there is some existing Postgres version in use or not.
+# If there is, it also tries to detect the Docker image that corresponds to that version.
+
+- name: Initialize Postgres version determination variables (default to empty)
+  set_fact:
+    matrix_postgres_backup_detection_pg_version_path: "{{ matrix_postgres_data_path }}/PG_VERSION"
+    matrix_postgres_backup_detected_existing: false
+    matrix_postgres_backup_detected_version: ""
+    matrix_postgres_backup_detected_version_corresponding_docker_image: ""
+
+- name: Determine existing Postgres version (check PG_VERSION file)
+  stat:
+    path: "{{ matrix_postgres_backup_detection_pg_version_path }}"
+  register: result_pg_version_stat
+
+- set_fact:
+    matrix_postgres_backup_detected_existing: true
+  when: "result_pg_version_stat.stat.exists"
+
+- name: Determine existing Postgres version (read PG_VERSION file)
+  slurp:
+    src: "{{ matrix_postgres_backup_detection_pg_version_path }}"
+  register: result_pg_version
+  when: matrix_postgres_backup_detected_existing|bool
+
+- name: Determine existing Postgres version (make sense of PG_VERSION file)
+  set_fact:
+    matrix_postgres_backup_detected_version: "{{ result_pg_version['content']|b64decode|replace('\n', '') }}"
+  when: matrix_postgres_backup_detected_existing|bool
+
+- name: Determine corresponding Docker image to detected version (assume default of latest)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_latest }}"
+  when: "matrix_postgres_backup_detected_version != ''"
+
+- name: Determine corresponding Docker image to detected version (use 9.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v9 }}"
+  when: "matrix_postgres_backup_detected_version.startswith('9.')"
+
+- name: Determine corresponding Docker image to detected version (use 10.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v10 }}"
+  when: "matrix_postgres_backup_detected_version == '10' or matrix_postgres_backup_detected_version.startswith('10.')"
+
+- name: Determine corresponding Docker image to detected version (use 11.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v11 }}"
+  when: "matrix_postgres_backup_detected_version == '11' or matrix_postgres_backup_detected_version.startswith('11.')"
+
+- name: Determine corresponding Docker image to detected version (use 12.x, if detected)
+  set_fact:
+    matrix_postgres_backup_detected_version_corresponding_docker_image: "{{ matrix_postgres_backup_docker_image_v12 }}"
+  when: "matrix_postgres_backup_detected_version == '12' or matrix_postgres_backup_detected_version.startswith('12.')"

roles/matrix-postgres-backup/tasks/validate_config.yml

@@ -0,0 +1,18 @@
+---
+
+- name: Fail if required Postgres settings not defined
+  fail:
+    msg: >-
+      You need to define a required configuration setting (`{{ item }}`).
+  when: "vars[item] == ''"
+  with_items:
+    - "matrix_postgres_backup_connection_hostname"
+    - "matrix_postgres_backup_connection_username"
+    - "matrix_postgres_backup_connection_password"
+    - "matrix_postgres_backup_connection_port"
+    - "matrix_postgres_backup_schedule"
+    - "matrix_postgres_backup_keep_days"
+    - "matrix_postgres_backup_keep_weeks"
+    - "matrix_postgres_backup_keep_months"
+    - "matrix_postgres_backup_path"
+    - "matrix_postgres_backup_databases"

roles/matrix-postgres-backup/templates/env-postgres-backup.j2

@@ -0,0 +1,12 @@
+#jinja2: lstrip_blocks: "True"
+POSTGRES_USER={{ matrix_postgres_backup_connection_username }}
+POSTGRES_PASSWORD={{ matrix_postgres_backup_connection_password }}
+POSTGRES_HOST={{ matrix_postgres_backup_connection_hostname }}
+POSTGRES_DB={{ matrix_postgres_backup_databases|join(', ') }}
+POSTGRES_EXTRA_OPTS={{ matrix_postgres_backup_extra_opts }}
+SCHEDULE={{ matrix_postgres_backup_schedule }}
+BACKUP_KEEP_DAYS={{ matrix_postgres_backup_keep_days }}
+BACKUP_KEEP_WEEKS={{ matrix_postgres_backup_keep_weeks }}
+BACKUP_KEEP_MONTHS={{ matrix_postgres_backup_keep_months }}
+HEALTHCHECK_PORT={{ matrix_postgres_backup_healthcheck_port }}
+POSTGRES_PORT={{ matrix_postgres_backup_connection_port }}

roles/matrix-postgres-backup/templates/systemd/matrix-postgres-backup.service.j2

@@ -0,0 +1,31 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description=Automatic Backup of  Matrix Postgres server
+After=docker.service
+Requires=docker.service
+DefaultDependencies=no
+
+[Service]
+Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
+ExecStartPre=-{{ matrix_host_command_docker }} stop matrix-postgres-backup
+ExecStartPre=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-postgres-backup 2>/dev/null'
+
+ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-postgres-backup \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--read-only \
+			--network={{ matrix_docker_network }} \
+			--env-file={{ matrix_postgres_backup_path }}/env-postgres-backup \
+			--mount type=bind,src={{ matrix_postgres_backup_path }},dst=/backups \
+			{{ matrix_postgres_backup_docker_image_to_use }}
+
+ExecStop=-{{ matrix_host_command_docker }} stop matrix-postgres-backup
+ExecStop=-{{ matrix_host_command_sh }} -c '{{ matrix_host_command_docker }} rm matrix-postgres-backup 2>/dev/null'
+Restart=always
+RestartSec=30
+SyslogIdentifier=matrix-postgres-backup
+
+[Install]
+WantedBy=multi-user.target

roles/matrix-postgres/defaults/main.yml

@@ -17,11 +17,11 @@ matrix_postgres_architecture: amd64
 # > LOG:  startup process (PID 37) was terminated by signal 11: Segmentation fault
 matrix_postgres_docker_image_suffix: "{{ '-alpine' if matrix_postgres_architecture in ['amd64', 'arm64'] else '' }}"
 
-matrix_postgres_docker_image_v9: "docker.io/postgres:9.6.21{{ matrix_postgres_docker_image_suffix }}"
-matrix_postgres_docker_image_v10: "docker.io/postgres:10.16{{ matrix_postgres_docker_image_suffix }}"
-matrix_postgres_docker_image_v11: "docker.io/postgres:11.11{{ matrix_postgres_docker_image_suffix }}"
-matrix_postgres_docker_image_v12: "docker.io/postgres:12.6{{ matrix_postgres_docker_image_suffix }}"
-matrix_postgres_docker_image_v13: "docker.io/postgres:13.2{{ matrix_postgres_docker_image_suffix }}"
+matrix_postgres_docker_image_v9: "{{ matrix_container_global_registry_prefix }}postgres:9.6.21{{ matrix_postgres_docker_image_suffix }}"
+matrix_postgres_docker_image_v10: "{{ matrix_container_global_registry_prefix }}postgres:10.16{{ matrix_postgres_docker_image_suffix }}"
+matrix_postgres_docker_image_v11: "{{ matrix_container_global_registry_prefix }}postgres:11.11{{ matrix_postgres_docker_image_suffix }}"
+matrix_postgres_docker_image_v12: "{{ matrix_container_global_registry_prefix }}postgres:12.6{{ matrix_postgres_docker_image_suffix }}"
+matrix_postgres_docker_image_v13: "{{ matrix_container_global_registry_prefix }}postgres:13.2{{ matrix_postgres_docker_image_suffix }}"
 matrix_postgres_docker_image_latest: "{{ matrix_postgres_docker_image_v13 }}"
 
 # This variable is assigned at runtime. Overriding its value has no effect.
@@ -90,6 +90,6 @@ matrix_postgres_pgloader_container_image_self_build_src_path: "{{ matrix_postgre
 # We use illagrenan/pgloader, instead of the more official dimitri/pgloader image,
 # because the official one only provides a `latest` tag.
 matrix_postgres_pgloader_docker_image: "{{ matrix_postgres_pgloader_docker_image_name_prefix }}illagrenan/pgloader:{{ matrix_postgres_pgloader_docker_image_tag }}"
-matrix_postgres_pgloader_docker_image_name_prefix: "{{ 'localhost/' if matrix_postgres_pgloader_container_image_self_build else 'docker.io/' }}"
+matrix_postgres_pgloader_docker_image_name_prefix: "{{ 'localhost/' if matrix_postgres_pgloader_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_postgres_pgloader_docker_image_tag: "3.6.2"
 matrix_postgres_pgloader_docker_image_force_pull: "{{ matrix_postgres_pgloader_docker_image.endswith(':latest') }}"

roles/matrix-prometheus-node-exporter/defaults/main.yml

@@ -4,7 +4,7 @@
 matrix_prometheus_node_exporter_enabled: false
 
 matrix_prometheus_node_exporter_version: v1.1.0
-matrix_prometheus_node_exporter_docker_image: "docker.io/prom/node-exporter:{{ matrix_prometheus_node_exporter_version }}"
+matrix_prometheus_node_exporter_docker_image: "{{ matrix_container_global_registry_prefix }}prom/node-exporter:{{ matrix_prometheus_node_exporter_version }}"
 matrix_prometheus_node_exporter_docker_image_force_pull: "{{ matrix_prometheus_node_exporter_docker_image.endswith(':latest') }}"
 
 # A list of extra arguments to pass to the container

roles/matrix-prometheus/defaults/main.yml

@@ -4,7 +4,7 @@
 matrix_prometheus_enabled: false
 
 matrix_prometheus_version: v2.24.1
-matrix_prometheus_docker_image: "docker.io/prom/prometheus:{{ matrix_prometheus_version }}"
+matrix_prometheus_docker_image: "{{ matrix_container_global_registry_prefix }}prom/prometheus:{{ matrix_prometheus_version }}"
 matrix_prometheus_docker_image_force_pull: "{{ matrix_prometheus_docker_image.endswith(':latest') }}"
 
 matrix_prometheus_base_path: "{{ matrix_base_data_path }}/prometheus"

roles/matrix-redis/defaults/main.yml

@@ -6,7 +6,7 @@ matrix_redis_base_path: "{{ matrix_base_data_path }}/redis"
 matrix_redis_data_path: "{{ matrix_redis_base_path }}/data"
 
 matrix_redis_version: 6.0.10-alpine
-matrix_redis_docker_image_v6: "docker.io/redis:{{ matrix_redis_version }}"
+matrix_redis_docker_image_v6: "{{ matrix_container_global_registry_prefix }}redis:{{ matrix_redis_version }}"
 matrix_redis_docker_image_latest: "{{ matrix_redis_docker_image_v6 }}"
 matrix_redis_docker_image_to_use: '{{ matrix_redis_docker_image_latest }}'
 

roles/matrix-registration/defaults/main.yml

@@ -15,7 +15,7 @@ matrix_registration_docker_src_files_path: "{{ matrix_registration_base_path }}/
 matrix_registration_version: "v0.7.2"
 
 matrix_registration_docker_image: "{{ matrix_registration_docker_image_name_prefix }}zeratax/matrix-registration:{{ matrix_registration_version }}"
-matrix_registration_docker_image_name_prefix: "{{ 'localhost/' if matrix_registration_container_image_self_build else 'docker.io/' }}"
+matrix_registration_docker_image_name_prefix: "{{ 'localhost/' if matrix_registration_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_registration_docker_image_force_pull: "{{ matrix_registration_docker_image.endswith(':latest') }}"
 
 # A list of extra arguments to pass to the container

roles/matrix-sygnal/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_sygnal_config_path: "{{ matrix_sygnal_base_path }}/config"
 matrix_sygnal_data_path: "{{ matrix_sygnal_base_path }}/data"
 
 matrix_sygnal_version: v0.9.0
-matrix_sygnal_docker_image: "docker.io/matrixdotorg/sygnal:{{ matrix_sygnal_version }}"
+matrix_sygnal_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/sygnal:{{ matrix_sygnal_version }}"
 matrix_sygnal_docker_image_force_pull: "{{ matrix_sygnal_docker_image.endswith(':latest') }}"
 
 # List of systemd services that matrix-sygnal.service depends on.

roles/matrix-synapse-admin/defaults/main.yml

@@ -10,7 +10,7 @@ matrix_synapse_admin_docker_src_files_path: "{{ matrix_base_data_path }}/synapse
 
 matrix_synapse_admin_version: 0.7.0
 matrix_synapse_admin_docker_image: "{{ matrix_synapse_admin_docker_image_name_prefix }}awesometechnologies/synapse-admin:{{ matrix_synapse_admin_version }}"
-matrix_synapse_admin_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_admin_container_self_build else 'docker.io/' }}"
+matrix_synapse_admin_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_admin_container_self_build else matrix_container_global_registry_prefix }}"
 matrix_synapse_admin_docker_image_force_pull: "{{ matrix_synapse_admin_docker_image.endswith(':latest') }}"
 
 # A list of extra arguments to pass to the container

roles/matrix-synapse/defaults/main.yml

@@ -7,7 +7,7 @@ matrix_synapse_container_image_self_build: false
 matrix_synapse_container_image_self_build_repo: "https://github.com/matrix-org/synapse.git"
 
 matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:{{ matrix_synapse_docker_image_tag }}"
-matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else 'docker.io/' }}"
+matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else matrix_container_global_registry_prefix }}"
 # The if statement below may look silly at times (leading to the same version being returned),
 # but ARM-compatible container images are only released 1-7 hours after a release,
 # so we may often be on different versions for different architectures when new Synapse releases come out.
@@ -15,8 +15,8 @@ matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_cont
 # amd64 gets released first.
 # arm32 relies on self-building, so the same version can be built immediately.
 # arm64 users need to wait for a prebuilt image to become available.
-matrix_synapse_version: v1.30.1
-matrix_synapse_version_arm64: v1.30.1
+matrix_synapse_version: v1.31.0
+matrix_synapse_version_arm64: v1.31.0
 matrix_synapse_docker_image_tag: "{{ matrix_synapse_version if matrix_architecture in ['arm32', 'amd64'] else matrix_synapse_version_arm64 }}"
 matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"
 

roles/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -869,10 +869,10 @@ rc_admin_redaction: {{ matrix_synapse_rc_admin_redaction|to_json }}
 #rc_joins:
 #  local:
 #    per_second: 0.1
-#    burst_count: 3
+#    burst_count: 10
 #  remote:
 #    per_second: 0.01
-#    burst_count: 3
+#    burst_count: 10
 rc_joins: {{ matrix_synapse_rc_joins|to_json }}
 #
 #rc_3pid_validation:
@@ -1772,6 +1772,9 @@ saml2_config:
 #       Note that, if this is changed, users authenticating via that provider
 #       will no longer be recognised as the same user!
 #
+#       (Use "oidc" here if you are migrating from an old "oidc_config"
+#       configuration.)
+#
 #   idp_name: A user-facing name for this identity provider, which is used to
 #       offer the user a choice of login mechanisms.
 #
@@ -1887,6 +1890,24 @@ saml2_config:
 #           which is set to the claims returned by the UserInfo Endpoint and/or
 #           in the ID Token.
 #
+#   It is possible to configure Synapse to only allow logins if certain attributes
+#   match particular values in the OIDC userinfo. The requirements can be listed under
+#   `attribute_requirements` as shown below. All of the listed attributes must
+#   match for the login to be permitted. Additional attributes can be added to
+#   userinfo by expanding the `scopes` section of the OIDC config to retrieve
+#   additional information from the OIDC provider.
+#
+#   If the OIDC claim is a list, then the attribute must match any value in the list.
+#   Otherwise, it must exactly match the value of the claim. Using the example
+#   below, the `family_name` claim MUST be "Stephensson", but the `groups`
+#   claim MUST contain "admin".
+#
+#   attribute_requirements:
+#     - attribute: family_name
+#       value: "Stephensson"
+#     - attribute: groups
+#       value: "admin"
+#
 # See https://github.com/matrix-org/synapse/blob/master/docs/openid.md
 # for information on how to configure these options.
 #
@@ -1918,34 +1939,9 @@ oidc_providers:
   #      localpart_template: "{% raw %}{{ user.login }}{% endraw %}"
   #      display_name_template: "{% raw %}{{ user.name }}{% endraw %}"
   #      email_template: "{% raw %}{{ user.email }}{% endraw %}"
-
-  # For use with Keycloak
-  #
-  #- idp_id: keycloak
-  #  idp_name: Keycloak
-  #  issuer: "https://127.0.0.1:8443/auth/realms/my_realm_name"
-  #  client_id: "synapse"
-  #  client_secret: "copy secret generated in Keycloak UI"
-  #  scopes: ["openid", "profile"]
-
-  # For use with Github
-  #
-  #- idp_id: github
-  #  idp_name: Github
-  #  idp_brand: github
-  #  discover: false
-  #  issuer: "https://github.com/"
-  #  client_id: "your-client-id" # TO BE FILLED
-  #  client_secret: "your-client-secret" # TO BE FILLED
-  #  authorization_endpoint: "https://github.com/login/oauth/authorize"
-  #  token_endpoint: "https://github.com/login/oauth/access_token"
-  #  userinfo_endpoint: "https://api.github.com/user"
-  #  scopes: ["read:user"]
-  #  user_mapping_provider:
-  #    config:
-  #      subject_claim: "id"
-  #      localpart_template: "{% raw %}{{ user.login }}{% endraw %}"
-  #      display_name_template: "{% raw %}{{ user.name }}{% endraw %}"
+  #  attribute_requirements:
+  #    - attribute: userGroup
+  #      value: "synapseUsers"
 
 
 # Enable Central Authentication Service (CAS) for registration and login.

roles/matrix-synapse/vars/workers.yml

@@ -55,9 +55,6 @@ matrix_synapse_workers_generic_worker_endpoints:
   # Registration/login requests
   - ^/_matrix/client/(api/v1|r0|unstable)/login$
   - ^/_matrix/client/(r0|unstable)/register$
-  # FIXME: possible bug with SSO and multiple generic workers
-  # see https://github.com/matrix-org/synapse/issues/7530
-  # ^/_matrix/client/(r0|unstable)/auth/.*/fallback/web$
 
   # Event sending requests
   - ^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/redact
@@ -107,7 +104,7 @@ matrix_synapse_workers_generic_worker_endpoints:
 
   # Ensure that all SSO logins go to a single process.
   # For multiple workers not handling the SSO endpoints properly, see
-  # [#7530](https://github.com/matrix-org/synapse/issues/7530) and 
+  # [#7530](https://github.com/matrix-org/synapse/issues/7530) and
   # [#9427](https://github.com/matrix-org/synapse/issues/9427).
 
   # Note that a HTTP listener with `client` and `federation` resources must be

setup.yml

@@ -51,4 +51,6 @@
     - matrix-nginx-proxy
     - matrix-coturn
     - matrix-aux
+    - matrix-postgres-backup
     - matrix-common-after
+