Merge branch 'master' into renovate-config

docs/configuring-playbook-synapse-admin.md

@@ -15,7 +15,7 @@ Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.
 matrix_synapse_admin_enabled: true
 ```
 
-**Note**: Synapse Admin requires Synapse's [Admin APIs](https://github.com/matrix-org/synapse/tree/master/docs/admin_api) to function. Access to them is restricted with a valid access token, so exposing them publicly should not be a real security concern. Still, for additional security, we normally leave them unexposed, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). Because Synapse Admin needs these APIs to function, when installing Synapse Admin, we **automatically** exposes them publicly for you (equivalent to `matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true`).
+**Note**: Synapse Admin requires Synapse's [Admin APIs](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html) to function. Access to them is restricted with a valid access token, so exposing them publicly should not be a real security concern. Still, for additional security, we normally leave them unexposed, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). Because Synapse Admin needs these APIs to function, when installing Synapse Admin, we **automatically** exposes them publicly for you (equivalent to `matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true`).
 
 
 ## Installing

docs/configuring-playbook-turn.md

@@ -16,13 +16,24 @@ matrix_coturn_enabled: false
 In that case, Synapse would not point to any Coturn servers and audio/video call functionality may fail.
 
 ## Manually defining your public IP
+
 In the `hosts` file we explicitly ask for your server's external IP address when defining `ansible_host`, because the same value is used for configuring Coturn.
+
 If you'd rather use a local IP for `ansible_host`, make sure to set up `matrix_coturn_turn_external_ip_address` replacing `YOUR_PUBLIC_IP` with the pubic IP used by the server.
 
 ```yaml
 matrix_coturn_turn_external_ip_address: "YOUR_PUBLIC_IP"
 ```
 
+If you'd like to rely on external IP address auto-detection (not recommended unless you need it), set `matrix_coturn_turn_external_ip_address` to an empty value. The playbook will automatically contact an [EchoIP](https://github.com/mpolden/echoip)-compatible service (`https://ifconfig.co/json` by default) to determine your server's IP address. This API endpoint is configurable via the `matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url` variable.
+
+If your server has multiple external IP addresses, the Coturn role offers a different variable for specifying them:
+
+```yaml
+# Note: matrix_coturn_turn_external_ip_addresses is different than matrix_coturn_turn_external_ip_address
+matrix_coturn_turn_external_ip_addresses: ['1.2.3.4', '4.5.6.7']
+```
+
 ## Using your own external Coturn server
 
 If you'd like to use another TURN server (be it Coturn or some other one), you can configure the playbook like this:

docs/maintenance-postgres.md

@@ -87,7 +87,7 @@ This playbook can upgrade your existing Postgres setup with the following comman
 just run-tags upgrade-postgres
 ```
 
-**Warning: If you're using Borg Backup keep in mind that there is no official Postgres 15 support yet.**
+**Warning: If you're using Borg Backup keep in mind that there is no official Postgres 16 support yet.**
 
 **The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres/data-auto-upgrade-backup`.
 To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"`

examples/vars.yml

@@ -41,3 +41,19 @@ devture_traefik_config_certificatesResolvers_acme_email: ''
 # The playbook creates additional Postgres users and databases (one for each enabled service)
 # using this superuser account.
 devture_postgres_connection_password: ''
+
+# By default, we configure Coturn's external IP address using the value specified for `ansible_host` in your `inventory/hosts` file.
+# If this value is an external IP address, you can skip this section.
+#
+# If `ansible_host` is not the server's external IP address, you have 2 choices:
+# 1. Uncomment the line below, to allow IP address auto-detection to happen (more on this below)
+# 2. Uncomment and adjust the line below to specify an IP address manually
+#
+# By default, auto-detection will be attempted using the `https://ifconfig.co/json` API.
+# Default values for this are specified in `matrix_coturn_turn_external_ip_address_auto_detection_*` variables in the Coturn role
+# (see `roles/custom/matrix-coturn/defaults/main.yml`).
+#
+# If your server has multiple IP addresses, you may define them in another variable which allows a list of addresses.
+# Example: `matrix_coturn_turn_external_ip_addresses: ['1.2.3.4', '4.5.6.7']`
+#
+# matrix_coturn_turn_external_ip_address: ''

group_vars/matrix_servers

@@ -2245,6 +2245,9 @@ matrix_coturn_enabled: true
 
 matrix_coturn_container_image_self_build: "{{ matrix_architecture not in ['amd64', 'arm32', 'arm64'] }}"
 
+# We make the assumption that `ansible_host` points to an external IP address, which may not always be the case.
+# Users are free to set `matrix_coturn_turn_external_ip_address` to an empty string
+# to allow auto-detection (via an EchoIP service) to happen at runtime.
 matrix_coturn_turn_external_ip_address: "{{ ansible_host }}"
 
 matrix_coturn_turn_static_auth_secret: "{{ '%s' | format(matrix_homeserver_generic_secret_key) | password_hash('sha512', 'coturn.sas', rounds=655555) | to_uuid }}"

requirements.yml

@@ -4,7 +4,7 @@
   version: v1.0.0-1
   name: auxiliary
 - src: git+https://gitlab.com/etke.cc/roles/backup_borg.git
-  version: v1.2.6-1.8.2-0
+  version: v1.2.6-1.8.3-0
 - src: git+https://github.com/devture/com.devture.ansible.role.container_socket_proxy.git
   version: v0.1.1-2
 - src: git+https://github.com/devture/com.devture.ansible.role.docker_sdk_for_python.git
@@ -26,7 +26,7 @@
 - src: git+https://github.com/devture/com.devture.ansible.role.timesync.git
   version: v1.0.0-0
 - src: git+https://github.com/devture/com.devture.ansible.role.traefik.git
-  version: v2.10.4-1
+  version: v2.10.5-0
 - src: git+https://github.com/devture/com.devture.ansible.role.traefik_certs_dumper.git
   version: v2.8.1-0
 - src: git+https://gitlab.com/etke.cc/roles/etherpad.git
@@ -35,17 +35,18 @@
   version: 7.0.1
   name: geerlingguy.docker
 - src: git+https://gitlab.com/etke.cc/roles/grafana.git
-  version: v10.1.4-0
+  version: v10.1.5-0
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-jitsi.git
   version: v8960-1
   name: jitsi
 - src: git+https://gitlab.com/etke.cc/roles/ntfy.git
   version: v2.7.0-2
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus.git
-  version: v2.47.1-0
+  version: v2.47.2-0
   name: prometheus
-- src: git+https://gitlab.com/etke.cc/roles/prometheus_node_exporter.git
+- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus-node-exporter.git
   version: v1.6.1-0
+  name: prometheus_node_exporter
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus-postgres-exporter.git
   version: v0.14.0-0
   name: prometheus_postgres_exporter

roles/custom/matrix-bridge-mautrix-discord/defaults/main.yml

@@ -9,7 +9,8 @@ matrix_mautrix_discord_container_image_self_build_repo: "https://mau.dev/mautrix
 matrix_mautrix_discord_container_image_self_build_branch: "{{ 'main' if matrix_mautrix_discord_version == 'latest' else matrix_mautrix_discord_version }}"
 
 # renovate: datasource=docker depName=dock.mau.dev/mautrix/discord
-matrix_mautrix_discord_version: v0.6.2
+matrix_mautrix_discord_version: v0.6.3
+
 # See: https://mau.dev/mautrix/discord/container_registry
 matrix_mautrix_discord_docker_image: "{{ matrix_mautrix_discord_docker_image_name_prefix }}mautrix/discord:{{ matrix_mautrix_discord_version }}"
 matrix_mautrix_discord_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_discord_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-bridge-mautrix-gmessages/defaults/main.yml

@@ -9,7 +9,8 @@ matrix_mautrix_gmessages_container_image_self_build_repo: "https://github.com/ma
 matrix_mautrix_gmessages_container_image_self_build_branch: "{{ 'main' if matrix_mautrix_gmessages_version == 'latest' else matrix_mautrix_gmessages_version }}"
 
 # renovate: datasource=docker depName=dock.mau.dev/mautrix/gmessages
-matrix_mautrix_gmessages_version: v0.2.0
+matrix_mautrix_gmessages_version: v0.2.1
+
 # See: https://mau.dev/mautrix/gmessages/container_registry
 matrix_mautrix_gmessages_docker_image: "{{ matrix_mautrix_gmessages_docker_image_name_prefix }}mautrix/gmessages:{{ matrix_mautrix_gmessages_version }}"
 matrix_mautrix_gmessages_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_gmessages_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-bridge-mautrix-whatsapp/defaults/main.yml

@@ -9,7 +9,8 @@ matrix_mautrix_whatsapp_container_image_self_build_repo: "https://mau.dev/mautri
 matrix_mautrix_whatsapp_container_image_self_build_branch: "{{ 'master' if matrix_mautrix_whatsapp_version == 'latest' else matrix_mautrix_whatsapp_version }}"
 
 # renovate: datasource=docker depName=dock.mau.dev/mautrix/whatsapp
-matrix_mautrix_whatsapp_version: v0.10.2
+matrix_mautrix_whatsapp_version: v0.10.3
+
 # See: https://mau.dev/mautrix/whatsapp/container_registry
 matrix_mautrix_whatsapp_docker_image: "{{ matrix_mautrix_whatsapp_docker_image_name_prefix }}mautrix/whatsapp:{{ matrix_mautrix_whatsapp_version }}"
 matrix_mautrix_whatsapp_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_whatsapp_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-client-element/defaults/main.yml

@@ -11,7 +11,8 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
 matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
 
 # renovate: datasource=docker depName=vectorim/element-web
-matrix_client_element_version: v1.11.45
+matrix_client_element_version: v1.11.46
+
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

roles/custom/matrix-client-hydrogen/tasks/setup_install.yml

@@ -12,6 +12,39 @@
     - {path: "{{ matrix_client_hydrogen_docker_src_files_path }}", when: "{{ matrix_client_hydrogen_container_image_self_build }}"}
   when: "item.when | bool"
 
+- when: "matrix_client_hydrogen_container_image_self_build | bool"
+  block:
+    - name: Ensure Hydrogen repository is present on self-build
+      ansible.builtin.git:
+        repo: "{{ matrix_client_hydrogen_container_image_self_build_repo }}"
+        dest: "{{ matrix_client_hydrogen_docker_src_files_path }}"
+        version: "{{ matrix_client_hydrogen_docker_image.split(':')[1] }}"
+        force: "yes"
+      become: true
+      become_user: "{{ matrix_user_username }}"
+      register: matrix_client_hydrogen_git_pull_results
+
+    - name: Check if Hydrogen Docker image exists
+      ansible.builtin.command: "{{ devture_systemd_docker_base_host_command_docker }} images --quiet --filter 'reference={{ matrix_client_hydrogen_docker_image }}'"
+      register: matrix_client_hydrogen_docker_image_check_result
+      changed_when: false
+
+    # Invoking the `docker build` command here, instead of calling the `docker_image` Ansible module,
+    # because the latter does not support BuildKit.
+    # See: https://github.com/ansible-collections/community.general/issues/514
+    - name: Ensure Hydrogen Docker image is built
+      ansible.builtin.shell:
+        chdir: "{{ matrix_client_hydrogen_docker_src_files_path }}"
+        cmd: |
+          {{ devture_systemd_docker_base_host_command_docker }} build \
+          -t "{{ matrix_client_hydrogen_docker_image }}" \
+          -f Dockerfile \
+          .
+      environment:
+        DOCKER_BUILDKIT: 1
+      changed_when: true
+      when: "matrix_client_hydrogen_git_pull_results.changed | bool or matrix_client_hydrogen_docker_image_check_result.stdout == ''"
+
 - name: Ensure Hydrogen Docker image is pulled
   community.docker.docker_image:
     name: "{{ matrix_client_hydrogen_docker_image }}"
@@ -24,17 +57,6 @@
   delay: "{{ devture_playbook_help_container_retries_delay }}"
   until: result is not failed
 
-- name: Ensure Hydrogen repository is present on self-build
-  ansible.builtin.git:
-    repo: "{{ matrix_client_hydrogen_container_image_self_build_repo }}"
-    dest: "{{ matrix_client_hydrogen_docker_src_files_path }}"
-    version: "{{ matrix_client_hydrogen_docker_image.split(':')[1] }}"
-    force: "yes"
-  become: true
-  become_user: "{{ matrix_user_username }}"
-  register: matrix_client_hydrogen_git_pull_results
-  when: "matrix_client_hydrogen_container_image_self_build | bool"
-
 - name: Ensure Hydrogen configuration installed
   ansible.builtin.copy:
     content: "{{ matrix_client_hydrogen_configuration | to_nice_json }}"
@@ -54,17 +76,6 @@
     - {src: "{{ role_path }}/templates/nginx.conf.j2", name: "nginx.conf"}
     - {src: "{{ role_path }}/templates/labels.j2", name: "labels"}
 
-- name: Ensure Hydrogen Docker image is built
-  community.docker.docker_image:
-    name: "{{ matrix_client_hydrogen_docker_image }}"
-    source: build
-    force_source: "{{ matrix_client_hydrogen_git_pull_results.changed }}"
-    build:
-      dockerfile: Dockerfile
-      path: "{{ matrix_client_hydrogen_docker_src_files_path }}"
-      pull: true
-  when: "matrix_client_hydrogen_container_image_self_build | bool"
-
 - name: Ensure Hydrogen container network is created
   community.general.docker_network:
     name: "{{ matrix_client_hydrogen_container_network }}"

roles/custom/matrix-coturn/defaults/main.yml

@@ -74,8 +74,25 @@ matrix_coturn_turn_udp_max_port: 49172
 matrix_coturn_turn_static_auth_secret: ""
 
 # The external IP address of the machine where Coturn is.
+# If do not define an IP address here or in `matrix_coturn_turn_external_ip_addresses`, auto-detection via an EchoIP service will be done.
+# See `matrix_coturn_turn_external_ip_address_auto_detection_enabled`
 matrix_coturn_turn_external_ip_address: ''
-matrix_coturn_turn_external_ip_addresses: ["{{ matrix_coturn_turn_external_ip_address }}"]
+matrix_coturn_turn_external_ip_addresses: "{{ [matrix_coturn_turn_external_ip_address] if matrix_coturn_turn_external_ip_address != '' else [] }}"
+
+# Controls whether external IP address auto-detection should be attempted.
+# We try to do this if there is no external IP address explicitly configured and if an EchoIP service URL is specified.
+# See matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url
+matrix_coturn_turn_external_ip_address_auto_detection_enabled: "{{ matrix_coturn_turn_external_ip_addresses | length == 0 and matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url != '' }}"
+
+# Specifies the address of the EchoIP service (https://github.com/mpolden/echoip) to use for detecting the external IP address.
+# By default, we use the official public instance.
+matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url: https://ifconfig.co/json
+
+# Controls whether SSL certificates will be validated when contacting the EchoIP service (matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url)
+matrix_coturn_turn_external_ip_address_auto_detection_echoip_validate_certs: true
+
+matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_retries_count: "{{ devture_playbook_help_geturl_retries_count }}"
+matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_retries_delay: "{{ devture_playbook_help_geturl_retries_delay }}"
 
 matrix_coturn_allowed_peer_ips: []
 

roles/custom/matrix-coturn/tasks/setup_install.yml

@@ -1,5 +1,37 @@
 ---
 
+- when: matrix_coturn_turn_external_ip_address_auto_detection_enabled | bool
+  block:
+    - name: Fail if enabled, but EchoIP service URL unset
+      when: matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url == ''
+      ansible.builtin.fail:
+        msg: "To use the external IP address auto-detection feature, you need to set matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url"
+
+    # NOTE:
+    # `ansible.builtin.uri` does not provide a way to configure whether IPv4 or IPv6 is used.
+    # Luckily, the default instance we use does not define AAAA records for now, so it's always IPv4.
+    - name: Fetch IP address information from EchoIP service
+      ansible.builtin.uri:
+        url: "{{ matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url }}"
+        headers:
+          Content-Type: application/json
+        follow_redirects: none
+        validate_certs: "{{ matrix_coturn_turn_external_ip_address_auto_detection_echoip_validate_certs }}"
+      register: result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response
+      ignore_errors: true
+      check_mode: false
+      retries: "{{ matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_retries_count }}"
+      delay: "{{ matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_retries_delay }}"
+      until: not result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response.failed
+
+    - name: Fail if EchoIP service failed
+      when: "(result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response.failed or 'json' not in result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response)"
+      ansible.builtin.fail:
+        msg: "Failed contacting EchoIP service API at `{{ matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url }}` (controlled by `matrix_coturn_turn_external_ip_address_auto_detection_echoip_service_url`). Full error: {{ result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response }}"
+
+    - ansible.builtin.set_fact:
+        matrix_coturn_turn_external_ip_address: "{{ result_matrix_coturn_turn_external_ip_address_auto_detection_echoip_response.json.ip }}"
+
 - name: Ensure Matrix Coturn path exists
   ansible.builtin.file:
     path: "{{ item.path }}"

roles/custom/matrix-coturn/templates/turnserver.conf.j2

@@ -5,7 +5,7 @@ realm=turn.{{ matrix_server_fqn_matrix }}
 
 min-port={{ matrix_coturn_turn_udp_min_port }}
 max-port={{ matrix_coturn_turn_udp_max_port }}
-{% for ip in matrix_coturn_turn_external_ip_addresses|select('ne', '') %}
+{% for ip in matrix_coturn_turn_external_ip_addresses %}
 external-ip={{ ip }}
 {% endfor %}
 

roles/custom/matrix-dynamic-dns/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_dynamic_dns_enabled: true
 matrix_dynamic_dns_daemon_interval: '300'
 
 # renovate: datasource=docker depName=linuxserver/ddclient
-matrix_dynamic_dns_version: v3.10.0-ls135
+matrix_dynamic_dns_version: v3.10.0-ls136
 
 # The docker container to use when in mode
 matrix_dynamic_dns_docker_image: "{{ matrix_dynamic_dns_docker_image_name_prefix }}linuxserver/ddclient:{{ matrix_dynamic_dns_version }}"

roles/custom/matrix-sliding-sync/defaults/main.yml

@@ -6,7 +6,7 @@
 matrix_sliding_sync_enabled: true
 
 # renovate: datasource=docker depName=ghcr.io/matrix-org/sliding-sync
-matrix_sliding_sync_version: v0.99.10
+matrix_sliding_sync_version: v0.99.11
 
 matrix_sliding_sync_scheme: https
 
@@ -86,6 +86,9 @@ matrix_sliding_sync_environment_variable_syncv3_secret: ''
 # Controls the SYNCV3_DB environment variable
 matrix_sliding_sync_environment_variable_syncv3_db: 'user={{ matrix_sliding_sync_database_username }} password={{ matrix_sliding_sync_database_password }} host={{ matrix_sliding_sync_database_hostname }} port={{ matrix_sliding_sync_database_port }} dbname={{ matrix_sliding_sync_database_name }} sslmode={{ matrix_sliding_sync_database_sslmode }}'
 
+# Additional environment variables.
+matrix_sliding_sync_environment_variables_additional_variables: ''
+
 matrix_sliding_sync_database_username: 'matrix_sliding_sync'
 matrix_sliding_sync_database_password: ''
 matrix_sliding_sync_database_hostname: ''

roles/custom/matrix-sliding-sync/templates/env.j2

@@ -2,3 +2,5 @@ SYNCV3_SERVER={{ matrix_sliding_sync_environment_variable_syncv3_server }}
 SYNCV3_SECRET={{ matrix_sliding_sync_environment_variable_syncv3_secret }}
 SYNCV3_BINDADDR=:8008
 SYNCV3_DB={{ matrix_sliding_sync_environment_variable_syncv3_db }}
+
+{{ matrix_sliding_sync_environment_variables_additional_variables }}

roles/custom/matrix-synapse/defaults/main.yml

@@ -5,7 +5,7 @@
 matrix_synapse_enabled: true
 
 # renovate: datasource=docker depName=matrixdotorg/synapse
-matrix_synapse_version: v1.93.0
+matrix_synapse_version: v1.94.0
 
 matrix_synapse_username: ''
 matrix_synapse_uid: ''