From 94c1503a609f4f6f76815f03ae99432865a10f49 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 16 Dec 2023 09:23:35 +0200 Subject: [PATCH] Add support for experimental encryption in Hookshot Squashed based on the work done in https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3042 commit 49932b8f3c17c4c4db7a884658c42f9a8b0550ca Author: Slavi Pantaleev Date: Sat Dec 16 09:21:31 2023 +0200 Fix syntax in matrix-bridge-hookshot/tasks/reset_encryption.yml Also, this task always does work and side-effects, so it should always report changes (`changed_when: true`). commit 6bdf7a9dcb73385313a1f34d52e27ad0cf95fb3e Author: Slavi Pantaleev Date: Sat Dec 16 09:12:41 2023 +0200 Add Hookshot validation task to ensure queue settings are set when encryption is enabled commit 8c531b7971b5dfd15ca541b5072b3eb8237cdcf9 Author: Slavi Pantaleev Date: Sat Dec 16 09:10:17 2023 +0200 Add missing variables rewiring in group_vars/matrix_servers for Hookshot commit 7d26dabc2fe692f5e1236c0e250f85996f3fd0c2 Author: Slavi Pantaleev Date: Sat Dec 16 09:08:19 2023 +0200 Add defaults for matrix_hookshot_queue_host and matrix_hookshot_queue_port commit 74f91138c92f1d1b69eb973803b882849e31a259 Author: Slavi Pantaleev Date: Sat Dec 16 09:06:17 2023 +0200 Fix syntax for connecting to additional networks for Hookshot commit ca7b41f3f2d0c2900b2805294476f70e82461304 Author: Slavi Pantaleev Date: Sat Dec 16 09:05:28 2023 +0200 Fix indentation and remove unnecessary if-statements commit ac4a918d58fc76e7332446a65609b56c5e4da00c Author: Slavi Pantaleev Date: Sat Dec 16 09:04:44 2023 +0200 Add missing --network for Hookshot This seems to have been removed by accident. commit 6a81fa208fca28951fc131cbf33b95ec78748a40 Author: Slavi Pantaleev Date: Sat Dec 16 09:02:47 2023 +0200 Make automatic Redis enabling safer, when Hookshot encryption enabled If we ever default encryption to enabled for Hookshot, we only wish to force-enable Redis if Hookshot is actually enabled. commit 75a8e0f2a6cbf1562cb99c68ad1f20e4d47735ed Author: Slavi Pantaleev Date: Sat Dec 16 09:01:10 2023 +0200 Fix typo commit 98ad182eaccc7ab457ead4e03cc0d4f2a525a47d Author: Joshua Hoffmann Date: Fri Dec 15 22:37:40 2023 +0100 Add defaults for Hookshot's encryption commit 29fa9fab151f513908d3e45882003da107a63c93 Author: Joshua Hoffmann Date: Fri Dec 15 22:35:11 2023 +0100 Improve wording of Hookshot's encryption section commit 4f835e0560012754d7ce0b56619a97c68a297992 Author: Joshua Hoffmann Date: Fri Dec 15 22:28:52 2023 +0100 use safer mount options for the container's files commit 8c93327e25c5e6af2442c676d5f264d4051c80e2 Author: Joshua Hoffmann Date: Fri Dec 15 22:26:01 2023 +0100 fix filename commit 03a7bb6e7798dd95f7894311e4dd34dfa09f70fa Merge: e55d7694 06047763 Author: Joshua Hoffmann Date: Fri Dec 15 22:23:44 2023 +0100 Merge branch 'HarHarLinks/hookshot-encryption' of https://github.com/real-joshua/matrix-docker-ansible-deploy into HarHarLinks/hookshot-encryption commit 06047763bbd427dde117c6635ac7301198571158 Author: Joshua Hoffmann Date: Fri Dec 15 22:15:54 2023 +0100 Update roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 change the if statement to not require a variable with a length > 0 and add a filter to json for the redis host Co-authored-by: Slavi Pantaleev commit e55d769465bd299081464e68d34851729d42d5ff Author: Joshua Hoffmann Date: Fri Dec 15 22:13:50 2023 +0100 clarify that Redis is required, standardadise on Hookshot with an upper-case first letter for consistency commit 66706e4535704deba63e5aa2102f324f9b14dae3 Author: Joshua Hoffmann Date: Fri Dec 15 22:08:20 2023 +0100 Update roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 fix for a typo Co-authored-by: Slavi Pantaleev commit f6aaeb9a16f84409d31188e696885afed89fbdcc Merge: e5d34002 869dd33f Author: Joshua Hoffmann Date: Fri Dec 15 00:22:34 2023 +0100 Merge branch 'master' into HarHarLinks/hookshot-encryption commit e5d34002fd6c39c3e18df04e20d0c8b25475db78 Author: Joshua Hoffmann Date: Fri Dec 15 00:09:27 2023 +0100 Add Jinja loop to allow adding multiple networks commit 69f947782d6e072c3edc4ee4c7d5ccf69eccf3fd Author: Joshua Hoffmann Date: Thu Dec 14 23:52:41 2023 +0100 split if statements for the message queue and experimental encryption support into seperate statements commit 4c13be1c89ffb1b06475c7da546f7956e67b36a1 Author: Joshua Hoffmann Date: Thu Dec 14 23:31:19 2023 +0100 change variable name per spantaleev's suggestion (https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2979#discussion_r1379015551) commit 9905309aa9448f91297f8c7618bf62682eee2af7 Author: HarHarLinks Date: Wed Nov 1 16:14:04 2023 +0100 amend docs commit 94abf2d5bde63919c6b5597f3142eea5fed73815 Author: HarHarLinks Date: Wed Nov 1 16:05:22 2023 +0100 draft encryption support for hookshot --- docs/configuring-playbook-bridge-hookshot.md | 5 +++++ group_vars/matrix_servers | 12 ++++++++++- .../matrix-bridge-hookshot/defaults/main.yml | 18 ++++++++++++++++- .../matrix-bridge-hookshot/tasks/main.yml | 6 ++++++ .../tasks/reset_encryption.yml | 14 +++++++++++++ .../tasks/validate_config.yml | 6 ++++++ .../templates/config.yml.j2 | 10 ++++++++++ .../templates/registration.yml.j2 | 6 ++++++ .../systemd/matrix-hookshot.service.j2 | 20 ++++++++++++------- 9 files changed, 88 insertions(+), 9 deletions(-) create mode 100644 roles/custom/matrix-bridge-hookshot/tasks/reset_encryption.yml diff --git a/docs/configuring-playbook-bridge-hookshot.md b/docs/configuring-playbook-bridge-hookshot.md index 48a66a8ea..e3f04c5cc 100644 --- a/docs/configuring-playbook-bridge-hookshot.md +++ b/docs/configuring-playbook-bridge-hookshot.md @@ -23,6 +23,11 @@ Other configuration options are available via the `matrix_hookshot_configuration Finally, run the playbook (see [installing](installing.md)). +### End-to-bridge encryption + +You can enable [experimental encryption](https://matrix-org.github.io/matrix-hookshot/latest/advanced/encryption.html) for Hookshot by adding `matrix_hookshot_experimental_encryption_enabled: true` to your configuration (`vars.yml`) and [executing the playbook](installing.md) again. + +Should the crypto store be corrupted, you can reset it by executing this Ansible playbook with the tag `reset-hookshot-encryption` added, for example `ansible-playbook -i inventory/hosts setup.yml -K --tags=reset-hookshot-encryption`). ## Usage diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 5ccc8e823..957a15339 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -1385,6 +1385,16 @@ matrix_hookshot_systemd_wanted_services_list: | (['matrix-' + matrix_homeserver_implementation + '.service']) + (['matrix-nginx-proxy.service'] if matrix_nginx_proxy_enabled else []) + + + ([(redis_identifier + '.service')] if redis_enabled and matrix_hookshot_queue_host == redis_identifier else []) + }} + +# Hookshot's experimental encryption feature (and possibly others) may benefit from Redis, if available. +matrix_hookshot_queue_host: "{{ redis_identifier if redis_enabled else '' }}" + +matrix_hookshot_container_additional_networks_auto: | + {{ + ([redis_container_network] if redis_enabled and matrix_hookshot_queue_host == redis_identifier else []) }} matrix_hookshot_container_http_host_bind_ports_defaultmapping: @@ -3359,7 +3369,7 @@ ntfy_visitor_request_limit_exempt_hosts_hostnames_auto: | # ###################################################################### -redis_enabled: "{{ matrix_synapse_workers_enabled }}" +redis_enabled: "{{ matrix_synapse_workers_enabled or (matrix_hookshot_enabled and matrix_hookshot_experimental_encryption_enabled) }}" redis_identifier: matrix-redis diff --git a/roles/custom/matrix-bridge-hookshot/defaults/main.yml b/roles/custom/matrix-bridge-hookshot/defaults/main.yml index 1cbe16388..03f0e9530 100644 --- a/roles/custom/matrix-bridge-hookshot/defaults/main.yml +++ b/roles/custom/matrix-bridge-hookshot/defaults/main.yml @@ -10,6 +10,11 @@ matrix_hookshot_container_image_self_build: false matrix_hookshot_container_image_self_build_repo: "https://github.com/matrix-org/matrix-hookshot.git" matrix_hookshot_container_image_self_build_branch: "{{ 'main' if matrix_hookshot_version == 'latest' else matrix_hookshot_version }}" +# Specifies additional networks for the Hookshot container to connect with +matrix_hookshot_container_additional_networks: "{{ matrix_hookshot_container_additional_networks_auto + matrix_hookshot_container_additional_networks_custom }}" +matrix_hookshot_container_additional_networks_auto: [] +matrix_hookshot_container_additional_networks_custom: [] + # renovate: datasource=docker depName=halfshot/matrix-hookshot matrix_hookshot_version: 4.7.0 @@ -30,6 +35,17 @@ matrix_hookshot_public_endpoint: /hookshot matrix_hookshot_appservice_port: 9993 matrix_hookshot_appservice_endpoint: "{{ matrix_hookshot_public_endpoint }}/_matrix/app" +# The variables below control the queue parameters and may optionally be pointed to a Redis instance. +# These are required when experimental encryption is enabled (`matrix_hookshot_experimental_encryption_enabled`). +matrix_hookshot_queue_host: '' +matrix_hookshot_queue_port: 6739 + +# Controls whether the experimental end-to-bridge encryption support is enabled. +# This requires that: +# - support to also be enabled in the homeserver, see the documentation of Hookshot. +# - Hookshot to be pointed at a Redis instance via the `matrix_hookshot_queue_*` variables. +matrix_hookshot_experimental_encryption_enabled: false + # Controls whether metrics are enabled in the bridge configuration. # Enabling them is usually enough for a local (in-container) Prometheus to consume them. # If metrics need to be consumed by another (external) Prometheus server, consider exposing them via `matrix_hookshot_metrics_proxying_enabled`. @@ -41,7 +57,7 @@ matrix_hookshot_metrics_enabled: false matrix_hookshot_metrics_proxying_enabled: false # There is no need to edit ports. -# Read the documentation to learn about using hookshot metrics with external Prometheus +# Read the documentation to learn about using Hookshot metrics with external Prometheus # If you still want something different, use matrix_hookshot_container_http_host_bind_ports below to expose ports instead. matrix_hookshot_metrics_port: 9001 diff --git a/roles/custom/matrix-bridge-hookshot/tasks/main.yml b/roles/custom/matrix-bridge-hookshot/tasks/main.yml index e2fa9936a..44ad5229b 100644 --- a/roles/custom/matrix-bridge-hookshot/tasks/main.yml +++ b/roles/custom/matrix-bridge-hookshot/tasks/main.yml @@ -9,6 +9,12 @@ - when: matrix_hookshot_enabled | bool ansible.builtin.include_tasks: "{{ role_path }}/tasks/inject_into_nginx_proxy.yml" +- tags: + - reset-hookshot-encryption + block: + - when: matrix_hookshot_enabled | bool + ansible.builtin.include_tasks: "{{ role_path }}/tasks/reset_encryption.yml" + - tags: - setup-all - setup-hookshot diff --git a/roles/custom/matrix-bridge-hookshot/tasks/reset_encryption.yml b/roles/custom/matrix-bridge-hookshot/tasks/reset_encryption.yml new file mode 100644 index 000000000..8521c3483 --- /dev/null +++ b/roles/custom/matrix-bridge-hookshot/tasks/reset_encryption.yml @@ -0,0 +1,14 @@ +--- + +- name: Resetting Hookshot's crypto store + ansible.builtin.command: + cmd: | + {{ devture_systemd_docker_base_host_command_docker }} run + --rm + --name={{ matrix_hookshot_container_ident }}-reset-crypto + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --mount type=bind,src={{ matrix_hookshot_base_path }}/config.yml,dst=/config.yml + {{ matrix_hookshot_docker_image }} + yarn start:resetcrypto + changed_when: true diff --git a/roles/custom/matrix-bridge-hookshot/tasks/validate_config.yml b/roles/custom/matrix-bridge-hookshot/tasks/validate_config.yml index 91d29eceb..f02fb1a0e 100644 --- a/roles/custom/matrix-bridge-hookshot/tasks/validate_config.yml +++ b/roles/custom/matrix-bridge-hookshot/tasks/validate_config.yml @@ -87,6 +87,12 @@ with_items: - "matrix_hookshot_provisioning_secret" +- name: Fail if no Redis queue enabled when Hookshot encryption is enabled + ansible.builtin.fail: + msg: >- + You need to define a required configuration setting (`{{ item }}`) to enable Hookshot encryption. + when: "matrix_hookshot_experimental_encryption_enabled and matrix_hookshot_queue_host == ''" + - name: (Deprecation) Catch and report old metrics usage ansible.builtin.fail: msg: >- diff --git a/roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 b/roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 index c3b0bbd48..77036b52c 100644 --- a/roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 +++ b/roles/custom/matrix-bridge-hookshot/templates/config.yml.j2 @@ -107,6 +107,16 @@ metrics: # (Optional) Prometheus metrics support # enabled: {{ matrix_hookshot_metrics_enabled | to_json }} +{% if matrix_hookshot_queue_host != '' %} +queue: + monolithic: true + port: {{ matrix_hookshot_queue_port }} + host: {{ matrix_hookshot_queue_host | to_json }} +{% endif %} +{% if matrix_hookshot_experimental_encryption_enabled %} +experimentalEncryption: + storagePath: /data/encryption +{% endif %} logging: # (Optional) Logging settings. You can have a severity debug,info,warn,error # diff --git a/roles/custom/matrix-bridge-hookshot/templates/registration.yml.j2 b/roles/custom/matrix-bridge-hookshot/templates/registration.yml.j2 index 87509a127..557bd85d6 100644 --- a/roles/custom/matrix-bridge-hookshot/templates/registration.yml.j2 +++ b/roles/custom/matrix-bridge-hookshot/templates/registration.yml.j2 @@ -28,3 +28,9 @@ namespaces: sender_localpart: hookshot url: "http://{{ matrix_hookshot_container_url }}:{{ matrix_hookshot_appservice_port }}" # This should match the bridge.port in your config file rate_limited: false + +{% if matrix_hookshot_experimental_encryption_enabled %} +de.sorunome.msc2409.push_ephemeral: true +push_ephemeral: true +org.matrix.msc3202: true +{% endif %} diff --git a/roles/custom/matrix-bridge-hookshot/templates/systemd/matrix-hookshot.service.j2 b/roles/custom/matrix-bridge-hookshot/templates/systemd/matrix-hookshot.service.j2 index 240598018..ce6cab46c 100644 --- a/roles/custom/matrix-bridge-hookshot/templates/systemd/matrix-hookshot.service.j2 +++ b/roles/custom/matrix-bridge-hookshot/templates/systemd/matrix-hookshot.service.j2 @@ -13,10 +13,9 @@ DefaultDependencies=no [Service] Type=simple Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}" -ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} {{ matrix_hookshot_container_url }} -ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_hookshot_container_url }} - -ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name {{ matrix_hookshot_container_url }} \ +ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} {{ matrix_hookshot_container_ident }} +ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_hookshot_container_ident }} +ExecStartPre={{ devture_systemd_docker_base_host_command_docker }} create --rm --name {{ matrix_hookshot_container_ident }} \ --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ @@ -30,11 +29,18 @@ ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name {% endfor %} {{ matrix_hookshot_docker_image }} -ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} {{ matrix_hookshot_container_url }} -ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_hookshot_container_url }} +{% for network in matrix_hookshot_container_additional_networks %} +ExecStartPre={{ devture_systemd_docker_base_host_command_docker }} network connect {{ network }} {{ matrix_hookshot_container_ident }} +{% endfor %} + +ExecStart={{ devture_systemd_docker_base_host_command_docker }} start --attach {{ matrix_hookshot_container_ident }} + +ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} {{ matrix_hookshot_container_ident }} +ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} rm {{ matrix_hookshot_container_ident }} + Restart=always RestartSec=30 -SyslogIdentifier={{ matrix_hookshot_container_url }} +SyslogIdentifier={{ matrix_hookshot_container_ident }} [Install] WantedBy=multi-user.target