From 2e16257e50c6057dacf7ab5babe2b9885a2c98a2 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 12 Jun 2019 14:51:10 +0300 Subject: [PATCH 01/15] Do not ask for _matrix._tcp SRV records anymore With most people on Synapse v0.99+ and Synapse v1.0 now available, we should no longer try to be backward compatible with Synapse 0.34, because this just complicates the instructions for no good reason. --- CHANGELOG.md | 13 ++++++++++ docs/configuring-dns.md | 29 ++-------------------- docs/configuring-well-known.md | 7 +----- roles/matrix-base/tasks/self_check_dns.yml | 9 ++----- 4 files changed, 18 insertions(+), 40 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 16b2d25ff..ed443aa80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,16 @@ +# 2019-06-12 + +## Synapse v1.0 + +With [Synapse v1.0 now available](https://matrix.org/blog/2019/06/11/introducing-matrix-1-0-and-the-matrix-org-foundation) and most people being on at least Synapse v0.99, it's time to remove the `_matrix._tcp` DNS SRV record that we've been keeping for compatibility with old Synapse versions (<= 0.34). + +According to the [Server Discovery specification](https://matrix.org/docs/spec/server_server/r0.1.2.html#server-discovery), it's no harm to keep the DNS SRV record. But since it's not necessary for federating with the larger Matrix network anymore, you should be safe to get rid of it. + +**Note**: don't confuse the `_matrix._tcp` and `_matrix-identity._tcp` DNS SRV records. The latter, **must not** be removed. + +Besides this optional/non-urgent DNS change, assuming you're already on Synapse v0.99, upgrading to Synapse v1.0 should be as simple as [re-running the playbook](docs/maintenance-upgrading-services.md). + + # 2019-06-07 ## (BC Break) Facebook bridge configuration is now entirely managed by the playbook diff --git a/docs/configuring-dns.md b/docs/configuring-dns.md index 7638033be..678c1679e 100644 --- a/docs/configuring-dns.md +++ b/docs/configuring-dns.md @@ -9,16 +9,12 @@ You do, however need to instruct the Matrix network that Matrix services for ``. As we discuss in [Server Delegation](howto-server-delegation.md), there are 2 different ways to set up such delegation: -- using a `https:///.well-known/matrix/server` file (on the base domain!) -- using DNS SRV records +- either by serving a `https:///.well-known/matrix/server` file (from the base domain!) +- or by using a `_matrix._tcp` DNS SRV record (don't confuse this with the `_matrix-identity._tcp` SRV record described below) This playbook mostly discusses the well-known file method, because it's easier to manage with regard to certificates. If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do. -To make matters worse, for backward compatibility until the Synapse server reaches v1.0, you need to set up a DNS SRV record anyway. -So don't be confused if the general flow of this playbook asks you for both DNS SRV records and for setting up a well-known file. -It's a temporary requirement during the Synapse v0.99/v1.0 transition. - ## General outline of DNS settings you need to do @@ -27,12 +23,8 @@ It's a temporary requirement during the Synapse v0.99/v1.0 transition. | A | `matrix` | - | - | - | `matrix-server-IP` | | CNAME | `riot` | - | - | - | `matrix.` | | CNAME | `dimension` | - | - | - | `matrix.` | -| SRV | `_matrix._tcp` | 10 | 0 | 8448 | `matrix.` | | SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.` | -The `_matrix._tcp` SRV record is a temporary measure and will not be necessary in the near future. -In fact, it will have to be removed at some point. To learn more about that, read below. - ## Subdomains setup @@ -44,23 +36,6 @@ If you'd rather instruct the playbook not to install Riot (`matrix_riot_web_enab The `dimension.` subdomain may be necessary, because this playbook could install the [Dimension integrations manager](http://dimension.t2bot.io/) for you. Dimension installation is disabled by default, because it's only possible to install it after the other Matrix services are working (see [Setting up Dimension](configuring-playbook-dimension.md) later). If you do not wish to set up Dimension, feel free to skip the `dimension.` DNS record. -## `_matrix._tcp` SRV record setup (temporary requirement) - -All services created by this playbook are meant to be installed on their own server (such as `matrix.`). - -To use a Matrix user identifier like `@:` while hosting services on `matrix.`, we need to instruct the Matrix network of such a delegation/redirection by means of setting up a DNS SRV record. - -The SRV record should look like this: -- Name: `_matrix._tcp` (use this text as-is) -- Content: `10 0 8448 matrix.` (replace `` with your own) - -A [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) is superseding the `_matrix._tcp` SRV record for our use case. **During the transition phase, you'll need to set up both mechanisms**. We'll instruct you how to set up the file-based mechanism after the [installation phase](installing.md) for this playbook. - -Doing delegation/redirection of Matrix services using a DNS SRV record (`_matrix._tcp`) is a **temporary measure** for our use-case, that will only be necessary before Synapse v1.0 is released. - -As more and more people upgrade to the Synapse v0.99 transitional release and just before the final Synapse v1.0 gets released, at some point in the near future **you will need to remove the `_matrix._tcp` SRV record** and leave only the [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) in place. - - ## `_matrix-identity._tcp` SRV record setup To make the [mxisd](https://github.com/kamax-io/mxisd) Identity Server (which this playbook installs for you) be authoritative for your domain name, set up one more SRV record that looks like this: diff --git a/docs/configuring-well-known.md b/docs/configuring-well-known.md index 50344c05f..a8e81c732 100644 --- a/docs/configuring-well-known.md +++ b/docs/configuring-well-known.md @@ -17,13 +17,8 @@ As [per the Server-Server specification](https://matrix.org/docs/spec/server_ser Server delegation can be configured using DNS SRV records or by setting up a `/.well-known/matrix/server` file on the base domain (``). -We have discussed the DNS SRV record method already in the "`_matrix._tcp` SRV record setup (temporary requirement)" section of [Configuring DNS](configuring-dns.md). - -Both methods have their place and will continue to do so. Usually, you would need to use just one of these delegation methods. +Both methods have their place and will continue to do so. You only need to use just one of these delegation methods. For simplicity reasons, our setup advocates for the `/.well-known/matrix/server` method and guides you into using that. -For backward compatibility with older Synapse servers (< v0.99), however, for now you are also required to set up a `_matrix._tcp` DNS SRV record (in addition to the `/.well-known/matrix/server` file on the base domain). - -As the Synapse server progresses towards v1.0, only the `/.well-known/matrix/server` file will be used by us, unless you have a more special setup necessitating a DNS SRV record. At that future moment, you would need to remove the `_matrix._tcp` SRV record because Synapse v1.0+ will do the wrong thing if a SRV record exists. To learn how to set up `/.well-known/matrix/server`, read the Installing section below. diff --git a/roles/matrix-base/tasks/self_check_dns.yml b/roles/matrix-base/tasks/self_check_dns.yml index ceab00dab..d17b2149a 100644 --- a/roles/matrix-base/tasks/self_check_dns.yml +++ b/roles/matrix-base/tasks/self_check_dns.yml @@ -1,12 +1,7 @@ --- -- name: Determine DNS SRV records to check (Matrix) - set_fact: - dns_srv_record_checks: - - service_and_protocol: "_matrix._tcp" - domain: "{{ (matrix_domain + '.') }}" - expected_target: "{{ (matrix_server_fqn_matrix + '.') }}" - expected_port: 8448 +- set_fact: + dns_srv_record_checks: [] - block: - set_fact: From 3956b300ed849f87fb31343e44a84805e0b079c1 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Fri, 14 Jun 2019 07:40:24 +0300 Subject: [PATCH 02/15] Disable riot-web's welcome bot I've not found this welcome bot to work at all in my previous attempts. It would simply not reply, even though federation works. It seems like this is also a potential privacy issue, as per https://gist.github.com/maxidorius/5736fd09c9194b7a6dc03b6b8d7220d0 --- group_vars/matrix_servers | 2 ++ roles/matrix-riot-web/templates/config.json.j2 | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index cf8d4830b..876a00fa4 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -438,6 +438,8 @@ matrix_riot_web_enable_presence_by_hs_url: | else {matrix_riot_web_default_hs_url: false} }} +matrix_riot_web_welcome_user_id: ~ + ###################################################################### # # /matrix-riot-web diff --git a/roles/matrix-riot-web/templates/config.json.j2 b/roles/matrix-riot-web/templates/config.json.j2 index 92753a15f..e8c183b4a 100644 --- a/roles/matrix-riot-web/templates/config.json.j2 +++ b/roles/matrix-riot-web/templates/config.json.j2 @@ -14,7 +14,7 @@ "roomDirectory": { "servers": {{ matrix_riot_web_roomdir_servers|to_json }} }, - "welcomeUserId": {{ matrix_riot_web_welcome_user_id|string|to_json }}, + "welcomeUserId": {{ matrix_riot_web_welcome_user_id|to_json }}, {% if matrix_riot_web_enable_presence_by_hs_url is not none %} "enable_presence_by_hs_url": {{ matrix_riot_web_enable_presence_by_hs_url|to_json }}, {% endif %} From d8a400722028868ea775fa168b14211eba847f6e Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Fri, 14 Jun 2019 08:07:54 +0300 Subject: [PATCH 03/15] Upgrade exim (4.91 -> 4.92) Note: https://www.us-cert.gov/ncas/current-activity/2019/06/13/Exim-Releases-Security-Patches That said, I don't believe we've been affected. Not in a bad way at least, because: - we run exim as non-root and capabilities dropped - we run exim in a private Docker network with known trusted relayers (Synapse and mxisd) --- roles/matrix-mailer/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/matrix-mailer/defaults/main.yml b/roles/matrix-mailer/defaults/main.yml index 6e5b34675..f40420eaa 100644 --- a/roles/matrix-mailer/defaults/main.yml +++ b/roles/matrix-mailer/defaults/main.yml @@ -2,7 +2,7 @@ matrix_mailer_enabled: true matrix_mailer_base_path: "{{ matrix_base_data_path }}/mailer" -matrix_mailer_docker_image: "devture/exim-relay:4.91-r3-0" +matrix_mailer_docker_image: "devture/exim-relay:4.92-r0-0" matrix_mailer_docker_image_force_pull: "{{ matrix_mailer_docker_image.endswith(':latest') }}" # The user/group that the container runs with. From 00383a73ac1b3eb791e8aec6e57af4855a412aed Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Fri, 14 Jun 2019 10:19:52 +0300 Subject: [PATCH 04/15] Make running --tags=setup-synapse only not fail to register bridges Until now, if `--tags=setup-synapse` was used, bridge tasks would not run and bridges would fail to register with the `matrix-synapse` role. This means that Synapse's configuration would be generated with an empty list of appservices (`app_service_config_files: []`). .. and then bridges would fail, because Synapse would not be aware of there being any bridges. From now on, bridges always run their init tasks and always register with Synapse. For the Telegram bridge, the same applies to registering with matrix-nginx-proxy. Previously, running `--tags=setup-nginx-proxy` would get rid of the Telegram endpoint configuration for the same reason. Not anymore. --- .../tasks/init.yml | 13 ++++ .../tasks/setup_install.yml | 12 ---- .../tasks/init.yml | 13 ++++ .../tasks/setup_install.yml | 12 ---- .../tasks/init.yml | 13 ++++ .../tasks/setup_install.yml | 12 ---- .../tasks/init.yml | 60 +++++++++++++++++++ .../tasks/setup_install.yml | 57 ------------------ .../tasks/init.yml | 13 ++++ .../tasks/setup_install.yml | 12 ---- 10 files changed, 112 insertions(+), 105 deletions(-) diff --git a/roles/matrix-bridge-appservice-discord/tasks/init.yml b/roles/matrix-bridge-appservice-discord/tasks/init.yml index 9d7b1939b..7370b0bde 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/init.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-discord'] }}" when: matrix_appservice_discord_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_appservice_discord_base_path }}/discord-registration.yaml,dst=/matrix-appservice-discord-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-appservice-discord-registration.yaml"] }} + when: matrix_appservice_discord_enabled|bool diff --git a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml index 0f683c84f..3dc168391 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml @@ -78,15 +78,3 @@ {{ matrix_appservice_discord_docker_image }} /bin/sh -c "node .././build/tools/addbot.js > invite_link" when: "not appservice_discord_link_generated.stat.exists" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_appservice_discord_base_path }}/discord-registration.yaml,dst=/matrix-appservice-discord-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-appservice-discord-registration.yaml"] }} diff --git a/roles/matrix-bridge-appservice-irc/tasks/init.yml b/roles/matrix-bridge-appservice-irc/tasks/init.yml index 2bd8c2a71..1ebfd0735 100644 --- a/roles/matrix-bridge-appservice-irc/tasks/init.yml +++ b/roles/matrix-bridge-appservice-irc/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-irc'] }}" when: matrix_appservice_irc_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_appservice_irc_base_path }}/registration.yaml,dst=/matrix-appservice-irc-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-appservice-irc-registration.yaml"] }} + when: matrix_appservice_irc_enabled|bool diff --git a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml index 62a81e37c..8681c4e06 100644 --- a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml @@ -72,18 +72,6 @@ -l irc_bot when: "not appservice_irc_registration_file.stat.exists" -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_appservice_irc_base_path }}/registration.yaml,dst=/matrix-appservice-irc-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-appservice-irc-registration.yaml"] }} - - name: Ensure IRC configuration directory permissions are correct file: path: "{{ matrix_appservice_irc_base_path }}" diff --git a/roles/matrix-bridge-mautrix-facebook/tasks/init.yml b/roles/matrix-bridge-mautrix-facebook/tasks/init.yml index b19c90cd9..99e8a9ec6 100644 --- a/roles/matrix-bridge-mautrix-facebook/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-facebook/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-facebook'] }}" when: matrix_mautrix_facebook_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_facebook_config_path }}/registration.yaml,dst=/matrix-mautrix-facebook-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-facebook-registration.yaml"] }} + when: matrix_mautrix_facebook_enabled|bool diff --git a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml index a69a58c99..68c823c6c 100644 --- a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml @@ -71,15 +71,3 @@ service: daemon_reload: yes when: "matrix_mautrix_facebook_systemd_service_result.changed" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_facebook_config_path }}/registration.yaml,dst=/matrix-mautrix-facebook-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-facebook-registration.yaml"] }} diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml index 27a84f1f6..14ace842c 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml @@ -1,3 +1,63 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-telegram'] }}" when: matrix_mautrix_telegram_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_base_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-telegram-registration.yaml"] }} + when: matrix_mautrix_telegram_enabled|bool + +- block: + - name: Fail if matrix-nginx-proxy role already executed + fail: + msg: >- + Trying to append Mautrix Telegram's reverse-proxying configuration to matrix-nginx-proxy, + but it's pointless since the matrix-nginx-proxy role had already executed. + To fix this, please change the order of roles in your plabook, + so that the matrix-nginx-proxy role would run after the matrix-bridge-mautrix-telegram role. + when: matrix_nginx_proxy_role_executed|default(False)|bool + + - name: Generate Mautrix Telegram proxying configuration for matrix-nginx-proxy + set_fact: + matrix_mautrix_telegram_matrix_nginx_proxy_configuration: | + location {{ matrix_mautrix_telegram_public_endpoint }} { + {% if matrix_nginx_proxy_enabled|default(False) %} + {# Use the embedded DNS resolver in Docker containers to discover the service #} + resolver 127.0.0.11 valid=5s; + set $backend "matrix-mautrix-telegram:8080"; + proxy_pass http://$backend; + {% else %} + {# Generic configuration for use outside of our container setup #} + proxy_pass http://127.0.0.1:9006; + {% endif %} + } + + - name: Register Mautrix Telegram proxying configuration with matrix-nginx-proxy + set_fact: + matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: | + {{ + matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks|default([]) + + + [matrix_mautrix_telegram_matrix_nginx_proxy_configuration] + }} + tags: + - always + when: matrix_mautrix_telegram_enabled|bool + +- name: Warn about reverse-proxying if matrix-nginx-proxy not used + debug: + msg: >- + NOTE: You've enabled the Mautrix Telegram bridge but are not using the matrix-nginx-proxy + reverse proxy. + Please make sure that you're proxying the `{{ matrix_mautrix_telegram_public_endpoint }}` + URL endpoint to the matrix-mautrix-telegram container. + You can expose the container's port using the `matrix_mautrix_telegram_container_http_host_bind_port` variable. + when: "matrix_mautrix_telegram_enabled|bool and matrix_nginx_proxy_enabled is not defined" diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml index 2ad0694a7..7a39fe137 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml @@ -78,60 +78,3 @@ python3 -m mautrix_telegram -g -c /data/config.yaml -r /data/registration.yaml when: "not mautrix_telegram_registration_file_stat.stat.exists" -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_base_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-telegram-registration.yaml"] }} - -- block: - - name: Fail if matrix-nginx-proxy role already executed - fail: - msg: >- - Trying to append Mautrix Telegram's reverse-proxying configuration to matrix-nginx-proxy, - but it's pointless since the matrix-nginx-proxy role had already executed. - To fix this, please change the order of roles in your plabook, - so that the matrix-nginx-proxy role would run after the matrix-bridge-mautrix-telegram role. - when: matrix_nginx_proxy_role_executed|default(False)|bool - - - name: Generate Mautrix Telegram proxying configuration for matrix-nginx-proxy - set_fact: - matrix_mautrix_telegram_matrix_nginx_proxy_configuration: | - location {{ matrix_mautrix_telegram_public_endpoint }} { - {% if matrix_nginx_proxy_enabled|default(False) %} - {# Use the embedded DNS resolver in Docker containers to discover the service #} - resolver 127.0.0.11 valid=5s; - set $backend "matrix-mautrix-telegram:8080"; - proxy_pass http://$backend; - {% else %} - {# Generic configuration for use outside of our container setup #} - proxy_pass http://127.0.0.1:9006; - {% endif %} - } - - - name: Register Mautrix Telegram proxying configuration with matrix-nginx-proxy - set_fact: - matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: | - {{ - matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks|default([]) - + - [matrix_mautrix_telegram_matrix_nginx_proxy_configuration] - }} - tags: - - always - -- name: Warn about reverse-proxying if matrix-nginx-proxy not used - debug: - msg: >- - NOTE: You've enabled the Mautrix Telegram bridge but are not using the matrix-nginx-proxy - reverse proxy. - Please make sure that you're proxying the `{{ matrix_mautrix_telegram_public_endpoint }}` - URL endpoint to the matrix-mautrix-telegram container. - You can expose the container's port using the `matrix_mautrix_telegram_container_http_host_bind_port` variable. - when: "matrix_nginx_proxy_enabled is not defined" diff --git a/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml b/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml index 4b67e2ba4..aff34f6dd 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-whatsapp'] }}" when: matrix_mautrix_whatsapp_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_whatsapp_base_path }}/registration.yaml,dst=/matrix-mautrix-whatsapp-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-whatsapp-registration.yaml"] }} + when: matrix_mautrix_whatsapp_enabled|bool diff --git a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml index 38e4e035d..5c176e3a1 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml @@ -66,15 +66,3 @@ {{ matrix_mautrix_whatsapp_docker_image }} /usr/bin/mautrix-whatsapp -g -c /data/config.yaml -r /data/registration.yaml when: "not mautrix_whatsapp_registration_file_stat.stat.exists" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_whatsapp_base_path }}/registration.yaml,dst=/matrix-mautrix-whatsapp-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-whatsapp-registration.yaml"] }} From 2902b53267544702adc3160b6349b45814889372 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 15 Jun 2019 09:42:40 +0300 Subject: [PATCH 05/15] Minor fixes for consistency --- group_vars/matrix_servers | 8 ++++---- .../templates/systemd/matrix-mautrix-facebook.service.j2 | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 876a00fa4..147ee5baa 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -90,10 +90,6 @@ matrix_appservice_irc_systemd_required_services_list: | # We don't enable bridges by default. matrix_mautrix_facebook_enabled: false -matrix_mautrix_facebook_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-appservice-token') | to_uuid }}" - -matrix_mautrix_facebook_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-homeserver-token') | to_uuid }}" - matrix_mautrix_facebook_systemd_required_services_list: | {{ ['docker.service'] @@ -101,6 +97,10 @@ matrix_mautrix_facebook_systemd_required_services_list: | (['matrix-synapse.service'] if matrix_synapse_enabled else []) }} +matrix_mautrix_facebook_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-appservice-token') | to_uuid }}" + +matrix_mautrix_facebook_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-homeserver-token') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-facebook diff --git a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 index 7bc4f1890..e0ac9084c 100644 --- a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 +++ b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 @@ -17,8 +17,8 @@ ExecStartPre=/usr/bin/docker run --rm --name matrix-mautrix-facebook-db \ --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ - -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ -v {{ matrix_mautrix_facebook_config_path }}:/config:z \ + -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ {{ matrix_mautrix_facebook_docker_image }} \ alembic -x config=/config/config.yaml upgrade head @@ -30,8 +30,8 @@ ExecStart=/usr/bin/docker run --rm --name matrix-mautrix-facebook \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ --network={{ matrix_docker_network }} \ - -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ -v {{ matrix_mautrix_facebook_config_path }}:/config:z \ + -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ {% for arg in matrix_mautrix_facebook_container_extra_arguments %} {{ arg }} \ {% endfor %} From 4e8543ce21b88b55952590d7a873ec3f480419d9 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 15 Jun 2019 09:43:43 +0300 Subject: [PATCH 06/15] Make Telegram bridge configuration playbook-managed --- CHANGELOG.md | 23 ++ group_vars/matrix_servers | 5 + .../defaults/main.yml | 341 ++++++++++++++++++ .../tasks/setup_install.yml | 71 ++-- .../tasks/validate_config.yml | 2 + .../templates/config.yaml.j2 | 266 -------------- .../matrix-mautrix-telegram.service.j2 | 10 +- 7 files changed, 409 insertions(+), 309 deletions(-) delete mode 100644 roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 diff --git a/CHANGELOG.md b/CHANGELOG.md index ed443aa80..3a652f1c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,26 @@ +# 2019-06-15 + +## (BC Break) Telegram bridge configuration is now entirely managed by the playbook + +Until now, configuration files for the [Telegram bridge](docs/configuring-playbook-bridge-mautrix-telegram.md) were created by the playbook initially, but never modified later on. + +From now on, the playbook will keep those configuration in sync for you. + +This means that if you were making manual changes to the `/matrix/mautrix-telegram/config.yaml` or `/matrix/mautrix-telegram/registration.yaml` configuration files, those would be lost the next time you run the playbook. + +The bridge now stores configuration in a subdirectory (`/matrix/mautrix-telegram/config`), so your old configuration remains in the base directory (`/matrix/mautrix-telegram`). +You need to migrate any manual changes over to the new `matrix_mautrix_telegram_configuration_extension_yaml` variable, so that the playbook would apply them for you. + +Likewise, data is now also stored in a subdirectory (`/matrix/mautrix-telegram/data`). When you run the playbook with an existing database file (`/matrix/mautrix-telegram/mautrix-telegram.db`), the playbook will stop the bridge and relocate the database file to the `./data` directory. There's no data-loss involved. You'll need to restart the bridge manually though (`--tags=start`). + +Also, we're now following the default configuration for the Telegram bridge, so some default configuration values are different: + +- `edits_as_replies` (used to be `false`, now `true`) - previously replies were not sent over to Matrix at all; ow they are sent over as a reply to the original message +- `inline_images` (used to be `true`, now `false`) - this has to do with captioned images. Inline-image (included caption) are said to exhibit troubles on Riot iOS. When `false`, the caption arrives on the Matrix side as a separate message. +- `authless_portals` (used to be `false`, now `true`) - creating portals from the Telegram side is now possible +- `whitelist_group_admins` (used to be `false`, now `true`) - allows Telegram group admins to use the bot commands + + # 2019-06-12 ## Synapse v1.0 diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 147ee5baa..1c16e87b4 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -117,6 +117,7 @@ matrix_mautrix_facebook_homeserver_token: "{{ matrix_synapse_macaroon_secret_key # We don't enable bridges by default. matrix_mautrix_telegram_enabled: false + matrix_mautrix_telegram_systemd_required_services_list: | {{ ['docker.service'] @@ -124,6 +125,10 @@ matrix_mautrix_telegram_systemd_required_services_list: | (['matrix-synapse.service'] if matrix_synapse_enabled else []) }} +matrix_mautrix_telegram_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram-appservice-token') | to_uuid }}" + +matrix_mautrix_telegram_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram-homeserver-token') | to_uuid }}" + matrix_mautrix_telegram_public_endpoint: "/{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram') | to_uuid }}" matrix_mautrix_telegram_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9006' }}" diff --git a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml index f06f0a76b..09aa35cf3 100644 --- a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml @@ -7,6 +7,8 @@ matrix_mautrix_telegram_docker_image: "tulir/mautrix-telegram:v0.5.2" matrix_mautrix_telegram_docker_image_force_pull: "{{ matrix_mautrix_telegram_docker_image.endswith(':latest') }}" matrix_mautrix_telegram_base_path: "{{ matrix_base_data_path }}/mautrix-telegram" +matrix_mautrix_telegram_config_path: "{{ matrix_mautrix_telegram_base_path }}/config" +matrix_mautrix_telegram_data_path: "{{ matrix_mautrix_telegram_base_path }}/data" # Get your own API keys at https://my.telegram.org/apps matrix_mautrix_telegram_api_id: '' @@ -35,3 +37,342 @@ matrix_mautrix_telegram_systemd_required_services_list: ['docker.service'] # List of systemd services that matrix-mautrix-telegram.service wants matrix_mautrix_telegram_systemd_wanted_services_list: [] + +matrix_mautrix_telegram_appservice_token: '' +matrix_mautrix_telegram_homeserver_token: '' + +# Default mxisd configuration template which covers the generic use case. +# You can customize it by controlling the various variables inside it. +# +# For a more advanced customization, you can extend the default (see `matrix_mautrix_telegram_configuration_extension_yaml`) +# or completely replace this variable with your own template. +matrix_mautrix_telegram_configuration_yaml: | + #jinja2: lstrip_blocks: "True" + # Homeserver details + homeserver: + # The address that this appservice can use to connect to the homeserver. + address: {{ matrix_mautrix_telegram_homeserver_address }} + # The domain of the homeserver (for MXIDs, etc). + domain: {{ matrix_mautrix_telegram_homeserver_domain }} + # Whether or not to verify the SSL certificate of the homeserver. + # Only applies if address starts with https:// + verify_ssl: true + + # Application service host/registration related details + # Changing these values requires regeneration of the registration. + appservice: + # The address that the homeserver can use to connect to this appservice. + address: {{ matrix_mautrix_telegram_appservice_address }} + + # The hostname and port where this appservice should listen. + hostname: 0.0.0.0 + port: 8080 + # The maximum body size of appservice API requests (from the homeserver) in mebibytes + # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s + max_body_size: 1 + + # The full URI to the database. SQLite and Postgres are fully supported. + # Other DBMSes supported by SQLAlchemy may or may not work. + # Format examples: + # SQLite: sqlite:///filename.db + # Postgres: postgres://username:password@hostname/dbname + database: sqlite:////data/mautrix-telegram.db + + # Public part of web server for out-of-Matrix interaction with the bridge. + # Used for things like login if the user wants to make sure the 2FA password isn't stored in + # the HS database. + public: + # Whether or not the public-facing endpoints should be enabled. + enabled: true + # The prefix to use in the public-facing endpoints. + prefix: {{ matrix_mautrix_telegram_public_endpoint }} + # The base URL where the public-facing endpoints are available. The prefix is not added + # implicitly. + external: {{ matrix_mautrix_telegram_appservice_public_external }} + + # Provisioning API part of the web server for automated portal creation and fetching information. + # Used by things like Dimension (https://dimension.t2bot.io/). + provisioning: + # Whether or not the provisioning API should be enabled. + enabled: false + # The prefix to use in the provisioning API endpoints. + prefix: /_matrix/provision/v1 + # The shared secret to authorize users of the API. + # Set to "generate" to generate and save a new token. + shared_secret: generate + + # The unique ID of this appservice. + id: telegram + # Username of the appservice bot. + bot_username: telegrambot + # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty + # to leave display name/avatar as-is. + bot_displayname: Telegram bridge bot + bot_avatar: mxc://maunium.net/tJCRmUyJDsgRNgqhOgoiHWbX + + # Authentication tokens for AS <-> HS communication. + as_token: "{{ matrix_mautrix_telegram_appservice_token }}" + hs_token: "{{ matrix_mautrix_telegram_homeserver_token }}" + + # Bridge config + bridge: + # Localpart template of MXIDs for Telegram users. + # {userid} is replaced with the user ID of the Telegram user. + username_template: "telegram_{userid}" + # Localpart template of room aliases for Telegram portal rooms. + # {groupname} is replaced with the name part of the public channel/group invite link ( https://t.me/{} ) + alias_template: "telegram_{groupname}" + # Displayname template for Telegram users. + # {displayname} is replaced with the display name of the Telegram user. + displayname_template: "{displayname} (Telegram)" + + # Set the preferred order of user identifiers which to use in the Matrix puppet display name. + # In the (hopefully unlikely) scenario that none of the given keys are found, the numeric user + # ID is used. + # + # If the bridge is working properly, a phone number or an username should always be known, but + # the other one can very well be empty. + # + # Valid keys: + # "full name" (First and/or last name) + # "full name reversed" (Last and/or first name) + # "first name" + # "last name" + # "username" + # "phone number" + displayname_preference: + - full name + - username + - phone number + + # Maximum number of members to sync per portal when starting up. Other members will be + # synced when they send messages. The maximum is 10000, after which the Telegram server + # will not send any more members. + # Defaults to no local limit (-> limited to 10000 by server) + max_initial_member_sync: -1 + # Whether or not to sync the member list in channels. + # If no channel admins have logged into the bridge, the bridge won't be able to sync the member + # list regardless of this setting. + sync_channel_members: true + # Whether or not to skip deleted members when syncing members. + skip_deleted_members: true + # Whether or not to automatically synchronize contacts and chats of Matrix users logged into + # their Telegram account at startup. + startup_sync: true + # Number of most recently active dialogs to check when syncing chats. + # Dialogs include groups and private chats, but only groups are synced. + # Set to 0 to remove limit. + sync_dialog_limit: 30 + # The maximum number of simultaneous Telegram deletions to handle. + # A large number of simultaneous redactions could put strain on your homeserver. + max_telegram_delete: 10 + # Whether or not to automatically sync the Matrix room state (mostly unpuppeted displaynames) + # at startup and when creating a bridge. + sync_matrix_state: true + # Allow logging in within Matrix. If false, the only way to log in is using the out-of-Matrix + # login website (see appservice.public config section) + allow_matrix_login: true + # Whether or not to bridge plaintext highlights. + # Only enable this if your displayname_template has some static part that the bridge can use to + # reliably identify what is a plaintext highlight. + plaintext_highlights: false + # Show message editing as a reply to the original message. + # If this is false, message edits are not shown at all, as Matrix does not support editing yet. + edits_as_replies: true + # Highlight changed/added parts in edits. Requires lxml. + highlight_edits: false + # Whether or not to make portals of publicly joinable channels/supergroups publicly joinable on Matrix. + public_portals: true + # Whether or not to fetch and handle Telegram updates at startup from the time the bridge was down. + # Currently only works for private chats and normal groups. + catch_up: false + # Whether or not to use /sync to get presence, read receipts and typing notifications when using + # your own Matrix account as the Matrix puppet for your Telegram account. + sync_with_custom_puppets: true + # Set to false to disable link previews in messages sent to Telegram. + telegram_link_preview: true + # Use inline images instead of a separate message for the caption. + # N.B. Inline images are not supported on all clients (e.g. Riot iOS). + inline_images: false + # Maximum size of image in megabytes before sending to Telegram as a document. + image_as_file_size: 10 + + # Whether to bridge Telegram bot messages as m.notices or m.texts. + bot_messages_as_notices: true + bridge_notices: + # Whether or not Matrix bot messages (type m.notice) should be bridged. + default: false + # List of user IDs for whom the previous flag is flipped. + # e.g. if bridge_notices.default is false, notices from other users will not be bridged, but + # notices from users listed here will be bridged. + exceptions: [] + + # Some config options related to Telegram message deduplication. + # The default values are usually fine, but some debug messages/warnings might recommend you + # change these. + deduplication: + # Whether or not to check the database if the message about to be sent is a duplicate. + pre_db_check: false + # The number of latest events to keep when checking for duplicates. + # You might need to increase this on high-traffic bridge instances. + cache_queue_length: 20 + + + # The formats to use when sending messages to Telegram via the relay bot. + # + # Telegram doesn't have built-in emotes, so the m.emote format is also used for non-relaybot users. + # + # Available variables: + # $sender_displayname - The display name of the sender (e.g. Example User) + # $sender_username - The username (Matrix ID localpart) of the sender (e.g. exampleuser) + # $sender_mxid - The Matrix ID of the sender (e.g. @exampleuser:example.com) + # $message - The message content as HTML + message_formats: + m.text: "$sender_displayname: $message" + m.emote: "* $sender_displayname $message" + m.file: "$sender_displayname sent a file: $message" + m.image: "$sender_displayname sent an image: $message" + m.audio: "$sender_displayname sent an audio file: $message" + m.video: "$sender_displayname sent a video: $message" + m.location: "$sender_displayname sent a location: $message" + + # The formats to use when sending state events to Telegram via the relay bot. + # + # Variables from `message_formats` that have the `sender_` prefix are available without the prefix. + # In name_change events, `$prev_displayname` is the previous displayname. + # + # Set format to an empty string to disable the messages for that event. + state_event_formats: + join: "$displayname joined the room." + leave: "$displayname left the room." + name_change: "$prev_displayname changed their name to $displayname" + + # Filter rooms that can/can't be bridged. Can also be managed using the `filter` and + # `filter-mode` management commands. + # + # Filters do not affect direct chats. + # An empty blacklist will essentially disable the filter. + filter: + # Filter mode to use. Either "blacklist" or "whitelist". + # If the mode is "blacklist", the listed chats will never be bridged. + # If the mode is "whitelist", only the listed chats can be bridged. + mode: blacklist + # The list of group/channel IDs to filter. + list: [] + + # The prefix for commands. Only required in non-management rooms. + command_prefix: "!tg" + + # Permissions for using the bridge. + # Permitted values: + # relaybot - Only use the bridge via the relaybot, no access to commands. + # user - Relaybot level + access to commands to create bridges. + # puppeting - User level + logging in with a Telegram account. + # full - Full access to use the bridge, i.e. previous levels + Matrix login. + # admin - Full access to use the bridge and some extra administration commands. + # Permitted keys: + # * - All Matrix users + # domain - All users on that homeserver + # mxid - Specific user + permissions: + '{{ matrix_mautrix_telegram_homeserver_domain }}': full + + # Options related to the message relay Telegram bot. + relaybot: + # Whether or not to allow creating portals from Telegram. + authless_portals: true + # Whether or not to allow Telegram group admins to use the bot commands. + whitelist_group_admins: true + # Whether or not to ignore incoming events sent by the relay bot. + ignore_own_incoming_events: true + # List of usernames/user IDs who are also allowed to use the bot commands. + whitelist: [] + + # Telegram config + telegram: + # Get your own API keys at https://my.telegram.org/apps + api_id: {{ matrix_mautrix_telegram_api_id }} + api_hash: {{ matrix_mautrix_telegram_api_hash }} + # (Optional) Create your own bot at https://t.me/BotFather + bot_token: disabled + # Custom server to connect to. + server: + # Set to true to use these server settings. If false, will automatically + # use production server assigned by Telegram. Set to false in production. + enabled: false + # The DC ID to connect to. + dc: 2 + # The IP to connect to. + ip: 149.154.167.40 + # The port to connect to. 443 may not work, 80 is better and both are equally secure. + port: 80 + # Telethon proxy configuration. + # You must install PySocks from pip for proxies to work. + proxy: + # Allowed types: disabled, socks4, socks5, http + type: disabled + # Proxy IP address and port. + address: 127.0.0.1 + port: 1080 + # Whether or not to perform DNS resolving remotely. + rdns: true + # Proxy authentication (optional). + username: "" + password: "" + + # Python logging configuration. + # + # See section 16.7.2 of the Python documentation for more info: + # https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema + logging: + version: 1 + formatters: + precise: + format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" + handlers: + console: + class: logging.StreamHandler + formatter: precise + loggers: + mau: + level: DEBUG + telethon: + level: DEBUG + aiohttp: + level: INFO + root: + level: DEBUG + handlers: [console] + + +matrix_mautrix_telegram_configuration_extension_yaml: | + # Your custom YAML configuration goes here. + # This configuration extends the default starting configuration (`matrix_mautrix_telegram_configuration_yaml`). + # + # You can override individual variables from the default configuration, or introduce new ones. + # + # If you need something more special, you can take full control by + # completely redefining `matrix_mautrix_telegram_configuration_yaml`. + +matrix_mautrix_telegram_configuration_extension: "{{ matrix_mautrix_telegram_configuration_extension_yaml|from_yaml if matrix_mautrix_telegram_configuration_extension_yaml|from_yaml is mapping else {} }}" + +# Holds the final configuration (a combination of the default and its extension). +# You most likely don't need to touch this variable. Instead, see `matrix_mautrix_telegram_configuration_yaml`. +matrix_mautrix_telegram_configuration: "{{ matrix_mautrix_telegram_configuration_yaml|from_yaml|combine(matrix_mautrix_telegram_configuration_extension, recursive=True) }}" + +matrix_mautrix_telegram_registration_yaml: | + id: telegram + as_token: "{{ matrix_mautrix_telegram_appservice_token }}" + hs_token: "{{ matrix_mautrix_telegram_homeserver_token }}" + namespaces: + users: + - exclusive: true + regex: '@telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain }}' + aliases: + - exclusive: true + regex: '#telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain }}' + url: {{ matrix_mautrix_telegram_appservice_address }} + sender_localpart: telegrambot + rate_limited: false + +matrix_mautrix_telegram_registration: "{{ matrix_mautrix_telegram_registration_yaml|from_yaml }}" diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml index 7a39fe137..a5f6983a8 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml @@ -15,38 +15,50 @@ force_source: "{{ matrix_mautrix_telegram_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_telegram_docker_image_force_pull }}" -- name: Ensure Mautrix Telegram base directory exists +- name: Ensure Mautrix Telegram paths exist file: - path: "{{ matrix_mautrix_telegram_base_path }}" + path: "{{ item }}" state: directory mode: 0750 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" + with_items: + - "{{ matrix_mautrix_telegram_base_path }}" + - "{{ matrix_mautrix_telegram_config_path }}" + - "{{ matrix_mautrix_telegram_data_path }}" -- name: Check if a mautrix-telegram configuration file exists +- name: Check if an old database file already exists stat: - path: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" - register: mautrix_telegram_config_file_stat + path: "{{ matrix_mautrix_telegram_base_path }}/mautrix-telegram.db" + register: matrix_mautrix_telegram_stat_database -- name: Ensure Matrix Mautrix telegram config installed - template: - src: "{{ role_path }}/templates/config.yaml.j2" - dest: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" +- name: (Data relocation) Ensure matrix-mautrix-telegram.service is stopped + service: + name: matrix-mautrix-telegram + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mautrix_telegram_stat_database.stat.exists" + +- name: (Data relocation) Move mautrix-telegram database file to ./data directory + command: "mv {{ matrix_mautrix_telegram_base_path }}/mautrix-telegram.db {{ matrix_mautrix_telegram_data_path }}/mautrix-telegram.db" + when: "matrix_mautrix_telegram_stat_database.stat.exists" + +- name: Ensure mautrix-telegram config.yaml installed + copy: + content: "{{ matrix_mautrix_telegram_configuration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_telegram_config_path }}/config.yaml" mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" - when: "not mautrix_telegram_config_file_stat.stat.exists" -- name: (Migration) Fix up old configuration - lineinfile: - path: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" - regexp: "{{ item.regexp }}" - line: "{{ item.line }}" - backrefs: yes - with_items: - - {'regexp': '^(\s+)filename: \./mautrix-telegram.log', 'line': '\1filename: /data/mautrix-telegram.log'} - - {'regexp': '^(\s+)database:', 'line': '\1database: sqlite:////data/mautrix-telegram.db'} - when: "mautrix_telegram_config_file_stat.stat.exists" +- name: Ensure mautrix-telegram registration.yaml installed + copy: + content: "{{ matrix_mautrix_telegram_registration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_telegram_config_path }}/registration.yaml" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_username }}" - name: Ensure matrix-mautrix-telegram.service installed template: @@ -59,22 +71,3 @@ service: daemon_reload: yes when: "matrix_mautrix_telegram_systemd_service_result.changed" - -- name: Check if a mautrix-telegram registration file exists - stat: - path: "{{ matrix_mautrix_telegram_base_path }}/registration.yaml" - register: mautrix_telegram_registration_file_stat - -- name: Generate matrix-mautrix-telegram registration.yaml if it doesn't exist - shell: - cmd: >- - /usr/bin/docker run - --rm - --user={{ matrix_user_uid }}:{{ matrix_user_gid }} - --cap-drop=ALL - --name matrix-mautrix-telegram-gen - -v {{ matrix_mautrix_telegram_base_path }}:/data:z - {{ matrix_mautrix_telegram_docker_image }} - python3 -m mautrix_telegram -g -c /data/config.yaml -r /data/registration.yaml - when: "not mautrix_telegram_registration_file_stat.stat.exists" - diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml b/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml index d30be92e2..5b1f3b006 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml @@ -9,6 +9,8 @@ - "matrix_mautrix_telegram_api_id" - "matrix_mautrix_telegram_api_hash" - "matrix_mautrix_telegram_public_endpoint" + - "matrix_mautrix_telegram_appservice_token" + - "matrix_mautrix_telegram_homeserver_token" - name: (Deprecation) Catch and report renamed Telegram variables fail: diff --git a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 deleted file mode 100644 index 0a8869a6a..000000000 --- a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 +++ /dev/null @@ -1,266 +0,0 @@ -#jinja2: lstrip_blocks: "True" -# Homeserver details -homeserver: - # The address that this appservice can use to connect to the homeserver. - address: {{ matrix_mautrix_telegram_homeserver_address }} - # The domain of the homeserver (for MXIDs, etc). - domain: {{ matrix_mautrix_telegram_homeserver_domain }} - # Whether or not to verify the SSL certificate of the homeserver. - # Only applies if address starts with https:// - verify_ssl: true - -# Application service host/registration related details -# Changing these values requires regeneration of the registration. -appservice: - # The address that the homeserver can use to connect to this appservice. - address: {{ matrix_mautrix_telegram_appservice_address }} - - # The hostname and port where this appservice should listen. - hostname: 0.0.0.0 - port: 8080 - # The maximum body size of appservice API requests (from the homeserver) in mebibytes - # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s - max_body_size: 1 - - # The full URI to the database. SQLite and Postgres are fully supported. - # Other DBMSes supported by SQLAlchemy may or may not work. - # Format examples: - # SQLite: sqlite:///filename.db - # Postgres: postgres://username:password@hostname/dbname - database: sqlite:////data/mautrix-telegram.db - - # Public part of web server for out-of-Matrix interaction with the bridge. - # Used for things like login if the user wants to make sure the 2FA password isn't stored in - # the HS database. - public: - # Whether or not the public-facing endpoints should be enabled. - enabled: true - # The prefix to use in the public-facing endpoints. - prefix: {{ matrix_mautrix_telegram_public_endpoint }} - # The base URL where the public-facing endpoints are available. The prefix is not added - # implicitly. - external: {{ matrix_mautrix_telegram_appservice_public_external }} - - # Provisioning API part of the web server for automated portal creation and fetching information. - # Used by things like Dimension (https://dimension.t2bot.io/). - provisioning: - # Whether or not the provisioning API should be enabled. - enabled: false - # The prefix to use in the provisioning API endpoints. - prefix: /_matrix/provision/v1 - # The shared secret to authorize users of the API. - # Set to "generate" to generate and save a new token. - shared_secret: generate - - # The unique ID of this appservice. - id: telegram - # Username of the appservice bot. - bot_username: telegrambot - # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty - # to leave display name/avatar as-is. - bot_displayname: Telegram bridge bot - bot_avatar: mxc://maunium.net/tJCRmUyJDsgRNgqhOgoiHWbX - - # Authentication tokens for AS <-> HS communication. Autogenerated; do not modify. - as_token: "This value is generated when generating the registration" - hs_token: "This value is generated when generating the registration" - -# Bridge config -bridge: - # Localpart template of MXIDs for Telegram users. - # {userid} is replaced with the user ID of the Telegram user. - username_template: "telegram_{userid}" - # Localpart template of room aliases for Telegram portal rooms. - # {groupname} is replaced with the name part of the public channel/group invite link ( https://t.me/{} ) - alias_template: "telegram_{groupname}" - # Displayname template for Telegram users. - # {displayname} is replaced with the display name of the Telegram user. - displayname_template: "{displayname} (Telegram)" - - # Set the preferred order of user identifiers which to use in the Matrix puppet display name. - # In the (hopefully unlikely) scenario that none of the given keys are found, the numeric user - # ID is used. - # - # If the bridge is working properly, a phone number or an username should always be known, but - # the other one can very well be empty. - # - # Valid keys: - # "full name" (First and/or last name) - # "full name reversed" (Last and/or first name) - # "first name" - # "last name" - # "username" - # "phone number" - displayname_preference: - - full name - - username - - phone number - - # Show message editing as a reply to the original message. - # If this is false, message edits are not shown at all, as Matrix does not support editing yet. - edits_as_replies: false - # Highlight changed/added parts in edits. Requires lxml. - highlight_edits: false - # Whether or not Matrix bot messages (type m.notice) should be bridged. - bridge_notices: true - # Whether to bridge Telegram bot messages as m.notices or m.texts. - bot_messages_as_notices: true - # Maximum number of members to sync per portal when starting up. Other members will be - # synced when they send messages. The maximum is 10000, after which the Telegram server - # will not send any more members. - # Defaults to no local limit (-> limited to 10000 by server) - max_initial_member_sync: -1 - # Whether or not to sync the member list in channels. - # If no channel admins have logged into the bridge, the bridge won't be able to sync the member - # list regardless of this setting. - sync_channel_members: true - # The maximum number of simultaneous Telegram deletions to handle. - # A large number of simultaneous redactions could put strain on your homeserver. - max_telegram_delete: 10 - # Allow logging in within Matrix. If false, the only way to log in is using the out-of-Matrix - # login website (see appservice.public config section) - allow_matrix_login: true - # Use inline images instead of m.image to make rich captions possible. - # N.B. Inline images are not supported on all clients (e.g. Riot iOS). - inline_images: true - # Whether or not to bridge plaintext highlights. - # Only enable this if your displayname_template has some static part that the bridge can use to - # reliably identify what is a plaintext highlight. - plaintext_highlights: false - # Whether or not to make portals of publicly joinable channels/supergroups publicly joinable on Matrix. - public_portals: true - # Whether to send stickers as the new native m.sticker type or normal m.images. - # Old versions of Riot don't support the new type at all. - # Remember that proper sticker support always requires Pillow to convert webp into png. - native_stickers: true - # Whether or not to fetch and handle Telegram updates at startup from the time the bridge was down. - # WARNING: Probably buggy, might get stuck in infinite loop. - catch_up: false - # Whether or not to use /sync to get presence, read receipts and typing notifications when using - # your own Matrix account as the Matrix puppet for your Telegram account. - sync_with_custom_puppets: true - - # Some config options related to Telegram message deduplication. - # The default values are usually fine, but some debug messages/warnings might recommend you - # change these. - deduplication: - # Whether or not to check the database if the message about to be sent is a duplicate. - pre_db_check: false - # The number of latest events to keep when checking for duplicates. - # You might need to increase this on high-traffic bridge instances. - cache_queue_length: 20 - - # The formats to use when sending messages to Telegram via the relay bot. - # - # Telegram doesn't have built-in emotes, so the m.emote format is also used for non-relaybot users. - # - # Available variables: - # $sender_displayname - The display name of the sender (e.g. Example User) - # $sender_username - The username (Matrix ID localpart) of the sender (e.g. exampleuser) - # $sender_mxid - The Matrix ID of the sender (e.g. @exampleuser:example.com) - # $message - The message content as HTML - message_formats: - m.text: "$sender_displayname: $message" - m.emote: "* $sender_displayname $message" - m.file: "$sender_displayname sent a file: $message" - m.image: "$sender_displayname sent an image: $message" - m.audio: "$sender_displayname sent an audio file: $message" - m.video: "$sender_displayname sent a video: $message" - m.location: "$sender_displayname sent a location: $message" - - # The formats to use when sending state events to Telegram via the relay bot. - # - # Variables from `message_formats` that have the `sender_` prefix are available without the prefix. - # In name_change events, `$prev_displayname` is the previous displayname. - # - # Set format to an empty string to disable the messages for that event. - state_event_formats: - join: "$displayname joined the room." - leave: "$displayname left the room." - name_change: "$prev_displayname changed their name to $displayname" - - # Filter rooms that can/can't be bridged. Can also be managed using the `filter` and - # `filter-mode` management commands. - # - # Filters do not affect direct chats. - # An empty blacklist will essentially disable the filter. - filter: - # Filter mode to use. Either "blacklist" or "whitelist". - # If the mode is "blacklist", the listed chats will never be bridged. - # If the mode is "whitelist", only the listed chats can be bridged. - mode: blacklist - # The list of group/channel IDs to filter. - list: [] - - # The prefix for commands. Only required in non-management rooms. - command_prefix: "!tg" - - # Permissions for using the bridge. - # Permitted values: - # relaybot - Only use the bridge via the relaybot, no access to commands. - # user - Relaybot level + access to commands to create bridges. - # puppeting - User level + logging in with a Telegram account. - # full - Full access to use the bridge, i.e. previous levels + Matrix login. - # admin - Full access to use the bridge and some extra administration commands. - # Permitted keys: - # * - All Matrix users - # domain - All users on that homeserver - # mxid - Specific user - permissions: - '{{ matrix_mautrix_telegram_homeserver_domain }}': full - - # Options related to the message relay Telegram bot. - relaybot: - # Whether or not to allow creating portals from Telegram. - authless_portals: false - # Whether or not to allow Telegram group admins to use the bot commands. - whitelist_group_admins: false - # Whether or not to ignore incoming events sent by the relay bot. - ignore_own_incoming_events: true - # List of usernames/user IDs who are also allowed to use the bot commands. - whitelist: - -# Telegram config -telegram: - # Get your own API keys at https://my.telegram.org/apps - api_id: {{ matrix_mautrix_telegram_api_id }} - api_hash: {{ matrix_mautrix_telegram_api_hash }} - # (Optional) Create your own bot at https://t.me/BotFather - bot_token: disabled - # Telethon proxy configuration. - # You must install PySocks from pip for proxies to work. - proxy: - # Allowed types: disabled, socks4, socks5, http - type: disabled - # Proxy IP address and port. - address: 127.0.0.1 - port: 1080 - # Whether or not to perform DNS resolving remotely. - rdns: true - # Proxy authentication (optional). - username: "" - password: "" - -# Python logging configuration. -# -# See section 16.7.2 of the Python documentation for more info: -# https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema -logging: - version: 1 - formatters: - precise: - format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" - handlers: - console: - class: logging.StreamHandler - formatter: precise - loggers: - mau: - level: DEBUG - telethon: - level: DEBUG - aiohttp: - level: INFO - root: - level: DEBUG - handlers: [console] diff --git a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 index 878e25854..02264236d 100644 --- a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 +++ b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 @@ -17,9 +17,10 @@ ExecStartPre=/usr/bin/docker run --rm --name matrix-mautrix-telegram-db \ --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ - -v {{ matrix_mautrix_telegram_base_path }}:/data:z \ + -v {{ matrix_mautrix_telegram_config_path }}:/config:z \ + -v {{ matrix_mautrix_telegram_data_path }}:/data:z \ {{ matrix_mautrix_telegram_docker_image }} \ - alembic -x config=/data/config.yaml upgrade head + alembic -x config=/config/config.yaml upgrade head # Intentional delay, so that the homeserver (we likely depend on) can manage to start. ExecStartPre=/bin/sleep 5 @@ -32,12 +33,13 @@ ExecStart=/usr/bin/docker run --rm --name matrix-mautrix-telegram \ {% if matrix_mautrix_telegram_container_http_host_bind_port %} -p {{ matrix_mautrix_telegram_container_http_host_bind_port }}:8080 \ {% endif %} - -v {{ matrix_mautrix_telegram_base_path }}:/data:z \ + -v {{ matrix_mautrix_telegram_config_path }}:/config:z \ + -v {{ matrix_mautrix_telegram_data_path }}:/data:z \ {% for arg in matrix_mautrix_telegram_container_extra_arguments %} {{ arg }} \ {% endfor %} {{ matrix_mautrix_telegram_docker_image }} \ - python3 -m mautrix_telegram -c /data/config.yaml + python3 -m mautrix_telegram -c /config/config.yaml ExecStop=-/usr/bin/docker kill matrix-mautrix-telegram ExecStop=-/usr/bin/docker rm matrix-mautrix-telegram From 2a2e7a7f6c3a38c3fc976f4ca919cb3ef696ce06 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 15 Jun 2019 09:53:01 +0300 Subject: [PATCH 07/15] Minor changelog clarification --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a652f1c3..612ea127c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,8 @@ Also, we're now following the default configuration for the Telegram bridge, so - `authless_portals` (used to be `false`, now `true`) - creating portals from the Telegram side is now possible - `whitelist_group_admins` (used to be `false`, now `true`) - allows Telegram group admins to use the bot commands +If the new values are not to your liking, use `matrix_mautrix_telegram_configuration_extension_yaml` to specify an override (refer to `matrix_mautrix_telegram_configuration_yaml` to figure out which variable goes where). + # 2019-06-12 From 169b09f0edc21a42e22f23014ff0e14dfdb93ddc Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 15 Jun 2019 12:01:43 +0300 Subject: [PATCH 08/15] Fix token mismatch error for the Telegram bridge Regression since 4e8543ce21 --- roles/matrix-bridge-mautrix-telegram/tasks/init.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml index 14ace842c..1898b0993 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml @@ -7,7 +7,7 @@ matrix_synapse_container_extra_arguments: > {{ matrix_synapse_container_extra_arguments|default([]) }} + - {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_base_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} + {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_config_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} matrix_synapse_app_service_config_files: > {{ matrix_synapse_app_service_config_files|default([]) }} From 9b97a42ffb53adef0f8c8df657f116f802ef2a8b Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 15 Jun 2019 16:14:14 +0300 Subject: [PATCH 09/15] Add a note about DNS SRV records not being obsolete --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 612ea127c..58466aa8b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,8 @@ According to the [Server Discovery specification](https://matrix.org/docs/spec/s **Note**: don't confuse the `_matrix._tcp` and `_matrix-identity._tcp` DNS SRV records. The latter, **must not** be removed. +For completeness, we must say that using a `_matrix._tcp` [SRV record for Server Delegation](docs/howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced) is still valid and useful for certain deployments. It's just that our guide recommends the [`/.well-known/matrix/server` Server Delegation method](docs/howto-server-delegation.md#server-delegation-via-a-well-known-file), due to its easier implementation when using this playbook. + Besides this optional/non-urgent DNS change, assuming you're already on Synapse v0.99, upgrading to Synapse v1.0 should be as simple as [re-running the playbook](docs/maintenance-upgrading-services.md). From 6b023d09d4c3e863934ca44dbbe333beecfd99a7 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 09:21:13 +0300 Subject: [PATCH 10/15] Use container network address for communication between IRC bridge and homeserver This means we need to explicitly specify a `media_url` now, because without it, `url` would be used for building public URLs to files/images. That doesn't work when `url` is not a public URL. --- roles/matrix-bridge-appservice-irc/defaults/main.yml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/roles/matrix-bridge-appservice-irc/defaults/main.yml b/roles/matrix-bridge-appservice-irc/defaults/main.yml index 9f05fc4e3..d4177a4a6 100644 --- a/roles/matrix-bridge-appservice-irc/defaults/main.yml +++ b/roles/matrix-bridge-appservice-irc/defaults/main.yml @@ -8,6 +8,10 @@ matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_ matrix_appservice_irc_base_path: "{{ matrix_base_data_path }}/appservice-irc" +matrix_appservice_irc_homeserver_url: 'http://matrix-synapse:8008' +matrix_appservice_irc_homeserver_media_url: 'https://{{ matrix_server_fqn_matrix }}' +matrix_appservice_irc_homeserver_domain: '{{ matrix_domain }}' + # Controls whether the matrix-appservice-discord container exposes its HTTP port (tcp/9999 in the container). # # Takes an ":" or "" value (e.g. "127.0.0.1:9999"), or empty string to not expose. @@ -25,8 +29,9 @@ matrix_appservice_irc_systemd_wanted_services_list: [] matrix_appservice_irc_configuration_yaml: | #jinja2: lstrip_blocks: True homeserver: - url: "https://{{ matrix_server_fqn_matrix }}" - domain: "{{ matrix_domain }}" + url: {{ matrix_appservice_irc_homeserver_url }} + media_url: {{ matrix_appservice_irc_homeserver_media_url }} + domain: {{ matrix_appservice_irc_homeserver_domain }} enablePresence: true matrix_appservice_irc_configuration_extension_yaml: | From f994e40bb7941f3c097ef7c0918e84e421d01217 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 09:28:41 +0300 Subject: [PATCH 11/15] Extend IRC bridge configuration with some additional options --- .../defaults/main.yml | 38 ++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/roles/matrix-bridge-appservice-irc/defaults/main.yml b/roles/matrix-bridge-appservice-irc/defaults/main.yml index d4177a4a6..a29f25a69 100644 --- a/roles/matrix-bridge-appservice-irc/defaults/main.yml +++ b/roles/matrix-bridge-appservice-irc/defaults/main.yml @@ -11,6 +11,7 @@ matrix_appservice_irc_base_path: "{{ matrix_base_data_path }}/appservice-irc" matrix_appservice_irc_homeserver_url: 'http://matrix-synapse:8008' matrix_appservice_irc_homeserver_media_url: 'https://{{ matrix_server_fqn_matrix }}' matrix_appservice_irc_homeserver_domain: '{{ matrix_domain }}' +matrix_appservice_irc_homeserver_enablePresence: true # Controls whether the matrix-appservice-discord container exposes its HTTP port (tcp/9999 in the container). # @@ -29,10 +30,45 @@ matrix_appservice_irc_systemd_wanted_services_list: [] matrix_appservice_irc_configuration_yaml: | #jinja2: lstrip_blocks: True homeserver: + # The URL to the home server for client-server API calls, also used to form the + # media URLs as displayed in bridged IRC channels: url: {{ matrix_appservice_irc_homeserver_url }} + # + # The URL of the homeserver hosting media files. This is only used to transform + # mxc URIs to http URIs when bridging m.room.[file|image] events. Optional. By + # default, this is the homeserver URL, specified above. + # media_url: {{ matrix_appservice_irc_homeserver_media_url }} + + # Drop Matrix messages which are older than this number of seconds, according to + # the event's origin_server_ts. + # If the bridge is down for a while, the homeserver will attempt to send all missed + # events on reconnection. These events may be hours old, which can be confusing to + # IRC users if they are then bridged. This option allows these old messages to be + # dropped. + # CAUTION: This is a very coarse heuristic. Federated homeservers may have different + # clock times and hence produce different origin_server_ts values, which may be old + # enough to cause *all* events from the homeserver to be dropped. + # Default: 0 (don't ever drop) + # dropMatrixMessagesAfterSecs: 300 # 5 minutes + + # The 'domain' part for user IDs on this home server. Usually (but not always) + # is the "domain name" part of the HS URL. domain: {{ matrix_appservice_irc_homeserver_domain }} - enablePresence: true + + # Should presence be enabled for matrix clients on this bridge. If disabled on the + # homeserver then it should also be disabled here to avoid excess traffic. + # Default: true + enablePresence: {{ matrix_appservice_irc_homeserver_enablePresence|to_json }} + + # Options here are generally only applicable to large-scale bridges and may have + # consequences greater than other options in this configuration file. + advanced: + # The maximum number of HTTP(S) sockets to maintain. Usually this is unlimited + # however for large bridges it is important to rate limit the bridge to avoid + # accidentally overloading the homeserver. Defaults to 1000, which should be + # enough for the vast majority of use cases. + maxHttpSockets: 1000 matrix_appservice_irc_configuration_extension_yaml: | # Your custom YAML configuration for Appservice IRC servers goes here. From deeb5a96d5c70c204916c508d6262e7da32186a5 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 09:30:59 +0300 Subject: [PATCH 12/15] Disable IRC bridge presence if Synapse presence is disabled --- group_vars/matrix_servers | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 1c16e87b4..aa558e71d 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -67,6 +67,10 @@ matrix_appservice_irc_enabled: false # matrix-appservice-irc's client-server port to the local host. matrix_appservice_irc_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9999' }}" +# The IRC bridge docs say that if homeserver presence is disabled, it's better to also disable +# IRC bridge presence, for performance reasons. +matrix_appservice_irc_homeserver_enablePresence: "{{ matrix_synapse_use_presence }}" + matrix_appservice_irc_systemd_required_services_list: | {{ ['docker.service'] From 380714d290d1d4f5599bd9fe06e729830484a846 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 10:10:17 +0300 Subject: [PATCH 13/15] Talk to Telegram bridge over container network --- roles/matrix-bridge-mautrix-telegram/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml index 09aa35cf3..ca3738074 100644 --- a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml @@ -19,7 +19,7 @@ matrix_mautrix_telegram_api_hash: '' # Example: /741a0483-ba17-4682-9900-30bd7269f1cc matrix_mautrix_telegram_public_endpoint: '' -matrix_mautrix_telegram_homeserver_address: 'https://{{ matrix_server_fqn_matrix }}' +matrix_mautrix_telegram_homeserver_address: 'http://matrix-synapse:8008' matrix_mautrix_telegram_homeserver_domain: '{{ matrix_domain }}' matrix_mautrix_telegram_appservice_address: 'http://matrix-mautrix-telegram:8080' matrix_mautrix_telegram_appservice_public_external: 'https://{{ matrix_server_fqn_matrix }}{{ matrix_mautrix_telegram_public_endpoint }}' From 5002c7edaa3efd19e240aa5fce9e24730246beae Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 10:30:04 +0300 Subject: [PATCH 14/15] Fix broken docs link --- docs/maintenance-and-troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/maintenance-and-troubleshooting.md b/docs/maintenance-and-troubleshooting.md index ad0babc94..bb8730aec 100644 --- a/docs/maintenance-and-troubleshooting.md +++ b/docs/maintenance-and-troubleshooting.md @@ -18,4 +18,4 @@ sudo journalctl -fu matrix-synapse ## Postgres -See the dedicated [PostgreSQL Maintenance](maintenances-postgres.md) documentation page. +See the dedicated [PostgreSQL Maintenance](maintenance-postgres.md) documentation page. From 668f98a2d3151c643ed97ab1e04b9ba1234c19ef Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 19 Jun 2019 10:40:59 +0300 Subject: [PATCH 15/15] Escape domain in bridge registration regex --- roles/matrix-bridge-mautrix-facebook/defaults/main.yml | 2 +- roles/matrix-bridge-mautrix-telegram/defaults/main.yml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml index 75752cda9..fc8886751 100644 --- a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml @@ -171,7 +171,7 @@ matrix_mautrix_facebook_registration_yaml: | namespaces: users: - exclusive: true - regex: '@facebook_.+:{{ matrix_mautrix_facebook_homeserver_domain }}' + regex: '@facebook_.+:{{ matrix_mautrix_facebook_homeserver_domain|regex_escape }}' url: {{ matrix_mautrix_facebook_appservice_address }} sender_localpart: facebookbot rate_limited: false diff --git a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml index ca3738074..af80ff313 100644 --- a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml @@ -367,10 +367,10 @@ matrix_mautrix_telegram_registration_yaml: | namespaces: users: - exclusive: true - regex: '@telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain }}' + regex: '@telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain|regex_escape }}' aliases: - exclusive: true - regex: '#telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain }}' + regex: '#telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain|regex_escape }}' url: {{ matrix_mautrix_telegram_appservice_address }} sender_localpart: telegrambot rate_limited: false