diff --git a/CHANGELOG.md b/CHANGELOG.md index 16b2d25ff..58466aa8b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,43 @@ +# 2019-06-15 + +## (BC Break) Telegram bridge configuration is now entirely managed by the playbook + +Until now, configuration files for the [Telegram bridge](docs/configuring-playbook-bridge-mautrix-telegram.md) were created by the playbook initially, but never modified later on. + +From now on, the playbook will keep those configuration in sync for you. + +This means that if you were making manual changes to the `/matrix/mautrix-telegram/config.yaml` or `/matrix/mautrix-telegram/registration.yaml` configuration files, those would be lost the next time you run the playbook. + +The bridge now stores configuration in a subdirectory (`/matrix/mautrix-telegram/config`), so your old configuration remains in the base directory (`/matrix/mautrix-telegram`). +You need to migrate any manual changes over to the new `matrix_mautrix_telegram_configuration_extension_yaml` variable, so that the playbook would apply them for you. + +Likewise, data is now also stored in a subdirectory (`/matrix/mautrix-telegram/data`). When you run the playbook with an existing database file (`/matrix/mautrix-telegram/mautrix-telegram.db`), the playbook will stop the bridge and relocate the database file to the `./data` directory. There's no data-loss involved. You'll need to restart the bridge manually though (`--tags=start`). + +Also, we're now following the default configuration for the Telegram bridge, so some default configuration values are different: + +- `edits_as_replies` (used to be `false`, now `true`) - previously replies were not sent over to Matrix at all; ow they are sent over as a reply to the original message +- `inline_images` (used to be `true`, now `false`) - this has to do with captioned images. Inline-image (included caption) are said to exhibit troubles on Riot iOS. When `false`, the caption arrives on the Matrix side as a separate message. +- `authless_portals` (used to be `false`, now `true`) - creating portals from the Telegram side is now possible +- `whitelist_group_admins` (used to be `false`, now `true`) - allows Telegram group admins to use the bot commands + +If the new values are not to your liking, use `matrix_mautrix_telegram_configuration_extension_yaml` to specify an override (refer to `matrix_mautrix_telegram_configuration_yaml` to figure out which variable goes where). + + +# 2019-06-12 + +## Synapse v1.0 + +With [Synapse v1.0 now available](https://matrix.org/blog/2019/06/11/introducing-matrix-1-0-and-the-matrix-org-foundation) and most people being on at least Synapse v0.99, it's time to remove the `_matrix._tcp` DNS SRV record that we've been keeping for compatibility with old Synapse versions (<= 0.34). + +According to the [Server Discovery specification](https://matrix.org/docs/spec/server_server/r0.1.2.html#server-discovery), it's no harm to keep the DNS SRV record. But since it's not necessary for federating with the larger Matrix network anymore, you should be safe to get rid of it. + +**Note**: don't confuse the `_matrix._tcp` and `_matrix-identity._tcp` DNS SRV records. The latter, **must not** be removed. + +For completeness, we must say that using a `_matrix._tcp` [SRV record for Server Delegation](docs/howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced) is still valid and useful for certain deployments. It's just that our guide recommends the [`/.well-known/matrix/server` Server Delegation method](docs/howto-server-delegation.md#server-delegation-via-a-well-known-file), due to its easier implementation when using this playbook. + +Besides this optional/non-urgent DNS change, assuming you're already on Synapse v0.99, upgrading to Synapse v1.0 should be as simple as [re-running the playbook](docs/maintenance-upgrading-services.md). + + # 2019-06-07 ## (BC Break) Facebook bridge configuration is now entirely managed by the playbook diff --git a/docs/configuring-dns.md b/docs/configuring-dns.md index 7638033be..678c1679e 100644 --- a/docs/configuring-dns.md +++ b/docs/configuring-dns.md @@ -9,16 +9,12 @@ You do, however need to instruct the Matrix network that Matrix services for ``. As we discuss in [Server Delegation](howto-server-delegation.md), there are 2 different ways to set up such delegation: -- using a `https:///.well-known/matrix/server` file (on the base domain!) -- using DNS SRV records +- either by serving a `https:///.well-known/matrix/server` file (from the base domain!) +- or by using a `_matrix._tcp` DNS SRV record (don't confuse this with the `_matrix-identity._tcp` SRV record described below) This playbook mostly discusses the well-known file method, because it's easier to manage with regard to certificates. If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do. -To make matters worse, for backward compatibility until the Synapse server reaches v1.0, you need to set up a DNS SRV record anyway. -So don't be confused if the general flow of this playbook asks you for both DNS SRV records and for setting up a well-known file. -It's a temporary requirement during the Synapse v0.99/v1.0 transition. - ## General outline of DNS settings you need to do @@ -27,12 +23,8 @@ It's a temporary requirement during the Synapse v0.99/v1.0 transition. | A | `matrix` | - | - | - | `matrix-server-IP` | | CNAME | `riot` | - | - | - | `matrix.` | | CNAME | `dimension` | - | - | - | `matrix.` | -| SRV | `_matrix._tcp` | 10 | 0 | 8448 | `matrix.` | | SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.` | -The `_matrix._tcp` SRV record is a temporary measure and will not be necessary in the near future. -In fact, it will have to be removed at some point. To learn more about that, read below. - ## Subdomains setup @@ -44,23 +36,6 @@ If you'd rather instruct the playbook not to install Riot (`matrix_riot_web_enab The `dimension.` subdomain may be necessary, because this playbook could install the [Dimension integrations manager](http://dimension.t2bot.io/) for you. Dimension installation is disabled by default, because it's only possible to install it after the other Matrix services are working (see [Setting up Dimension](configuring-playbook-dimension.md) later). If you do not wish to set up Dimension, feel free to skip the `dimension.` DNS record. -## `_matrix._tcp` SRV record setup (temporary requirement) - -All services created by this playbook are meant to be installed on their own server (such as `matrix.`). - -To use a Matrix user identifier like `@:` while hosting services on `matrix.`, we need to instruct the Matrix network of such a delegation/redirection by means of setting up a DNS SRV record. - -The SRV record should look like this: -- Name: `_matrix._tcp` (use this text as-is) -- Content: `10 0 8448 matrix.` (replace `` with your own) - -A [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) is superseding the `_matrix._tcp` SRV record for our use case. **During the transition phase, you'll need to set up both mechanisms**. We'll instruct you how to set up the file-based mechanism after the [installation phase](installing.md) for this playbook. - -Doing delegation/redirection of Matrix services using a DNS SRV record (`_matrix._tcp`) is a **temporary measure** for our use-case, that will only be necessary before Synapse v1.0 is released. - -As more and more people upgrade to the Synapse v0.99 transitional release and just before the final Synapse v1.0 gets released, at some point in the near future **you will need to remove the `_matrix._tcp` SRV record** and leave only the [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) in place. - - ## `_matrix-identity._tcp` SRV record setup To make the [mxisd](https://github.com/kamax-io/mxisd) Identity Server (which this playbook installs for you) be authoritative for your domain name, set up one more SRV record that looks like this: diff --git a/docs/configuring-well-known.md b/docs/configuring-well-known.md index 50344c05f..a8e81c732 100644 --- a/docs/configuring-well-known.md +++ b/docs/configuring-well-known.md @@ -17,13 +17,8 @@ As [per the Server-Server specification](https://matrix.org/docs/spec/server_ser Server delegation can be configured using DNS SRV records or by setting up a `/.well-known/matrix/server` file on the base domain (``). -We have discussed the DNS SRV record method already in the "`_matrix._tcp` SRV record setup (temporary requirement)" section of [Configuring DNS](configuring-dns.md). - -Both methods have their place and will continue to do so. Usually, you would need to use just one of these delegation methods. +Both methods have their place and will continue to do so. You only need to use just one of these delegation methods. For simplicity reasons, our setup advocates for the `/.well-known/matrix/server` method and guides you into using that. -For backward compatibility with older Synapse servers (< v0.99), however, for now you are also required to set up a `_matrix._tcp` DNS SRV record (in addition to the `/.well-known/matrix/server` file on the base domain). - -As the Synapse server progresses towards v1.0, only the `/.well-known/matrix/server` file will be used by us, unless you have a more special setup necessitating a DNS SRV record. At that future moment, you would need to remove the `_matrix._tcp` SRV record because Synapse v1.0+ will do the wrong thing if a SRV record exists. To learn how to set up `/.well-known/matrix/server`, read the Installing section below. diff --git a/docs/maintenance-and-troubleshooting.md b/docs/maintenance-and-troubleshooting.md index ad0babc94..bb8730aec 100644 --- a/docs/maintenance-and-troubleshooting.md +++ b/docs/maintenance-and-troubleshooting.md @@ -18,4 +18,4 @@ sudo journalctl -fu matrix-synapse ## Postgres -See the dedicated [PostgreSQL Maintenance](maintenances-postgres.md) documentation page. +See the dedicated [PostgreSQL Maintenance](maintenance-postgres.md) documentation page. diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index cf8d4830b..aa558e71d 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -67,6 +67,10 @@ matrix_appservice_irc_enabled: false # matrix-appservice-irc's client-server port to the local host. matrix_appservice_irc_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9999' }}" +# The IRC bridge docs say that if homeserver presence is disabled, it's better to also disable +# IRC bridge presence, for performance reasons. +matrix_appservice_irc_homeserver_enablePresence: "{{ matrix_synapse_use_presence }}" + matrix_appservice_irc_systemd_required_services_list: | {{ ['docker.service'] @@ -90,10 +94,6 @@ matrix_appservice_irc_systemd_required_services_list: | # We don't enable bridges by default. matrix_mautrix_facebook_enabled: false -matrix_mautrix_facebook_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-appservice-token') | to_uuid }}" - -matrix_mautrix_facebook_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-homeserver-token') | to_uuid }}" - matrix_mautrix_facebook_systemd_required_services_list: | {{ ['docker.service'] @@ -101,6 +101,10 @@ matrix_mautrix_facebook_systemd_required_services_list: | (['matrix-synapse.service'] if matrix_synapse_enabled else []) }} +matrix_mautrix_facebook_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-appservice-token') | to_uuid }}" + +matrix_mautrix_facebook_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'facebook-homeserver-token') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-facebook @@ -117,6 +121,7 @@ matrix_mautrix_facebook_systemd_required_services_list: | # We don't enable bridges by default. matrix_mautrix_telegram_enabled: false + matrix_mautrix_telegram_systemd_required_services_list: | {{ ['docker.service'] @@ -124,6 +129,10 @@ matrix_mautrix_telegram_systemd_required_services_list: | (['matrix-synapse.service'] if matrix_synapse_enabled else []) }} +matrix_mautrix_telegram_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram-appservice-token') | to_uuid }}" + +matrix_mautrix_telegram_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram-homeserver-token') | to_uuid }}" + matrix_mautrix_telegram_public_endpoint: "/{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegram') | to_uuid }}" matrix_mautrix_telegram_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:9006' }}" @@ -438,6 +447,8 @@ matrix_riot_web_enable_presence_by_hs_url: | else {matrix_riot_web_default_hs_url: false} }} +matrix_riot_web_welcome_user_id: ~ + ###################################################################### # # /matrix-riot-web diff --git a/roles/matrix-base/tasks/self_check_dns.yml b/roles/matrix-base/tasks/self_check_dns.yml index ceab00dab..d17b2149a 100644 --- a/roles/matrix-base/tasks/self_check_dns.yml +++ b/roles/matrix-base/tasks/self_check_dns.yml @@ -1,12 +1,7 @@ --- -- name: Determine DNS SRV records to check (Matrix) - set_fact: - dns_srv_record_checks: - - service_and_protocol: "_matrix._tcp" - domain: "{{ (matrix_domain + '.') }}" - expected_target: "{{ (matrix_server_fqn_matrix + '.') }}" - expected_port: 8448 +- set_fact: + dns_srv_record_checks: [] - block: - set_fact: diff --git a/roles/matrix-bridge-appservice-discord/tasks/init.yml b/roles/matrix-bridge-appservice-discord/tasks/init.yml index 9d7b1939b..7370b0bde 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/init.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-discord'] }}" when: matrix_appservice_discord_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_appservice_discord_base_path }}/discord-registration.yaml,dst=/matrix-appservice-discord-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-appservice-discord-registration.yaml"] }} + when: matrix_appservice_discord_enabled|bool diff --git a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml index 0f683c84f..3dc168391 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml @@ -78,15 +78,3 @@ {{ matrix_appservice_discord_docker_image }} /bin/sh -c "node .././build/tools/addbot.js > invite_link" when: "not appservice_discord_link_generated.stat.exists" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_appservice_discord_base_path }}/discord-registration.yaml,dst=/matrix-appservice-discord-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-appservice-discord-registration.yaml"] }} diff --git a/roles/matrix-bridge-appservice-irc/defaults/main.yml b/roles/matrix-bridge-appservice-irc/defaults/main.yml index 9f05fc4e3..a29f25a69 100644 --- a/roles/matrix-bridge-appservice-irc/defaults/main.yml +++ b/roles/matrix-bridge-appservice-irc/defaults/main.yml @@ -8,6 +8,11 @@ matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_ matrix_appservice_irc_base_path: "{{ matrix_base_data_path }}/appservice-irc" +matrix_appservice_irc_homeserver_url: 'http://matrix-synapse:8008' +matrix_appservice_irc_homeserver_media_url: 'https://{{ matrix_server_fqn_matrix }}' +matrix_appservice_irc_homeserver_domain: '{{ matrix_domain }}' +matrix_appservice_irc_homeserver_enablePresence: true + # Controls whether the matrix-appservice-discord container exposes its HTTP port (tcp/9999 in the container). # # Takes an ":" or "" value (e.g. "127.0.0.1:9999"), or empty string to not expose. @@ -25,9 +30,45 @@ matrix_appservice_irc_systemd_wanted_services_list: [] matrix_appservice_irc_configuration_yaml: | #jinja2: lstrip_blocks: True homeserver: - url: "https://{{ matrix_server_fqn_matrix }}" - domain: "{{ matrix_domain }}" - enablePresence: true + # The URL to the home server for client-server API calls, also used to form the + # media URLs as displayed in bridged IRC channels: + url: {{ matrix_appservice_irc_homeserver_url }} + # + # The URL of the homeserver hosting media files. This is only used to transform + # mxc URIs to http URIs when bridging m.room.[file|image] events. Optional. By + # default, this is the homeserver URL, specified above. + # + media_url: {{ matrix_appservice_irc_homeserver_media_url }} + + # Drop Matrix messages which are older than this number of seconds, according to + # the event's origin_server_ts. + # If the bridge is down for a while, the homeserver will attempt to send all missed + # events on reconnection. These events may be hours old, which can be confusing to + # IRC users if they are then bridged. This option allows these old messages to be + # dropped. + # CAUTION: This is a very coarse heuristic. Federated homeservers may have different + # clock times and hence produce different origin_server_ts values, which may be old + # enough to cause *all* events from the homeserver to be dropped. + # Default: 0 (don't ever drop) + # dropMatrixMessagesAfterSecs: 300 # 5 minutes + + # The 'domain' part for user IDs on this home server. Usually (but not always) + # is the "domain name" part of the HS URL. + domain: {{ matrix_appservice_irc_homeserver_domain }} + + # Should presence be enabled for matrix clients on this bridge. If disabled on the + # homeserver then it should also be disabled here to avoid excess traffic. + # Default: true + enablePresence: {{ matrix_appservice_irc_homeserver_enablePresence|to_json }} + + # Options here are generally only applicable to large-scale bridges and may have + # consequences greater than other options in this configuration file. + advanced: + # The maximum number of HTTP(S) sockets to maintain. Usually this is unlimited + # however for large bridges it is important to rate limit the bridge to avoid + # accidentally overloading the homeserver. Defaults to 1000, which should be + # enough for the vast majority of use cases. + maxHttpSockets: 1000 matrix_appservice_irc_configuration_extension_yaml: | # Your custom YAML configuration for Appservice IRC servers goes here. diff --git a/roles/matrix-bridge-appservice-irc/tasks/init.yml b/roles/matrix-bridge-appservice-irc/tasks/init.yml index 2bd8c2a71..1ebfd0735 100644 --- a/roles/matrix-bridge-appservice-irc/tasks/init.yml +++ b/roles/matrix-bridge-appservice-irc/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-irc'] }}" when: matrix_appservice_irc_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_appservice_irc_base_path }}/registration.yaml,dst=/matrix-appservice-irc-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-appservice-irc-registration.yaml"] }} + when: matrix_appservice_irc_enabled|bool diff --git a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml index 62a81e37c..8681c4e06 100644 --- a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml @@ -72,18 +72,6 @@ -l irc_bot when: "not appservice_irc_registration_file.stat.exists" -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_appservice_irc_base_path }}/registration.yaml,dst=/matrix-appservice-irc-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-appservice-irc-registration.yaml"] }} - - name: Ensure IRC configuration directory permissions are correct file: path: "{{ matrix_appservice_irc_base_path }}" diff --git a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml index 75752cda9..fc8886751 100644 --- a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml @@ -171,7 +171,7 @@ matrix_mautrix_facebook_registration_yaml: | namespaces: users: - exclusive: true - regex: '@facebook_.+:{{ matrix_mautrix_facebook_homeserver_domain }}' + regex: '@facebook_.+:{{ matrix_mautrix_facebook_homeserver_domain|regex_escape }}' url: {{ matrix_mautrix_facebook_appservice_address }} sender_localpart: facebookbot rate_limited: false diff --git a/roles/matrix-bridge-mautrix-facebook/tasks/init.yml b/roles/matrix-bridge-mautrix-facebook/tasks/init.yml index b19c90cd9..99e8a9ec6 100644 --- a/roles/matrix-bridge-mautrix-facebook/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-facebook/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-facebook'] }}" when: matrix_mautrix_facebook_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_facebook_config_path }}/registration.yaml,dst=/matrix-mautrix-facebook-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-facebook-registration.yaml"] }} + when: matrix_mautrix_facebook_enabled|bool diff --git a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml index a69a58c99..68c823c6c 100644 --- a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml @@ -71,15 +71,3 @@ service: daemon_reload: yes when: "matrix_mautrix_facebook_systemd_service_result.changed" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_facebook_config_path }}/registration.yaml,dst=/matrix-mautrix-facebook-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-facebook-registration.yaml"] }} diff --git a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 index 7bc4f1890..e0ac9084c 100644 --- a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 +++ b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 @@ -17,8 +17,8 @@ ExecStartPre=/usr/bin/docker run --rm --name matrix-mautrix-facebook-db \ --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ - -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ -v {{ matrix_mautrix_facebook_config_path }}:/config:z \ + -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ {{ matrix_mautrix_facebook_docker_image }} \ alembic -x config=/config/config.yaml upgrade head @@ -30,8 +30,8 @@ ExecStart=/usr/bin/docker run --rm --name matrix-mautrix-facebook \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ --network={{ matrix_docker_network }} \ - -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ -v {{ matrix_mautrix_facebook_config_path }}:/config:z \ + -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ {% for arg in matrix_mautrix_facebook_container_extra_arguments %} {{ arg }} \ {% endfor %} diff --git a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml index f06f0a76b..af80ff313 100644 --- a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml @@ -7,6 +7,8 @@ matrix_mautrix_telegram_docker_image: "tulir/mautrix-telegram:v0.5.2" matrix_mautrix_telegram_docker_image_force_pull: "{{ matrix_mautrix_telegram_docker_image.endswith(':latest') }}" matrix_mautrix_telegram_base_path: "{{ matrix_base_data_path }}/mautrix-telegram" +matrix_mautrix_telegram_config_path: "{{ matrix_mautrix_telegram_base_path }}/config" +matrix_mautrix_telegram_data_path: "{{ matrix_mautrix_telegram_base_path }}/data" # Get your own API keys at https://my.telegram.org/apps matrix_mautrix_telegram_api_id: '' @@ -17,7 +19,7 @@ matrix_mautrix_telegram_api_hash: '' # Example: /741a0483-ba17-4682-9900-30bd7269f1cc matrix_mautrix_telegram_public_endpoint: '' -matrix_mautrix_telegram_homeserver_address: 'https://{{ matrix_server_fqn_matrix }}' +matrix_mautrix_telegram_homeserver_address: 'http://matrix-synapse:8008' matrix_mautrix_telegram_homeserver_domain: '{{ matrix_domain }}' matrix_mautrix_telegram_appservice_address: 'http://matrix-mautrix-telegram:8080' matrix_mautrix_telegram_appservice_public_external: 'https://{{ matrix_server_fqn_matrix }}{{ matrix_mautrix_telegram_public_endpoint }}' @@ -35,3 +37,342 @@ matrix_mautrix_telegram_systemd_required_services_list: ['docker.service'] # List of systemd services that matrix-mautrix-telegram.service wants matrix_mautrix_telegram_systemd_wanted_services_list: [] + +matrix_mautrix_telegram_appservice_token: '' +matrix_mautrix_telegram_homeserver_token: '' + +# Default mxisd configuration template which covers the generic use case. +# You can customize it by controlling the various variables inside it. +# +# For a more advanced customization, you can extend the default (see `matrix_mautrix_telegram_configuration_extension_yaml`) +# or completely replace this variable with your own template. +matrix_mautrix_telegram_configuration_yaml: | + #jinja2: lstrip_blocks: "True" + # Homeserver details + homeserver: + # The address that this appservice can use to connect to the homeserver. + address: {{ matrix_mautrix_telegram_homeserver_address }} + # The domain of the homeserver (for MXIDs, etc). + domain: {{ matrix_mautrix_telegram_homeserver_domain }} + # Whether or not to verify the SSL certificate of the homeserver. + # Only applies if address starts with https:// + verify_ssl: true + + # Application service host/registration related details + # Changing these values requires regeneration of the registration. + appservice: + # The address that the homeserver can use to connect to this appservice. + address: {{ matrix_mautrix_telegram_appservice_address }} + + # The hostname and port where this appservice should listen. + hostname: 0.0.0.0 + port: 8080 + # The maximum body size of appservice API requests (from the homeserver) in mebibytes + # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s + max_body_size: 1 + + # The full URI to the database. SQLite and Postgres are fully supported. + # Other DBMSes supported by SQLAlchemy may or may not work. + # Format examples: + # SQLite: sqlite:///filename.db + # Postgres: postgres://username:password@hostname/dbname + database: sqlite:////data/mautrix-telegram.db + + # Public part of web server for out-of-Matrix interaction with the bridge. + # Used for things like login if the user wants to make sure the 2FA password isn't stored in + # the HS database. + public: + # Whether or not the public-facing endpoints should be enabled. + enabled: true + # The prefix to use in the public-facing endpoints. + prefix: {{ matrix_mautrix_telegram_public_endpoint }} + # The base URL where the public-facing endpoints are available. The prefix is not added + # implicitly. + external: {{ matrix_mautrix_telegram_appservice_public_external }} + + # Provisioning API part of the web server for automated portal creation and fetching information. + # Used by things like Dimension (https://dimension.t2bot.io/). + provisioning: + # Whether or not the provisioning API should be enabled. + enabled: false + # The prefix to use in the provisioning API endpoints. + prefix: /_matrix/provision/v1 + # The shared secret to authorize users of the API. + # Set to "generate" to generate and save a new token. + shared_secret: generate + + # The unique ID of this appservice. + id: telegram + # Username of the appservice bot. + bot_username: telegrambot + # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty + # to leave display name/avatar as-is. + bot_displayname: Telegram bridge bot + bot_avatar: mxc://maunium.net/tJCRmUyJDsgRNgqhOgoiHWbX + + # Authentication tokens for AS <-> HS communication. + as_token: "{{ matrix_mautrix_telegram_appservice_token }}" + hs_token: "{{ matrix_mautrix_telegram_homeserver_token }}" + + # Bridge config + bridge: + # Localpart template of MXIDs for Telegram users. + # {userid} is replaced with the user ID of the Telegram user. + username_template: "telegram_{userid}" + # Localpart template of room aliases for Telegram portal rooms. + # {groupname} is replaced with the name part of the public channel/group invite link ( https://t.me/{} ) + alias_template: "telegram_{groupname}" + # Displayname template for Telegram users. + # {displayname} is replaced with the display name of the Telegram user. + displayname_template: "{displayname} (Telegram)" + + # Set the preferred order of user identifiers which to use in the Matrix puppet display name. + # In the (hopefully unlikely) scenario that none of the given keys are found, the numeric user + # ID is used. + # + # If the bridge is working properly, a phone number or an username should always be known, but + # the other one can very well be empty. + # + # Valid keys: + # "full name" (First and/or last name) + # "full name reversed" (Last and/or first name) + # "first name" + # "last name" + # "username" + # "phone number" + displayname_preference: + - full name + - username + - phone number + + # Maximum number of members to sync per portal when starting up. Other members will be + # synced when they send messages. The maximum is 10000, after which the Telegram server + # will not send any more members. + # Defaults to no local limit (-> limited to 10000 by server) + max_initial_member_sync: -1 + # Whether or not to sync the member list in channels. + # If no channel admins have logged into the bridge, the bridge won't be able to sync the member + # list regardless of this setting. + sync_channel_members: true + # Whether or not to skip deleted members when syncing members. + skip_deleted_members: true + # Whether or not to automatically synchronize contacts and chats of Matrix users logged into + # their Telegram account at startup. + startup_sync: true + # Number of most recently active dialogs to check when syncing chats. + # Dialogs include groups and private chats, but only groups are synced. + # Set to 0 to remove limit. + sync_dialog_limit: 30 + # The maximum number of simultaneous Telegram deletions to handle. + # A large number of simultaneous redactions could put strain on your homeserver. + max_telegram_delete: 10 + # Whether or not to automatically sync the Matrix room state (mostly unpuppeted displaynames) + # at startup and when creating a bridge. + sync_matrix_state: true + # Allow logging in within Matrix. If false, the only way to log in is using the out-of-Matrix + # login website (see appservice.public config section) + allow_matrix_login: true + # Whether or not to bridge plaintext highlights. + # Only enable this if your displayname_template has some static part that the bridge can use to + # reliably identify what is a plaintext highlight. + plaintext_highlights: false + # Show message editing as a reply to the original message. + # If this is false, message edits are not shown at all, as Matrix does not support editing yet. + edits_as_replies: true + # Highlight changed/added parts in edits. Requires lxml. + highlight_edits: false + # Whether or not to make portals of publicly joinable channels/supergroups publicly joinable on Matrix. + public_portals: true + # Whether or not to fetch and handle Telegram updates at startup from the time the bridge was down. + # Currently only works for private chats and normal groups. + catch_up: false + # Whether or not to use /sync to get presence, read receipts and typing notifications when using + # your own Matrix account as the Matrix puppet for your Telegram account. + sync_with_custom_puppets: true + # Set to false to disable link previews in messages sent to Telegram. + telegram_link_preview: true + # Use inline images instead of a separate message for the caption. + # N.B. Inline images are not supported on all clients (e.g. Riot iOS). + inline_images: false + # Maximum size of image in megabytes before sending to Telegram as a document. + image_as_file_size: 10 + + # Whether to bridge Telegram bot messages as m.notices or m.texts. + bot_messages_as_notices: true + bridge_notices: + # Whether or not Matrix bot messages (type m.notice) should be bridged. + default: false + # List of user IDs for whom the previous flag is flipped. + # e.g. if bridge_notices.default is false, notices from other users will not be bridged, but + # notices from users listed here will be bridged. + exceptions: [] + + # Some config options related to Telegram message deduplication. + # The default values are usually fine, but some debug messages/warnings might recommend you + # change these. + deduplication: + # Whether or not to check the database if the message about to be sent is a duplicate. + pre_db_check: false + # The number of latest events to keep when checking for duplicates. + # You might need to increase this on high-traffic bridge instances. + cache_queue_length: 20 + + + # The formats to use when sending messages to Telegram via the relay bot. + # + # Telegram doesn't have built-in emotes, so the m.emote format is also used for non-relaybot users. + # + # Available variables: + # $sender_displayname - The display name of the sender (e.g. Example User) + # $sender_username - The username (Matrix ID localpart) of the sender (e.g. exampleuser) + # $sender_mxid - The Matrix ID of the sender (e.g. @exampleuser:example.com) + # $message - The message content as HTML + message_formats: + m.text: "$sender_displayname: $message" + m.emote: "* $sender_displayname $message" + m.file: "$sender_displayname sent a file: $message" + m.image: "$sender_displayname sent an image: $message" + m.audio: "$sender_displayname sent an audio file: $message" + m.video: "$sender_displayname sent a video: $message" + m.location: "$sender_displayname sent a location: $message" + + # The formats to use when sending state events to Telegram via the relay bot. + # + # Variables from `message_formats` that have the `sender_` prefix are available without the prefix. + # In name_change events, `$prev_displayname` is the previous displayname. + # + # Set format to an empty string to disable the messages for that event. + state_event_formats: + join: "$displayname joined the room." + leave: "$displayname left the room." + name_change: "$prev_displayname changed their name to $displayname" + + # Filter rooms that can/can't be bridged. Can also be managed using the `filter` and + # `filter-mode` management commands. + # + # Filters do not affect direct chats. + # An empty blacklist will essentially disable the filter. + filter: + # Filter mode to use. Either "blacklist" or "whitelist". + # If the mode is "blacklist", the listed chats will never be bridged. + # If the mode is "whitelist", only the listed chats can be bridged. + mode: blacklist + # The list of group/channel IDs to filter. + list: [] + + # The prefix for commands. Only required in non-management rooms. + command_prefix: "!tg" + + # Permissions for using the bridge. + # Permitted values: + # relaybot - Only use the bridge via the relaybot, no access to commands. + # user - Relaybot level + access to commands to create bridges. + # puppeting - User level + logging in with a Telegram account. + # full - Full access to use the bridge, i.e. previous levels + Matrix login. + # admin - Full access to use the bridge and some extra administration commands. + # Permitted keys: + # * - All Matrix users + # domain - All users on that homeserver + # mxid - Specific user + permissions: + '{{ matrix_mautrix_telegram_homeserver_domain }}': full + + # Options related to the message relay Telegram bot. + relaybot: + # Whether or not to allow creating portals from Telegram. + authless_portals: true + # Whether or not to allow Telegram group admins to use the bot commands. + whitelist_group_admins: true + # Whether or not to ignore incoming events sent by the relay bot. + ignore_own_incoming_events: true + # List of usernames/user IDs who are also allowed to use the bot commands. + whitelist: [] + + # Telegram config + telegram: + # Get your own API keys at https://my.telegram.org/apps + api_id: {{ matrix_mautrix_telegram_api_id }} + api_hash: {{ matrix_mautrix_telegram_api_hash }} + # (Optional) Create your own bot at https://t.me/BotFather + bot_token: disabled + # Custom server to connect to. + server: + # Set to true to use these server settings. If false, will automatically + # use production server assigned by Telegram. Set to false in production. + enabled: false + # The DC ID to connect to. + dc: 2 + # The IP to connect to. + ip: 149.154.167.40 + # The port to connect to. 443 may not work, 80 is better and both are equally secure. + port: 80 + # Telethon proxy configuration. + # You must install PySocks from pip for proxies to work. + proxy: + # Allowed types: disabled, socks4, socks5, http + type: disabled + # Proxy IP address and port. + address: 127.0.0.1 + port: 1080 + # Whether or not to perform DNS resolving remotely. + rdns: true + # Proxy authentication (optional). + username: "" + password: "" + + # Python logging configuration. + # + # See section 16.7.2 of the Python documentation for more info: + # https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema + logging: + version: 1 + formatters: + precise: + format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" + handlers: + console: + class: logging.StreamHandler + formatter: precise + loggers: + mau: + level: DEBUG + telethon: + level: DEBUG + aiohttp: + level: INFO + root: + level: DEBUG + handlers: [console] + + +matrix_mautrix_telegram_configuration_extension_yaml: | + # Your custom YAML configuration goes here. + # This configuration extends the default starting configuration (`matrix_mautrix_telegram_configuration_yaml`). + # + # You can override individual variables from the default configuration, or introduce new ones. + # + # If you need something more special, you can take full control by + # completely redefining `matrix_mautrix_telegram_configuration_yaml`. + +matrix_mautrix_telegram_configuration_extension: "{{ matrix_mautrix_telegram_configuration_extension_yaml|from_yaml if matrix_mautrix_telegram_configuration_extension_yaml|from_yaml is mapping else {} }}" + +# Holds the final configuration (a combination of the default and its extension). +# You most likely don't need to touch this variable. Instead, see `matrix_mautrix_telegram_configuration_yaml`. +matrix_mautrix_telegram_configuration: "{{ matrix_mautrix_telegram_configuration_yaml|from_yaml|combine(matrix_mautrix_telegram_configuration_extension, recursive=True) }}" + +matrix_mautrix_telegram_registration_yaml: | + id: telegram + as_token: "{{ matrix_mautrix_telegram_appservice_token }}" + hs_token: "{{ matrix_mautrix_telegram_homeserver_token }}" + namespaces: + users: + - exclusive: true + regex: '@telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain|regex_escape }}' + aliases: + - exclusive: true + regex: '#telegram_.+:{{ matrix_mautrix_telegram_homeserver_domain|regex_escape }}' + url: {{ matrix_mautrix_telegram_appservice_address }} + sender_localpart: telegrambot + rate_limited: false + +matrix_mautrix_telegram_registration: "{{ matrix_mautrix_telegram_registration_yaml|from_yaml }}" diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml index 27a84f1f6..1898b0993 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/init.yml @@ -1,3 +1,63 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-telegram'] }}" when: matrix_mautrix_telegram_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_config_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-telegram-registration.yaml"] }} + when: matrix_mautrix_telegram_enabled|bool + +- block: + - name: Fail if matrix-nginx-proxy role already executed + fail: + msg: >- + Trying to append Mautrix Telegram's reverse-proxying configuration to matrix-nginx-proxy, + but it's pointless since the matrix-nginx-proxy role had already executed. + To fix this, please change the order of roles in your plabook, + so that the matrix-nginx-proxy role would run after the matrix-bridge-mautrix-telegram role. + when: matrix_nginx_proxy_role_executed|default(False)|bool + + - name: Generate Mautrix Telegram proxying configuration for matrix-nginx-proxy + set_fact: + matrix_mautrix_telegram_matrix_nginx_proxy_configuration: | + location {{ matrix_mautrix_telegram_public_endpoint }} { + {% if matrix_nginx_proxy_enabled|default(False) %} + {# Use the embedded DNS resolver in Docker containers to discover the service #} + resolver 127.0.0.11 valid=5s; + set $backend "matrix-mautrix-telegram:8080"; + proxy_pass http://$backend; + {% else %} + {# Generic configuration for use outside of our container setup #} + proxy_pass http://127.0.0.1:9006; + {% endif %} + } + + - name: Register Mautrix Telegram proxying configuration with matrix-nginx-proxy + set_fact: + matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: | + {{ + matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks|default([]) + + + [matrix_mautrix_telegram_matrix_nginx_proxy_configuration] + }} + tags: + - always + when: matrix_mautrix_telegram_enabled|bool + +- name: Warn about reverse-proxying if matrix-nginx-proxy not used + debug: + msg: >- + NOTE: You've enabled the Mautrix Telegram bridge but are not using the matrix-nginx-proxy + reverse proxy. + Please make sure that you're proxying the `{{ matrix_mautrix_telegram_public_endpoint }}` + URL endpoint to the matrix-mautrix-telegram container. + You can expose the container's port using the `matrix_mautrix_telegram_container_http_host_bind_port` variable. + when: "matrix_mautrix_telegram_enabled|bool and matrix_nginx_proxy_enabled is not defined" diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml index 2ad0694a7..a5f6983a8 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml @@ -15,38 +15,50 @@ force_source: "{{ matrix_mautrix_telegram_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_telegram_docker_image_force_pull }}" -- name: Ensure Mautrix Telegram base directory exists +- name: Ensure Mautrix Telegram paths exist file: - path: "{{ matrix_mautrix_telegram_base_path }}" + path: "{{ item }}" state: directory mode: 0750 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" + with_items: + - "{{ matrix_mautrix_telegram_base_path }}" + - "{{ matrix_mautrix_telegram_config_path }}" + - "{{ matrix_mautrix_telegram_data_path }}" -- name: Check if a mautrix-telegram configuration file exists +- name: Check if an old database file already exists stat: - path: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" - register: mautrix_telegram_config_file_stat + path: "{{ matrix_mautrix_telegram_base_path }}/mautrix-telegram.db" + register: matrix_mautrix_telegram_stat_database -- name: Ensure Matrix Mautrix telegram config installed - template: - src: "{{ role_path }}/templates/config.yaml.j2" - dest: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" +- name: (Data relocation) Ensure matrix-mautrix-telegram.service is stopped + service: + name: matrix-mautrix-telegram + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mautrix_telegram_stat_database.stat.exists" + +- name: (Data relocation) Move mautrix-telegram database file to ./data directory + command: "mv {{ matrix_mautrix_telegram_base_path }}/mautrix-telegram.db {{ matrix_mautrix_telegram_data_path }}/mautrix-telegram.db" + when: "matrix_mautrix_telegram_stat_database.stat.exists" + +- name: Ensure mautrix-telegram config.yaml installed + copy: + content: "{{ matrix_mautrix_telegram_configuration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_telegram_config_path }}/config.yaml" mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" - when: "not mautrix_telegram_config_file_stat.stat.exists" -- name: (Migration) Fix up old configuration - lineinfile: - path: "{{ matrix_mautrix_telegram_base_path }}/config.yaml" - regexp: "{{ item.regexp }}" - line: "{{ item.line }}" - backrefs: yes - with_items: - - {'regexp': '^(\s+)filename: \./mautrix-telegram.log', 'line': '\1filename: /data/mautrix-telegram.log'} - - {'regexp': '^(\s+)database:', 'line': '\1database: sqlite:////data/mautrix-telegram.db'} - when: "mautrix_telegram_config_file_stat.stat.exists" +- name: Ensure mautrix-telegram registration.yaml installed + copy: + content: "{{ matrix_mautrix_telegram_registration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_telegram_config_path }}/registration.yaml" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_username }}" - name: Ensure matrix-mautrix-telegram.service installed template: @@ -59,79 +71,3 @@ service: daemon_reload: yes when: "matrix_mautrix_telegram_systemd_service_result.changed" - -- name: Check if a mautrix-telegram registration file exists - stat: - path: "{{ matrix_mautrix_telegram_base_path }}/registration.yaml" - register: mautrix_telegram_registration_file_stat - -- name: Generate matrix-mautrix-telegram registration.yaml if it doesn't exist - shell: - cmd: >- - /usr/bin/docker run - --rm - --user={{ matrix_user_uid }}:{{ matrix_user_gid }} - --cap-drop=ALL - --name matrix-mautrix-telegram-gen - -v {{ matrix_mautrix_telegram_base_path }}:/data:z - {{ matrix_mautrix_telegram_docker_image }} - python3 -m mautrix_telegram -g -c /data/config.yaml -r /data/registration.yaml - when: "not mautrix_telegram_registration_file_stat.stat.exists" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_telegram_base_path }}/registration.yaml,dst=/matrix-mautrix-telegram-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-telegram-registration.yaml"] }} - -- block: - - name: Fail if matrix-nginx-proxy role already executed - fail: - msg: >- - Trying to append Mautrix Telegram's reverse-proxying configuration to matrix-nginx-proxy, - but it's pointless since the matrix-nginx-proxy role had already executed. - To fix this, please change the order of roles in your plabook, - so that the matrix-nginx-proxy role would run after the matrix-bridge-mautrix-telegram role. - when: matrix_nginx_proxy_role_executed|default(False)|bool - - - name: Generate Mautrix Telegram proxying configuration for matrix-nginx-proxy - set_fact: - matrix_mautrix_telegram_matrix_nginx_proxy_configuration: | - location {{ matrix_mautrix_telegram_public_endpoint }} { - {% if matrix_nginx_proxy_enabled|default(False) %} - {# Use the embedded DNS resolver in Docker containers to discover the service #} - resolver 127.0.0.11 valid=5s; - set $backend "matrix-mautrix-telegram:8080"; - proxy_pass http://$backend; - {% else %} - {# Generic configuration for use outside of our container setup #} - proxy_pass http://127.0.0.1:9006; - {% endif %} - } - - - name: Register Mautrix Telegram proxying configuration with matrix-nginx-proxy - set_fact: - matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: | - {{ - matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks|default([]) - + - [matrix_mautrix_telegram_matrix_nginx_proxy_configuration] - }} - tags: - - always - -- name: Warn about reverse-proxying if matrix-nginx-proxy not used - debug: - msg: >- - NOTE: You've enabled the Mautrix Telegram bridge but are not using the matrix-nginx-proxy - reverse proxy. - Please make sure that you're proxying the `{{ matrix_mautrix_telegram_public_endpoint }}` - URL endpoint to the matrix-mautrix-telegram container. - You can expose the container's port using the `matrix_mautrix_telegram_container_http_host_bind_port` variable. - when: "matrix_nginx_proxy_enabled is not defined" diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml b/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml index d30be92e2..5b1f3b006 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/validate_config.yml @@ -9,6 +9,8 @@ - "matrix_mautrix_telegram_api_id" - "matrix_mautrix_telegram_api_hash" - "matrix_mautrix_telegram_public_endpoint" + - "matrix_mautrix_telegram_appservice_token" + - "matrix_mautrix_telegram_homeserver_token" - name: (Deprecation) Catch and report renamed Telegram variables fail: diff --git a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 deleted file mode 100644 index 0a8869a6a..000000000 --- a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 +++ /dev/null @@ -1,266 +0,0 @@ -#jinja2: lstrip_blocks: "True" -# Homeserver details -homeserver: - # The address that this appservice can use to connect to the homeserver. - address: {{ matrix_mautrix_telegram_homeserver_address }} - # The domain of the homeserver (for MXIDs, etc). - domain: {{ matrix_mautrix_telegram_homeserver_domain }} - # Whether or not to verify the SSL certificate of the homeserver. - # Only applies if address starts with https:// - verify_ssl: true - -# Application service host/registration related details -# Changing these values requires regeneration of the registration. -appservice: - # The address that the homeserver can use to connect to this appservice. - address: {{ matrix_mautrix_telegram_appservice_address }} - - # The hostname and port where this appservice should listen. - hostname: 0.0.0.0 - port: 8080 - # The maximum body size of appservice API requests (from the homeserver) in mebibytes - # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s - max_body_size: 1 - - # The full URI to the database. SQLite and Postgres are fully supported. - # Other DBMSes supported by SQLAlchemy may or may not work. - # Format examples: - # SQLite: sqlite:///filename.db - # Postgres: postgres://username:password@hostname/dbname - database: sqlite:////data/mautrix-telegram.db - - # Public part of web server for out-of-Matrix interaction with the bridge. - # Used for things like login if the user wants to make sure the 2FA password isn't stored in - # the HS database. - public: - # Whether or not the public-facing endpoints should be enabled. - enabled: true - # The prefix to use in the public-facing endpoints. - prefix: {{ matrix_mautrix_telegram_public_endpoint }} - # The base URL where the public-facing endpoints are available. The prefix is not added - # implicitly. - external: {{ matrix_mautrix_telegram_appservice_public_external }} - - # Provisioning API part of the web server for automated portal creation and fetching information. - # Used by things like Dimension (https://dimension.t2bot.io/). - provisioning: - # Whether or not the provisioning API should be enabled. - enabled: false - # The prefix to use in the provisioning API endpoints. - prefix: /_matrix/provision/v1 - # The shared secret to authorize users of the API. - # Set to "generate" to generate and save a new token. - shared_secret: generate - - # The unique ID of this appservice. - id: telegram - # Username of the appservice bot. - bot_username: telegrambot - # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty - # to leave display name/avatar as-is. - bot_displayname: Telegram bridge bot - bot_avatar: mxc://maunium.net/tJCRmUyJDsgRNgqhOgoiHWbX - - # Authentication tokens for AS <-> HS communication. Autogenerated; do not modify. - as_token: "This value is generated when generating the registration" - hs_token: "This value is generated when generating the registration" - -# Bridge config -bridge: - # Localpart template of MXIDs for Telegram users. - # {userid} is replaced with the user ID of the Telegram user. - username_template: "telegram_{userid}" - # Localpart template of room aliases for Telegram portal rooms. - # {groupname} is replaced with the name part of the public channel/group invite link ( https://t.me/{} ) - alias_template: "telegram_{groupname}" - # Displayname template for Telegram users. - # {displayname} is replaced with the display name of the Telegram user. - displayname_template: "{displayname} (Telegram)" - - # Set the preferred order of user identifiers which to use in the Matrix puppet display name. - # In the (hopefully unlikely) scenario that none of the given keys are found, the numeric user - # ID is used. - # - # If the bridge is working properly, a phone number or an username should always be known, but - # the other one can very well be empty. - # - # Valid keys: - # "full name" (First and/or last name) - # "full name reversed" (Last and/or first name) - # "first name" - # "last name" - # "username" - # "phone number" - displayname_preference: - - full name - - username - - phone number - - # Show message editing as a reply to the original message. - # If this is false, message edits are not shown at all, as Matrix does not support editing yet. - edits_as_replies: false - # Highlight changed/added parts in edits. Requires lxml. - highlight_edits: false - # Whether or not Matrix bot messages (type m.notice) should be bridged. - bridge_notices: true - # Whether to bridge Telegram bot messages as m.notices or m.texts. - bot_messages_as_notices: true - # Maximum number of members to sync per portal when starting up. Other members will be - # synced when they send messages. The maximum is 10000, after which the Telegram server - # will not send any more members. - # Defaults to no local limit (-> limited to 10000 by server) - max_initial_member_sync: -1 - # Whether or not to sync the member list in channels. - # If no channel admins have logged into the bridge, the bridge won't be able to sync the member - # list regardless of this setting. - sync_channel_members: true - # The maximum number of simultaneous Telegram deletions to handle. - # A large number of simultaneous redactions could put strain on your homeserver. - max_telegram_delete: 10 - # Allow logging in within Matrix. If false, the only way to log in is using the out-of-Matrix - # login website (see appservice.public config section) - allow_matrix_login: true - # Use inline images instead of m.image to make rich captions possible. - # N.B. Inline images are not supported on all clients (e.g. Riot iOS). - inline_images: true - # Whether or not to bridge plaintext highlights. - # Only enable this if your displayname_template has some static part that the bridge can use to - # reliably identify what is a plaintext highlight. - plaintext_highlights: false - # Whether or not to make portals of publicly joinable channels/supergroups publicly joinable on Matrix. - public_portals: true - # Whether to send stickers as the new native m.sticker type or normal m.images. - # Old versions of Riot don't support the new type at all. - # Remember that proper sticker support always requires Pillow to convert webp into png. - native_stickers: true - # Whether or not to fetch and handle Telegram updates at startup from the time the bridge was down. - # WARNING: Probably buggy, might get stuck in infinite loop. - catch_up: false - # Whether or not to use /sync to get presence, read receipts and typing notifications when using - # your own Matrix account as the Matrix puppet for your Telegram account. - sync_with_custom_puppets: true - - # Some config options related to Telegram message deduplication. - # The default values are usually fine, but some debug messages/warnings might recommend you - # change these. - deduplication: - # Whether or not to check the database if the message about to be sent is a duplicate. - pre_db_check: false - # The number of latest events to keep when checking for duplicates. - # You might need to increase this on high-traffic bridge instances. - cache_queue_length: 20 - - # The formats to use when sending messages to Telegram via the relay bot. - # - # Telegram doesn't have built-in emotes, so the m.emote format is also used for non-relaybot users. - # - # Available variables: - # $sender_displayname - The display name of the sender (e.g. Example User) - # $sender_username - The username (Matrix ID localpart) of the sender (e.g. exampleuser) - # $sender_mxid - The Matrix ID of the sender (e.g. @exampleuser:example.com) - # $message - The message content as HTML - message_formats: - m.text: "$sender_displayname: $message" - m.emote: "* $sender_displayname $message" - m.file: "$sender_displayname sent a file: $message" - m.image: "$sender_displayname sent an image: $message" - m.audio: "$sender_displayname sent an audio file: $message" - m.video: "$sender_displayname sent a video: $message" - m.location: "$sender_displayname sent a location: $message" - - # The formats to use when sending state events to Telegram via the relay bot. - # - # Variables from `message_formats` that have the `sender_` prefix are available without the prefix. - # In name_change events, `$prev_displayname` is the previous displayname. - # - # Set format to an empty string to disable the messages for that event. - state_event_formats: - join: "$displayname joined the room." - leave: "$displayname left the room." - name_change: "$prev_displayname changed their name to $displayname" - - # Filter rooms that can/can't be bridged. Can also be managed using the `filter` and - # `filter-mode` management commands. - # - # Filters do not affect direct chats. - # An empty blacklist will essentially disable the filter. - filter: - # Filter mode to use. Either "blacklist" or "whitelist". - # If the mode is "blacklist", the listed chats will never be bridged. - # If the mode is "whitelist", only the listed chats can be bridged. - mode: blacklist - # The list of group/channel IDs to filter. - list: [] - - # The prefix for commands. Only required in non-management rooms. - command_prefix: "!tg" - - # Permissions for using the bridge. - # Permitted values: - # relaybot - Only use the bridge via the relaybot, no access to commands. - # user - Relaybot level + access to commands to create bridges. - # puppeting - User level + logging in with a Telegram account. - # full - Full access to use the bridge, i.e. previous levels + Matrix login. - # admin - Full access to use the bridge and some extra administration commands. - # Permitted keys: - # * - All Matrix users - # domain - All users on that homeserver - # mxid - Specific user - permissions: - '{{ matrix_mautrix_telegram_homeserver_domain }}': full - - # Options related to the message relay Telegram bot. - relaybot: - # Whether or not to allow creating portals from Telegram. - authless_portals: false - # Whether or not to allow Telegram group admins to use the bot commands. - whitelist_group_admins: false - # Whether or not to ignore incoming events sent by the relay bot. - ignore_own_incoming_events: true - # List of usernames/user IDs who are also allowed to use the bot commands. - whitelist: - -# Telegram config -telegram: - # Get your own API keys at https://my.telegram.org/apps - api_id: {{ matrix_mautrix_telegram_api_id }} - api_hash: {{ matrix_mautrix_telegram_api_hash }} - # (Optional) Create your own bot at https://t.me/BotFather - bot_token: disabled - # Telethon proxy configuration. - # You must install PySocks from pip for proxies to work. - proxy: - # Allowed types: disabled, socks4, socks5, http - type: disabled - # Proxy IP address and port. - address: 127.0.0.1 - port: 1080 - # Whether or not to perform DNS resolving remotely. - rdns: true - # Proxy authentication (optional). - username: "" - password: "" - -# Python logging configuration. -# -# See section 16.7.2 of the Python documentation for more info: -# https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema -logging: - version: 1 - formatters: - precise: - format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" - handlers: - console: - class: logging.StreamHandler - formatter: precise - loggers: - mau: - level: DEBUG - telethon: - level: DEBUG - aiohttp: - level: INFO - root: - level: DEBUG - handlers: [console] diff --git a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 index 878e25854..02264236d 100644 --- a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 +++ b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 @@ -17,9 +17,10 @@ ExecStartPre=/usr/bin/docker run --rm --name matrix-mautrix-telegram-db \ --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ - -v {{ matrix_mautrix_telegram_base_path }}:/data:z \ + -v {{ matrix_mautrix_telegram_config_path }}:/config:z \ + -v {{ matrix_mautrix_telegram_data_path }}:/data:z \ {{ matrix_mautrix_telegram_docker_image }} \ - alembic -x config=/data/config.yaml upgrade head + alembic -x config=/config/config.yaml upgrade head # Intentional delay, so that the homeserver (we likely depend on) can manage to start. ExecStartPre=/bin/sleep 5 @@ -32,12 +33,13 @@ ExecStart=/usr/bin/docker run --rm --name matrix-mautrix-telegram \ {% if matrix_mautrix_telegram_container_http_host_bind_port %} -p {{ matrix_mautrix_telegram_container_http_host_bind_port }}:8080 \ {% endif %} - -v {{ matrix_mautrix_telegram_base_path }}:/data:z \ + -v {{ matrix_mautrix_telegram_config_path }}:/config:z \ + -v {{ matrix_mautrix_telegram_data_path }}:/data:z \ {% for arg in matrix_mautrix_telegram_container_extra_arguments %} {{ arg }} \ {% endfor %} {{ matrix_mautrix_telegram_docker_image }} \ - python3 -m mautrix_telegram -c /data/config.yaml + python3 -m mautrix_telegram -c /config/config.yaml ExecStop=-/usr/bin/docker kill matrix-mautrix-telegram ExecStop=-/usr/bin/docker rm matrix-mautrix-telegram diff --git a/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml b/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml index 4b67e2ba4..aff34f6dd 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml @@ -1,3 +1,16 @@ - set_fact: matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-whatsapp'] }}" when: matrix_mautrix_whatsapp_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + {{ ["--mount type=bind,src={{ matrix_mautrix_whatsapp_base_path }}/registration.yaml,dst=/matrix-mautrix-whatsapp-registration.yaml,ro"] }} + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-whatsapp-registration.yaml"] }} + when: matrix_mautrix_whatsapp_enabled|bool diff --git a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml index 38e4e035d..5c176e3a1 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml @@ -66,15 +66,3 @@ {{ matrix_mautrix_whatsapp_docker_image }} /usr/bin/mautrix-whatsapp -g -c /data/config.yaml -r /data/registration.yaml when: "not mautrix_whatsapp_registration_file_stat.stat.exists" - -# If the matrix-synapse role is not used, these variables may not exist. -- set_fact: - matrix_synapse_container_extra_arguments: > - {{ matrix_synapse_container_extra_arguments|default([]) }} - + - {{ ["--mount type=bind,src={{ matrix_mautrix_whatsapp_base_path }}/registration.yaml,dst=/matrix-mautrix-whatsapp-registration.yaml,ro"] }} - - matrix_synapse_app_service_config_files: > - {{ matrix_synapse_app_service_config_files|default([]) }} - + - {{ ["/matrix-mautrix-whatsapp-registration.yaml"] }} diff --git a/roles/matrix-mailer/defaults/main.yml b/roles/matrix-mailer/defaults/main.yml index 6e5b34675..f40420eaa 100644 --- a/roles/matrix-mailer/defaults/main.yml +++ b/roles/matrix-mailer/defaults/main.yml @@ -2,7 +2,7 @@ matrix_mailer_enabled: true matrix_mailer_base_path: "{{ matrix_base_data_path }}/mailer" -matrix_mailer_docker_image: "devture/exim-relay:4.91-r3-0" +matrix_mailer_docker_image: "devture/exim-relay:4.92-r0-0" matrix_mailer_docker_image_force_pull: "{{ matrix_mailer_docker_image.endswith(':latest') }}" # The user/group that the container runs with. diff --git a/roles/matrix-riot-web/templates/config.json.j2 b/roles/matrix-riot-web/templates/config.json.j2 index 92753a15f..e8c183b4a 100644 --- a/roles/matrix-riot-web/templates/config.json.j2 +++ b/roles/matrix-riot-web/templates/config.json.j2 @@ -14,7 +14,7 @@ "roomDirectory": { "servers": {{ matrix_riot_web_roomdir_servers|to_json }} }, - "welcomeUserId": {{ matrix_riot_web_welcome_user_id|string|to_json }}, + "welcomeUserId": {{ matrix_riot_web_welcome_user_id|to_json }}, {% if matrix_riot_web_enable_presence_by_hs_url is not none %} "enable_presence_by_hs_url": {{ matrix_riot_web_enable_presence_by_hs_url|to_json }}, {% endif %}