Merge branch 'master' into feature/add-worker-support

CHANGELOG.md

@@ -1,3 +1,12 @@
+# 2020-09-01
+
+## matrix-registration support
+
+The playbook can now help you set up [matrix-registration](https://github.com/ZerataX/matrix-registration) - an application that lets you keep your Matrix server's registration private, but still allow certain users (those having a unique registration link) to register by themselves.
+
+See our [Setting up matrix-registration](docs/configuring-playbook-matrix-registration.md) documentation page to get started.
+
+
 # 2020-08-21
 
 ## rust-synapse-compress-state support
@@ -87,7 +96,7 @@ To reuse your existing rooms, invite `@smsbot:yourServer` to the room or write a
 
 Thanks to [benkuly](https://github.com/benkuly)'s efforts, the playbook now supports bridging to SMS (with one telephone number only) via [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge).
 
-See our [Setting up Matrix SMS bridging](docs/configuring-playbook-matrix-bridge-sms.md) documentation page for getting started.
+See our [Setting up Matrix SMS bridging](docs/configuring-playbook-bridge-matrix-bridge-sms.md) documentation page for getting started.
 
 
 # 2020-05-19

README.md

@@ -60,17 +60,19 @@ Using this playbook, you can get the following services configured on your serve
 
 - (optional) the [mx-puppet-steam](https://github.com/icewind1991/mx-puppet-steam) bridge for [Steam](https://steamapp.com/)) - see [docs/configuring-playbook-bridge-mx-puppet-steam.md](docs/configuring-playbook-bridge-mx-puppet-steam.md) for setup documentation
 
-- (optional) the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) for bridging your Matrix server to SMS
+- (optional) the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) for bridging your Matrix server to SMS - see [docs/configuring-playbook-bridge-matrix-bridge-sms.md](docs/configuring-playbook-bridge-matrix-bridge-sms.md) for setup documentation
 
-- (optional) [Email2Matrix](https://github.com/devture/email2matrix) for relaying email messages to Matrix rooms
+- (optional) [Email2Matrix](https://github.com/devture/email2matrix) for relaying email messages to Matrix rooms - see [docs/configuring-playbook-email2matrix.md](docs/configuring-playbook-email2matrix.md) for setup documentation
 
-- (optional) [Dimension](https://github.com/turt2live/matrix-dimension), an open source integrations manager for matrix clients
+- (optional) [Dimension](https://github.com/turt2live/matrix-dimension), an open source integrations manager for matrix clients - see [docs/configuring-playbook-dimension.md](docs/configuring-playbook-dimension.md) for setup documentation
 
-- (optional) [Jitsi](https://jitsi.org/), an open source video-conferencing platform
+- (optional) [Jitsi](https://jitsi.org/), an open source video-conferencing platform - see [docs/configuring-playbook-jitsi.md](docs/configuring-playbook-jitsi.md) for setup documentation
 
-- (optional) [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) for scheduling one-off & recurring reminders and alarms
+- (optional) [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) for scheduling one-off & recurring reminders and alarms - see [docs/configuring-playbook-bot-matrix-reminder-bot.md](docs/configuring-playbook-bot-matrix-reminder-bot.md) for setup documentation
 
-- (optional) [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin), a web UI tool for administrating users and rooms on your Matrix server
+- (optional) [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin), a web UI tool for administrating users and rooms on your Matrix server - see [docs/configuring-playbook-synapse-admin.md](docs/configuring-playbook-synapse-admin.md) for setup documentation
+
+- (optional) [matrix-registration](https://github.com/ZerataX/matrix-registration), a simple python application to have a token based matrix registration - see [docs/configuring-playbook-matrix-registration.md](docs/configuring-playbook-matrix-registration.md) for setup documentation
 
 Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else.
 
@@ -140,6 +142,8 @@ This playbook sets up your server using the following Docker images:
 
 - [devture/matrix-corporal](https://hub.docker.com/r/devture/matrix-corporal/) - [Matrix Corporal](https://github.com/devture/matrix-corporal): reconciliator and gateway for a managed Matrix server (optional)
 
+- [devture/zeratax-matrix-registration](https://hub.docker.com/r/devture/zeratax-matrix-registration/) - [matrix-registration](https://github.com/ZerataX/matrix-registration): a simple python application to have a token based matrix registration (optional)
+
 - [nginx](https://hub.docker.com/_/nginx/) - the [nginx](http://nginx.org/) web server (optional)
 
 - [certbot/certbot](https://hub.docker.com/r/certbot/certbot/) - the [certbot](https://certbot.eff.org/) tool for obtaining SSL certificates from [Let's Encrypt](https://letsencrypt.org/) (optional)

docs/configuring-playbook-bridge-appservice-discord.md

@@ -23,7 +23,7 @@ matrix_appservice_discord_bot_token: "YOUR DISCORD APP BOT TOKEN"
 ```
 
 4. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready.
-5. Retrieve Discord invite link from the `{{ matrix_appservice_discord_config_path }}/invite_link` file on the server (this defaults to `/matrix/appservice-discord/config/invite_link`)
+5. Retrieve Discord invite link from the `{{ matrix_appservice_discord_config_path }}/invite_link` file on the server (this defaults to `/matrix/appservice-discord/config/invite_link`). You need to peek at the file on the server via SSH, etc., because it's not available via HTTP(S).
 6. Invite the Bot to Discord servers you wish to bridge. Administrator permission is recommended.
 7. Room addresses follow this syntax: `#_discord_guildid_channelid`. You can easily find the guild and channel ids by logging into Discord in a browser and opening the desired channel. The URL will have this format: `discordapp.com/channels/guild_id/channel_id`. Once you have figured out the appropriate room addrss, you can join by doing `/join #_discord_guildid_channelid` in your Matrix client.
 

docs/configuring-playbook-matrix-registration.md

@@ -0,0 +1,53 @@
+# Setting up matrix-registration (optional)
+
+The playbook can install and configure [matrix-registration](https://github.com/ZerataX/matrix-registration) for you.
+
+> matrix-registration is a simple python application to have a token based matrix registration.
+
+Use matrix-registration to **create unique registration links**, which people can use to register on your Matrix server. It allows you to **keep your server's registration closed (private)**, but still allow certain people (these having a special link) to register a user account.
+
+**matrix-registration** provides 2 things:
+
+- **an API for creating registration tokens** (unique registration links). This API can be used via `curl` or via the playbook (see [Usage](#usage) below)
+
+- **a user registration page**, where people can use these registration tokens. By default, exposed at `https:///matrix.DOMAIN/matrix-registration`
+
+
+## Installing
+
+Adjust your playbook configuration (your `inventory/host_vars/matrix.DOMAIN/vars.yml` file):
+
+```yaml
+matrix_registration_enabled: true
+
+# Generate a strong secret using: `pwgen -s 64 1`.
+matrix_registration_admin_secret: "ENTER_SOME_SECRET_HERE"
+```
+
+Then, run the [installation](installing.md) command again:
+
+```
+ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
+```
+
+
+## Usage
+
+**matrix-registration** gets exposed at `https:///matrix.DOMAIN/matrix-registration`
+
+It provides various [APIs](https://github.com/ZerataX/matrix-registration/wiki/api) - for creating registration tokens, listing tokens, disabling tokens, etc. To make use of all of its capabilities, consider using `curl`.
+
+We make the most common API (the one for creating unique registration tokens) easy to use via the playbook.
+
+**To create a new user registration token (link)**, use this command:
+
+```
+ansible-playbook -i inventory/hosts setup.yml \
+--tags=generate-matrix-registration-token \
+--extra-vars="one_time=yes ex_date=2021-12-31"
+```
+
+The above command creates and returns a **one-time use** token, which **expires** on the 31st of December 2021.
+Adjust the `one_time` and `ex_date` variables as you see fit.
+
+Share the unique registration link (generated by the command above) with users to let them register on your Matrix server.

docs/configuring-playbook-telemetry.md

@@ -10,7 +10,7 @@ growth of the Matrix community, and helps to make Matrix a success.
 
 ## Enabling Telemetry
 
-If you'd like to **help by enabling submission of anonymized usage statistics** for your homeserver, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
+If you'd like to **help by enabling submission of general usage statistics** for your homeserver, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
 
 ```yaml
 matrix_synapse_report_stats: true

docs/configuring-playbook.md

@@ -70,6 +70,8 @@ When you're done with all the configuration you'd like to do, continue with [Ins
 
 - [Setting up Synapse Admin](configuring-playbook-synapse-admin.md) (optional)
 
+- [Setting up matrix-registration](configuring-playbook-matrix-registration.md) (optional)
+
 - [Setting up the REST authentication password provider module](configuring-playbook-rest-auth.md) (optional, advanced)
 
 - [Setting up the Shared Secret Auth password provider module](configuring-playbook-shared-secret-auth.md) (optional, advanced)
@@ -113,7 +115,7 @@ When you're done with all the configuration you'd like to do, continue with [Ins
 
 - [Setting up Email2Matrix](configuring-playbook-email2matrix.md) (optional)
 
-- [Setting up Matrix SMS bridging](configuring-playbook-matrix-bridge-sms.md) (optional)
+- [Setting up Matrix SMS bridging](configuring-playbook-bridge-matrix-bridge-sms.md) (optional)
 
 
 ### Bots

docs/maintenance-postgres.md

@@ -42,6 +42,7 @@ To make a back up of the current PostgreSQL database, make sure it's running and
 ```bash
 docker run \
 --rm \
+--log-driver=none \
 --network=matrix \
 --env-file=/matrix/postgres/env-postgres-psql \
 postgres:12.4-alpine \

docs/registering-users.md

@@ -1,6 +1,18 @@
 # Registering users
 
-Run this to create a new user account on your Matrix server.
+This documentation page tells you how to create user account on your Matrix server.
+
+Table of contents:
+
+- [Registering users](#registering-users)
+	- [Registering users manually](#registering-users-manually)
+	- [Managing users via a Web UI](#managing-users-via-a-web-ui)
+	- [Letting certain users register on your private server](#letting-certain-users-register-on-your-private-server)
+	- [Enabling public user registration](#enabling-public-user-registration)
+	- [Adding/Removing Administrator privileges to an existing user](#addingremoving-administrator-privileges-to-an-existing-user)
+
+
+## Registering users manually
 
 You can do it via this Ansible playbook (make sure to edit the `<your-username>` and `<your-password>` part below):
 
@@ -22,10 +34,31 @@ ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=<your-usern
 
 If you've just installed Matrix, **to finalize the installation process**, it's best if you proceed to [Configuring service discovery via .well-known](configuring-well-known.md)
 
------
+
+## Managing users via a Web UI
+
+To manage users more easily (via a web user-interace), you can install [Synapse Admin](configuring-playbook-synapse-admin.md).
 
 
-## Adding/Removing Administrator privileges to an existing user.
+## Letting certain users register on your private server
+
+If you'd rather **keep your server private** (public registration closed, as is the default), and **let certain people create accounts by themselves** (instead of creating user accounts manually like this), consider installing and making use of [matrix-registration](configuring-playbook-matrix-registration.md).
+
+
+## Enabling public user registration
+
+To **open up user registration publicly** (usually **not recommended**), consider using the following configuration:
+
+```yaml
+matrix_synapse_enable_registration: true
+```
+
+and running the [installation](installing.md) procedure once again.
+
+If you're opening up registrations publicly like this, you might also wish to [configure CAPTCHA protection](configuring-captcha.md).
+
+
+## Adding/Removing Administrator privileges to an existing user
 
 The script `/usr/local/bin/matrix-change-user-admin-status` may be used to change a user's admin privileges.
 
@@ -35,8 +68,3 @@ The script `/usr/local/bin/matrix-change-user-admin-status` may be used to chang
 ```
 /usr/local/bin/matrix-change-user-admin-status <username> <0/1>
 ```
-
-
-## Managing users via a Web UI
-
-To manage users more easily (via a web user-interace), you can install [Synapse Admin](configuring-playbook-synapse-admin.md).

docs/self-building.md

@@ -13,6 +13,7 @@ List of roles where self-building the Docker image is currently possible:
 - `matrix-synapse`
 - `matrix-synapse-admin`
 - `matrix-client-element`
+- `matrix-registration`
 - `matrix-coturn`
 - `matrix-ma1sd`
 - `matrix-mailer`

group_vars/matrix_servers

@@ -737,7 +737,7 @@ matrix_ma1sd_threepid_medium_email_connectors_smtp_host: "matrix-mailer"
 matrix_ma1sd_threepid_medium_email_connectors_smtp_port: 8025
 matrix_ma1sd_threepid_medium_email_connectors_smtp_tls: 0
 
-matrix_ma1sd_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else false }}"
+matrix_ma1sd_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}"
 
 matrix_ma1sd_systemd_wanted_services_list: |
   {{
@@ -799,7 +799,7 @@ matrix_nginx_proxy_proxy_matrix_user_directory_search_enabled: "{{ matrix_ma1sd_
 matrix_nginx_proxy_proxy_matrix_user_directory_search_addr_with_container: "{{ matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container }}"
 matrix_nginx_proxy_proxy_matrix_user_directory_search_addr_sans_container: "{{ matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container }}"
 
-matrix_nginx_proxy_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else false }}"
+matrix_nginx_proxy_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}"
 
 matrix_nginx_proxy_systemd_wanted_services_list: |
   {{
@@ -906,7 +906,7 @@ matrix_client_element_integrations_rest_url: "{{ matrix_dimension_integrations_r
 matrix_client_element_integrations_widgets_urls: "{{ matrix_dimension_integrations_widgets_urls if matrix_dimension_enabled else ['https://scalar.vector.im/api'] }}"
 matrix_client_element_integrations_jitsi_widget_url: "{{ matrix_dimension_integrations_jitsi_widget_url if matrix_dimension_enabled else 'https://scalar.vector.im/api/widgets/jitsi.html' }}"
 
-matrix_client_element_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else false }}"
+matrix_client_element_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}"
 
 matrix_client_element_registration_enabled: "{{ matrix_synapse_enable_registration }}"
 
@@ -1001,7 +1001,7 @@ matrix_synapse_turn_uris: |
 
 matrix_synapse_turn_shared_secret: "{{ matrix_coturn_turn_static_auth_secret if matrix_coturn_enabled else '' }}"
 
-matrix_synapse_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else false }}"
+matrix_synapse_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}"
 
 matrix_synapse_systemd_required_services_list: |
   {{
@@ -1050,3 +1050,34 @@ matrix_synapse_admin_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy
 # /matrix-synapse-admin
 #
 ######################################################################
+
+
+
+######################################################################
+#
+# matrix-registration
+#
+######################################################################
+
+matrix_registration_enabled: false
+
+# Normally, matrix-nginx-proxy is enabled and nginx can reach matrix-registration over the container network.
+# If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose
+# matrix-registration's HTTP port to the local host.
+matrix_registration_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else '127.0.0.1:8767' }}"
+
+matrix_registration_riot_instance: "{{ ('https://' + matrix_server_fqn_element) if matrix_client_element_enabled else 'https://riot.im/app/' }}"
+
+matrix_registration_shared_secret: "{{ matrix_synapse_registration_shared_secret if matrix_synapse_enabled else '' }}"
+
+matrix_registration_server_location: "{{ 'http://matrix-synapse:8008' if matrix_synapse_enabled else '' }}"
+
+matrix_registration_api_validate_certs: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}"
+
+matrix_registration_container_image_self_build: "{{ matrix_architecture != 'amd64' }}"
+
+######################################################################
+#
+# /matrix-registration
+#
+######################################################################

roles/matrix-client-element/defaults/main.yml

@@ -2,7 +2,7 @@ matrix_client_element_enabled: true
 
 matrix_client_element_container_image_self_build: false
 
-matrix_client_element_docker_image: "vectorim/riot-web:v1.7.4"
+matrix_client_element_docker_image: "vectorim/riot-web:v1.7.5"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"
 
 matrix_client_element_data_path: "{{ matrix_base_data_path }}/client-element"

roles/matrix-common-after/tasks/dump_runtime_results.yml

@@ -0,0 +1,6 @@
+# Ansible outputs the message in the `item=` field.
+# It's unnecessary to output it again in the actual message, so we don't.
+- debug:
+    msg: ""
+  with_items: "{{ matrix_playbook_runtime_results }}"
+  when: "matrix_playbook_runtime_results is defined and matrix_playbook_runtime_results|length > 0"

roles/matrix-common-after/tasks/main.yml

@@ -7,3 +7,7 @@
   when: run_stop|bool
   tags:
     - stop
+
+- import_tasks: "{{ role_path }}/tasks/dump_runtime_results.yml"
+  tags:
+    - always

roles/matrix-coturn/tasks/setup_coturn.yml

@@ -91,15 +91,10 @@
 # We optimize for the common use-case though (short-lived Let's Encrypt certificates).
 # Reloading doesn't hurt anyway, so there's no need to make this more flexible.
 - name: Ensure periodic reloading of matrix-coturn is configured for SSL renewal (matrix-coturn-reload)
-  cron:
+  template:
-    user: root
+    src: "{{ role_path }}/templates/cron.d/matrix-coturn-ssl-reload.j2"
-    cron_file: matrix-coturn-ssl-reload
+    dest: /etc/cron.d/matrix-coturn-ssl-reload
-    name: matrix-coturn-ssl-reload
+    mode: 0644
-    state: present
-    hour: "4"
-    minute: "20"
-    day: "*/5"
-    job: "{{ matrix_host_command_systemctl }} reload matrix-coturn.service"
   when: "matrix_coturn_enabled|bool and matrix_coturn_tls_enabled|bool"
 
 
@@ -108,9 +103,8 @@
 #
 
 - name: Ensure matrix-coturn-ssl-reload cronjob removed
-  cron:
+  file:
-    user: root
+    path: /etc/cron.d/matrix-coturn-ssl-reload
-    cron_file: matrix-coturn-ssl-reload
     state: absent
   when: "not matrix_coturn_enabled|bool or not matrix_coturn_tls_enabled|bool"
 

roles/matrix-coturn/templates/cron.d/matrix-coturn-ssl-reload.j2

@@ -0,0 +1 @@
+20 4 */5 * * root {{ matrix_host_command_systemctl }} reload matrix-coturn.service

roles/matrix-jitsi/defaults/main.yml

@@ -49,6 +49,8 @@ matrix_jitsi_jibri_xmpp_password: ''
 matrix_jitsi_jibri_recorder_user: recorder
 matrix_jitsi_jibri_recorder_password: ''
 
+matrix_jitsi_enable_lobby: false
+
 matrix_jitsi_container_image_tag: "stable-4857"
 
 matrix_jitsi_web_docker_image: "jitsi/web:{{ matrix_jitsi_container_image_tag }}"

roles/matrix-jitsi/templates/prosody/env.j2

@@ -42,4 +42,6 @@ JIBRI_XMPP_PASSWORD={{ matrix_jitsi_jibri_xmpp_password }}
 JIBRI_RECORDER_USER={{ matrix_jitsi_jibri_recorder_user }}
 JIBRI_RECORDER_PASSWORD={{ matrix_jitsi_jibri_recorder_password }}
 
+ENABLE_LOBBY={{ 1 if matrix_jitsi_enable_lobby else 0 }}
+
 TZ={{ matrix_jitsi_timezone }}

roles/matrix-nginx-proxy/tasks/ssl/setup_ssl_lets_encrypt.yml

@@ -55,37 +55,11 @@
     mode: 0750
   when: "matrix_ssl_retrieval_method == 'lets-encrypt'"
 
-- block:
+- name: Ensure periodic SSL renewal cronjob configured
-  - name: Ensure periodic SSL renewal cronjob configured (MAILTO)
+  template:
-    cron:
+    src: "{{ role_path }}/templates/cron.d/matrix-ssl-lets-encrypt.j2"
-      user: root
+    dest: /etc/cron.d/matrix-ssl-lets-encrypt
-      cron_file: matrix-ssl-lets-encrypt
+    mode: 0644
-      env: yes
-      name: MAILTO
-      value: "{{ matrix_ssl_lets_encrypt_support_email }}"
-
-  - name: Ensure periodic SSL renewal cronjob configured (matrix-ssl-lets-encrypt-certificates-renew)
-    cron:
-      user: root
-      cron_file: matrix-ssl-lets-encrypt
-      name: matrix-ssl-lets-encrypt-certificates-renew
-      state: present
-      hour: "4"
-      minute: "15"
-      day: "*"
-      job: "{{ matrix_local_bin_path }}/matrix-ssl-lets-encrypt-certificates-renew"
-
-  - name: Ensure periodic reloading of matrix-nginx-proxy is configured for SSL renewal (matrix-nginx-proxy-reload)
-    cron:
-      user: root
-      cron_file: matrix-ssl-lets-encrypt
-      name: matrix-nginx-proxy-reload
-      state: present
-      hour: "5"
-      minute: "20"
-      day: "*"
-      job: "{{ matrix_host_command_systemctl }} reload matrix-nginx-proxy.service"
-    when: matrix_nginx_proxy_enabled|bool
   when: "matrix_ssl_retrieval_method == 'lets-encrypt'"
 
 
@@ -93,21 +67,9 @@
 # Tasks related to getting rid of Let's Encrypt's management of certificates
 #
 
-# When nginx-proxy is disabled, make sure its reloading cronjob is gone.
-# Other cronjobs can potentially remain there (see below).
-- name: Ensure matrix-nginx-proxy-reload cronjob removed
-  cron:
-    user: root
-    cron_file: matrix-ssl-lets-encrypt
-    name: matrix-nginx-proxy-reload
-    state: absent
-  when: "not matrix_nginx_proxy_enabled|bool"
-
 - name: Ensure matrix-ssl-lets-encrypt-renew cronjob removed
-  cron:
+  file:
-    user: root
+    path: /etc/cron.d/matrix-ssl-lets-encrypt
-    cron_file: matrix-ssl-lets-encrypt
-    name: matrix-ssl-lets-encrypt-certificates-renew
     state: absent
   when: "matrix_ssl_retrieval_method != 'lets-encrypt'"
 

roles/matrix-nginx-proxy/templates/cron.d/matrix-ssl-lets-encrypt.j2

@@ -0,0 +1,5 @@
+MAILTO="{{ matrix_ssl_lets_encrypt_support_email }}"
+15 4 * * * root {{ matrix_local_bin_path }}/matrix-ssl-lets-encrypt-certificates-renew
+{% if matrix_nginx_proxy_enabled %}
+20 5 * * * root {{ matrix_host_command_systemctl }} reload matrix-nginx-proxy.service
+{% endif %}

roles/matrix-postgres/tasks/import_postgres.yml

@@ -64,6 +64,7 @@
   set_fact:
     matrix_postgres_import_command: >-
       {{ matrix_host_command_docker }} run --rm --name matrix-postgres-import
+      --log-driver=none
       --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
       --cap-drop=ALL
       --network={{ matrix_docker_network }}

roles/matrix-postgres/tasks/import_sqlite_db.yml

@@ -74,6 +74,7 @@
     docker run
     --rm
     --name=matrix-synapse-migrate
+    --log-driver=none
     --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
     --cap-drop=ALL
     --network={{ matrix_docker_network }}

roles/matrix-postgres/tasks/setup_postgres.yml

@@ -18,9 +18,16 @@
     matrix_postgres_docker_image_to_use: "{{ matrix_postgres_docker_image_latest if matrix_postgres_detected_version_corresponding_docker_image == '' else matrix_postgres_detected_version_corresponding_docker_image }}"
   when: matrix_postgres_enabled|bool
 
-- name: Warn if on an old version of Postgres
+- name: Inject warning if on an old version of Postgres
-  debug:
+  set_fact:
-    msg: "NOTE: Your setup is on an old Postgres version ({{ matrix_postgres_docker_image_to_use }}), while {{ matrix_postgres_docker_image_latest }} is supported. You can upgrade using --tags=upgrade-postgres"
+    matrix_playbook_runtime_results: |
+      {{
+        matrix_playbook_runtime_results|default([])
+        +
+        [
+          "NOTE: Your setup is on an old Postgres version ({{ matrix_postgres_docker_image_to_use }}), while {{ matrix_postgres_docker_image_latest }} is supported. You can upgrade using --tags=upgrade-postgres"
+        ]
+      }}
   when: "matrix_postgres_enabled|bool and matrix_postgres_docker_image_to_use != matrix_postgres_docker_image_latest"
 
 # Even if we don't run the internal server, we still need this for running the CLI

roles/matrix-postgres/tasks/upgrade_postgres.yml

@@ -80,6 +80,7 @@
 - name: Perform Postgres database dump
   command: >-
     {{ matrix_host_command_docker }} run --rm --name matrix-postgres-dump
+    --log-driver=none
     --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
     --network={{ matrix_docker_network }}
     --env-file={{ matrix_postgres_base_path }}/env-postgres-psql
@@ -124,6 +125,7 @@
   set_fact:
     matrix_postgres_import_command: >-
       {{ matrix_host_command_docker }} run --rm --name matrix-postgres-import
+      --log-driver=none
       --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
       --cap-drop=ALL
       --network={{ matrix_docker_network }}

roles/matrix-registration/defaults/main.yml

@@ -0,0 +1,83 @@
+# matrix-registration is a simple python application to have a token based matrix registration
+# See: https://zeratax.github.io/matrix-registration/
+
+matrix_registration_enabled: true
+
+matrix_registration_container_image_self_build: false
+
+matrix_registration_base_path: "{{ matrix_base_data_path }}/matrix-registration"
+matrix_registration_config_path: "{{ matrix_registration_base_path }}/config"
+matrix_registration_data_path: "{{ matrix_registration_base_path }}/data"
+matrix_registration_docker_src_files_path: "{{ matrix_registration_base_path }}/docker-src"
+
+matrix_registration_version: "v0.7.0"
+
+matrix_registration_docker_image: "devture/zeratax-matrix-registration:{{ matrix_registration_version }}"
+matrix_registration_docker_image_force_pull: "{{ matrix_registration_docker_image.endswith(':latest') }}"
+matrix_registration_docker_repo: "https://github.com/ZerataX/matrix-registration"
+
+# A list of extra arguments to pass to the container
+matrix_registration_container_extra_arguments: []
+
+# List of systemd services that matrix-registration.service depends on
+matrix_registration_systemd_required_services_list: ['docker.service']
+
+# List of systemd services that matrix-registration.service wants
+matrix_registration_systemd_wanted_services_list: []
+
+# Controls whether the matrix-registration container exposes its HTTP port (tcp/5000 in the container).
+#
+# Takes an "<ip>:<port>" or "<port>" value (e.g. "127.0.0.1:8767"), or empty string to not expose.
+matrix_registration_container_http_host_bind_port: ''
+
+# The path at which Matrix Registration will be exposed on `matrix.DOMAIN`
+# (only applies when matrix-nginx-proxy is used).
+matrix_registration_public_endpoint: /matrix-registration
+
+matrix_registration_api_register_endpoint: "{{ matrix_homeserver_url }}{{ matrix_registration_public_endpoint }}/register"
+matrix_registration_api_token_endpoint: "{{ matrix_homeserver_url }}{{ matrix_registration_public_endpoint }}/token"
+
+matrix_registration_api_validate_certs: true
+
+# The URL to your homeserver (e.g.: `https://matrix.DOMAIN`).
+# A local (in-container address) is preferable.
+matrix_registration_server_location: ""
+
+matrix_registration_server_name: "{{ matrix_domain }}"
+
+# matrix_registration_shared_secret needs to match the homeserver's registration secret.
+# For Synapse, that's the `registration_shared_secret` setting.
+matrix_registration_shared_secret: ""
+
+# matrix_registration_admin_secret is your own admin secret for using matrix-registration (creating new tokens, etc.)
+matrix_registration_admin_secret: ""
+
+matrix_registration_riot_instance: "https://riot.im/app/"
+
+
+# Default matrix-registration configuration template which covers the generic use case.
+# You can customize it by controlling the various variables inside it.
+#
+# For a more advanced customization, you can extend the default (see `matrix_registration_configuration_extension_yaml`)
+# or completely replace this variable with your own template.
+matrix_registration_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}"
+
+matrix_registration_configuration_extension_yaml: |
+  # Your custom YAML configuration for registration goes here.
+  # This configuration extends the default starting configuration (`matrix_registration_configuration_yaml`).
+  #
+  # You can override individual variables from the default configuration, or introduce new ones.
+  #
+  # If you need something more special, you can take full control by
+  # completely redefining `matrix_registration_configuration_yaml`.
+  #
+  # Example configuration extension follows:
+  #
+  # password:
+  #   min_length: 12
+
+matrix_registration_configuration_extension: "{{ matrix_registration_configuration_extension_yaml|from_yaml if matrix_registration_configuration_extension_yaml|from_yaml is mapping else {} }}"
+
+# Holds the final matrix-registration configuration (a combination of the default and its extension).
+# You most likely don't need to touch this variable. Instead, see `matrix_registration_configuration_yaml`.
+matrix_registration_configuration: "{{ matrix_registration_configuration_yaml|from_yaml|combine(matrix_registration_configuration_extension, recursive=True) }}"

roles/matrix-registration/tasks/generate_token.yml

@@ -0,0 +1,50 @@
+- name: Fail if playbook called incorrectly
+  fail:
+    msg: "The `one_time` variable needs to be provided to this playbook, via --extra-vars"
+  when: "one_time is not defined or one_time not in ['yes', 'no']"
+
+- name: Fail if playbook called incorrectly
+  fail:
+    msg: "The `ex_date` variable (expiration date) needs to be provided to this playbook, via --extra-vars"
+  when: "ex_date is not defined or ex_date == '<date>'"
+
+- name: Call matrix-registration token creation API
+  uri:
+    url: "{{ matrix_registration_api_token_endpoint }}"
+    follow_redirects: none
+    validate_certs: "{{ matrix_registration_api_validate_certs }}"
+    headers:
+      Content-Type: application/json
+      Authorization: "SharedSecret {{ matrix_registration_admin_secret }}"
+    method: POST
+    body_format: json
+    body: |
+      {
+        "one_time": {{ 'true' if one_time == 'yes' else 'false' }},
+        "ex_date": {{ ex_date|to_json }}
+      }
+  check_mode: no
+  register: matrix_registration_api_result
+
+- set_fact:
+    matrix_registration_api_result_message: >-
+      matrix-registration result:
+
+      Direct registration link (with the token prefilled):
+
+      {{ matrix_registration_api_register_endpoint }}?token={{ matrix_registration_api_result.json.name }}
+
+      Full token details are:
+
+      {{ matrix_registration_api_result.json }}
+  check_mode: no
+
+- name: Inject result message into matrix_playbook_runtime_results
+  set_fact:
+    matrix_playbook_runtime_results: |
+      {{
+        matrix_playbook_runtime_results|default([])
+        +
+        [matrix_registration_api_result_message]
+      }}
+  check_mode: no

roles/matrix-registration/tasks/init.yml

@@ -0,0 +1,64 @@
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-registration'] }}"
+  when: matrix_registration_enabled|bool
+
+- block:
+  - name: Fail if matrix-nginx-proxy role already executed
+    fail:
+      msg: >-
+        Trying to append matrix-registration's reverse-proxying configuration to matrix-nginx-proxy,
+        but it's pointless since the matrix-nginx-proxy role had already executed.
+        To fix this, please change the order of roles in your plabook,
+        so that the matrix-nginx-proxy role would run after the matrix-registration role.
+    when: matrix_nginx_proxy_role_executed|default(False)|bool
+
+  - name: Generate matrix-registration proxying configuration for matrix-nginx-proxy
+    set_fact:
+      matrix_registration_matrix_nginx_proxy_configuration: |
+        rewrite ^{{ matrix_registration_public_endpoint }}$ $scheme://$server_name{{ matrix_registration_public_endpoint }}/ permanent;
+        rewrite ^{{ matrix_registration_public_endpoint }}/$ $scheme://$server_name{{ matrix_registration_public_endpoint }}/register redirect;
+
+        location ~ ^{{ matrix_registration_public_endpoint }}/(.*) {
+        {% if matrix_nginx_proxy_enabled|default(False) %}
+          {# Use the embedded DNS resolver in Docker containers to discover the service #}
+          resolver 127.0.0.11 valid=5s;
+          set $backend "matrix-registration:5000";
+          proxy_pass http://$backend/$1;
+        {% else %}
+          {# Generic configuration for use outside of our container setup #}
+          proxy_pass http://127.0.0.1:8767/$1;
+        {% endif %}
+
+          {#
+            Workaround matrix-registration serving static files at /static
+            (see https://github.com/ZerataX/matrix-registration/issues/29)
+
+            Also fixing the form, which goes to /register.
+          #}
+          sub_filter_once off;
+          sub_filter_types text/html text/css;
+          sub_filter "/static/" "{{ matrix_registration_public_endpoint }}/static/";
+          sub_filter "/register" "{{ matrix_registration_public_endpoint }}/register";
+        }
+
+  - name: Register matrix-registration proxying configuration with matrix-nginx-proxy
+    set_fact:
+      matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: |
+        {{
+          matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks|default([])
+          +
+          [matrix_registration_matrix_nginx_proxy_configuration]
+        }}
+  tags:
+    - always
+  when: matrix_registration_enabled|bool
+
+- name: Warn about reverse-proxying if matrix-nginx-proxy not used
+  debug:
+    msg: >-
+      NOTE: You've enabled the matrix-registration tool but are not using the matrix-nginx-proxy
+      reverse proxy.
+      Please make sure that you're proxying the `{{ matrix_registration_public_endpoint }}`
+      URL endpoint to the matrix-registration container.
+      You can expose the container's port using the `matrix_registration_container_http_host_bind_port` variable.
+  when: "matrix_registration_enabled|bool and matrix_nginx_proxy_enabled is not defined"

roles/matrix-registration/tasks/main.yml

@@ -0,0 +1,19 @@
+- import_tasks: "{{ role_path }}/tasks/init.yml"
+  tags:
+    - always
+
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: "run_setup|bool and matrix_registration_enabled|bool"
+  tags:
+    - setup-all
+    - setup-matrix-registration
+
+- import_tasks: "{{ role_path }}/tasks/setup.yml"
+  tags:
+    - setup-all
+    - setup-matrix-registration
+
+- import_tasks: "{{ role_path }}/tasks/generate_token.yml"
+  when: "run_setup|bool and matrix_registration_enabled|bool"
+  tags:
+    - generate-matrix-registration-token

roles/matrix-registration/tasks/setup.yml

@@ -0,0 +1,103 @@
+---
+
+#
+# Tasks related to setting up matrix-registration
+#
+
+- name: Ensure matrix-registration paths exist
+  file:
+    path: "{{ item.path }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - { path: "{{ matrix_registration_base_path }}", when: true }
+    - { path: "{{ matrix_registration_config_path }}", when: true }
+    - { path: "{{ matrix_registration_data_path }}", when: true }
+    - { path: "{{ matrix_registration_docker_src_files_path }}", when: "{{ matrix_registration_container_image_self_build }}"}
+  when: matrix_registration_enabled|bool and item.when
+
+- name: Ensure matrix-registration image is pulled
+  docker_image:
+    name: "{{ matrix_registration_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_registration_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_registration_docker_image_force_pull }}"
+  when: "matrix_registration_enabled|bool and not matrix_registration_container_image_self_build|bool"
+
+- name: Ensure matrix-registration repository is present when self-building
+  git:
+    repo: "{{ matrix_registration_docker_repo }}"
+    dest: "{{ matrix_registration_docker_src_files_path }}"
+    version: "{{ matrix_registration_version }}"
+    force: "yes"
+  register: matrix_registration_git_pull_results
+  when: "matrix_registration_enabled|bool and matrix_registration_container_image_self_build|bool"
+
+- name: Ensure matrix-registration Docker image is built
+  docker_image:
+    name: "{{ matrix_registration_docker_image }}"
+    source: build
+    force_source: yes
+    build:
+      dockerfile: Dockerfile
+      path: "{{ matrix_registration_docker_src_files_path }}"
+      pull: yes
+  when: "matrix_registration_enabled|bool and matrix_registration_container_image_self_build|bool and matrix_registration_git_pull_results.changed"
+
+- name: Ensure matrix-registration config installed
+  copy:
+    content: "{{ matrix_registration_configuration|to_nice_yaml }}"
+    dest: "{{ matrix_registration_config_path }}/config.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  when: matrix_registration_enabled|bool
+
+- name: Ensure matrix-registration.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-registration.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-registration.service"
+    mode: 0644
+  register: matrix_registration_systemd_service_result
+  when: matrix_registration_enabled|bool
+
+- name: Ensure systemd reloaded after matrix-registration.service installation
+  service:
+    daemon_reload: yes
+  when: "matrix_registration_enabled|bool and matrix_registration_systemd_service_result.changed"
+
+#
+# Tasks related to getting rid of matrix-registration (if it was previously enabled)
+#
+
+- name: Check existence of matrix-registration service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-registration.service"
+  register: matrix_registration_service_stat
+
+- name: Ensure matrix-registration is stopped
+  service:
+    name: matrix-registration
+    state: stopped
+    daemon_reload: yes
+  register: stopping_result
+  when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists"
+
+- name: Ensure matrix-registration.service doesn't exist
+  file:
+    path: "{{ matrix_systemd_path }}/matrix-registration.service"
+    state: absent
+  when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists"
+
+- name: Ensure systemd reloaded after matrix-registration.service removal
+  service:
+    daemon_reload: yes
+  when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists"
+
+- name: Ensure matrix-registration Docker image doesn't exist
+  docker_image:
+    name: "{{ matrix_registration_docker_image }}"
+    state: absent
+  when: "not matrix_registration_enabled|bool"

roles/matrix-registration/tasks/validate_config.yml

@@ -0,0 +1,11 @@
+---
+
+- name: Fail if required matrix-registration settings not defined
+  fail:
+    msg: >
+      You need to define a required configuration setting (`{{ item }}`) for using matrix-registration.
+  when: "vars[item] == ''"
+  with_items:
+    - "matrix_registration_shared_secret"
+    - "matrix_registration_admin_secret"
+    - "matrix_registration_server_location"

roles/matrix-registration/templates/config.yaml.j2

@@ -0,0 +1,30 @@
+server_location: {{ matrix_registration_server_location|to_json }}
+server_name: {{ matrix_registration_server_name|to_json }}
+shared_secret: {{ matrix_registration_shared_secret|to_json }}
+admin_secret: {{ matrix_registration_admin_secret|to_json }}
+riot_instance: {{ matrix_registration_riot_instance|to_json }}
+db: 'sqlite:////data/db.sqlite3'
+host: '0.0.0.0'
+port: 5000
+rate_limit: ["100 per day", "10 per minute"]
+allow_cors: false
+logging:
+  disable_existing_loggers: False
+  version: 1
+  root:
+    level: DEBUG
+    handlers: [console]
+  formatters:
+    brief:
+      format: '%(name)s - %(levelname)s - %(message)s'
+    precise:
+      format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+  handlers:
+    console:
+      class: logging.StreamHandler
+      level: INFO
+      formatter: brief
+      stream: ext://sys.stdout
+# password requirements
+password:
+  min_length: 8

roles/matrix-registration/templates/systemd/matrix-registration.service.j2

@@ -0,0 +1,40 @@
+#jinja2: lstrip_blocks: "True"
+[Unit]
+Description=matrix-registration
+{% for service in matrix_registration_systemd_required_services_list %}
+Requires={{ service }}
+After={{ service }}
+{% endfor %}
+{% for service in matrix_registration_systemd_wanted_services_list %}
+Wants={{ service }}
+{% endfor %}
+
+[Service]
+Type=simple
+ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-registration
+ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-registration
+
+ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-registration \
+			--log-driver=none \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--network={{ matrix_docker_network }} \
+			{% if matrix_registration_container_http_host_bind_port %}
+			-p {{ matrix_registration_container_http_host_bind_port }}:5000 \
+			{% endif %}
+			-v {{ matrix_registration_config_path }}:/config:ro \
+			-v {{ matrix_registration_data_path }}:/data \
+			{% for arg in matrix_registration_container_extra_arguments %}
+			{{ arg }} \
+			{% endfor %}
+			{{ matrix_registration_docker_image }} \
+			serve
+
+ExecStop=-{{ matrix_host_command_docker }} kill matrix-registration
+ExecStop=-{{ matrix_host_command_docker }} rm matrix-registration
+Restart=always
+RestartSec=30
+SyslogIdentifier=matrix-registration
+
+[Install]
+WantedBy=multi-user.target

roles/matrix-synapse-admin/tasks/setup.yml

@@ -20,7 +20,7 @@
   register: matrix_synapse_admin_git_pull_results
   when: "matrix_synapse_admin_enabled|bool and matrix_synapse_admin_container_self_build|bool"
 
-- name: Ensure matrix-synapse-admin Docker image is build
+- name: Ensure matrix-synapse-admin Docker image is built
   docker_image:
     name: "{{ matrix_synapse_admin_docker_image }}"
     source: build

setup.yml

@@ -27,6 +27,7 @@
     - matrix-bot-matrix-reminder-bot
     - matrix-synapse
     - matrix-synapse-admin
+    - matrix-registration
     - matrix-client-element
     - matrix-jitsi
     - matrix-ma1sd