Make matrix-nginx-proxy role independent of others

The matrix-nginx-proxy role can now be used independently.
This makes it consistent with all other roles, with
the `matrix-base` role remaining as their only dependency.

Separating matrix-nginx-proxy was relatively straightforward, with
the exception of the Mautrix Telegram reverse-proxying configuration.
Mautrix Telegram, being an extension/bridge, does not feel important enough
to justify its own special handling in matrix-nginx-proxy.

Thus, we've introduced the concept of "additional configuration blocks"
(`matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks`),
where any module can register its own custom nginx server blocks.

For such dynamic registration to work, the order of role execution
becomes important. To make it possible for each module participating
in dynamic registration to verify that the order of execution is
correct, we've also introduced a `matrix_nginx_proxy_role_executed`
variable.

It should be noted that this doesn't make the matrix-synapse role
dependent on matrix-nginx-proxy. It's optional runtime detection
and registration, and it only happens in the matrix-synapse role
when `matrix_mautrix_telegram_enabled: true`.

CHANGELOG.md

@@ -8,21 +8,12 @@ The following change **affects people running a more non-standard setup** - exte
 Because people like using the playbook's components independently (outside of this playbook) and because it's much better for maintainability, we've continued working on separating them.
 Still, we'd like to offer a turnkey solution for running a fully-featured Matrix server, so this playbook remains important for wiring up the various components.
 
-With the new changes, the following roles are now only dependent on the minimal `matrix-base` role:
+With the new changes, **all roles are now only dependent on the minimal `matrix-base` role**. They are no longer dependent among themselves.
-- `matrix-corporal`
-- `matrix-coturn`
-- `matrix-mailer`
-- `matrix-mxisd`
-- `matrix-postgres`
-- `matrix-riot-web`
-- `matrix-synapse`
 
-The `matrix-nginx-proxy` role still does too much and remains dependent on the others.
+In addition, the following components can now be completely disabled (for those who want/need to):
-
+- `matrix-coturn` by using `matrix_coturn_enabled: false`
-In addition, the following components can be completely disabled now (for those who want/need to):
+- `matrix-mailer` by using `matrix_mailer_enabled: false`
-- `matrix-coturn`
+- `matrix-postgres` by using `matrix_postgres_enabled: false`
-- `matrix-mailer`
-- `matrix-postgres`
 
 The following changes had to be done:
 
@@ -30,7 +21,7 @@ The following changes had to be done:
 
 - `matrix_postgres_use_external` has been superceeded by the more consistently named `matrix_postgres_enabled` variable and a few other `matrix_synapse_database_` variables. See the [Using an external PostgreSQL server (optional)](docs/configuring-playbook-external-postgres.md) documentation page for an up-to-date replacement.
 
-- Postgres tools (`matrix-postgres-cli` and `matrix-make-user-admin`) are no longer installed if you're using an external Postgres server (`matrix_postgres_enabled: false`)
+- Postgres tools (`matrix-postgres-cli` and `matrix-make-user-admin`) are no longer installed if you're not enabling the `matrix-postgres` role (`matrix_postgres_enabled: false`)
 
 - roles, being more independent now, are more minimal and do not do so much magic for you. People that are building their own playbook using our roles will definitely need to take a look at the [`group_vars/matrix-servers`](group_vars/matrix-servers) file and adapt their playbooks with the same (or similar) wiring logic.
 

group_vars/matrix-servers

@@ -149,12 +149,22 @@ matrix_mxisd_systemd_wanted_services_list: |
 # This is fine if you're dedicating the whole server to Matrix.
 # If that's not the case, you may wish to disable this and take care of proxying yourself.
 matrix_nginx_proxy_enabled: true
-matrix_nginx_proxy_matrix_client_api_addr_with_proxy_container: "{{ 'matrix-corporal:41080' if matrix_corporal_enabled else 'matrix-synapse:8008' }}"
+
-matrix_nginx_proxy_matrix_client_api_addr_sans_proxy_container: "{{ 'localhost:41080' if matrix_corporal_enabled else 'localhost:8008' }}"
+matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container: "{{ 'matrix-corporal:41080' if matrix_corporal_enabled else 'matrix-synapse:8008' }}"
+matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container: "{{ 'localhost:41080' if matrix_corporal_enabled else 'localhost:8008' }}"
+matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size: "{{ matrix_synapse_max_upload_size_mb }}M"
 
 matrix_nginx_proxy_proxy_matrix_enabled: true
 matrix_nginx_proxy_proxy_riot_enabled: "{{ matrix_riot_web_enabled }}"
 
+matrix_nginx_proxy_proxy_matrix_corporal_api_enabled: "{{ matrix_corporal_enabled and matrix_corporal_http_api_enabled }}"
+matrix_nginx_proxy_proxy_matrix_corporal_api_addr_with_container: "matrix-corporal:41081"
+matrix_nginx_proxy_proxy_matrix_corporal_api_addr_sans_container: "localhost:41081"
+
+matrix_nginx_proxy_proxy_matrix_identity_api_enabled: "{{ matrix_mxisd_enabled }}"
+matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container: "matrix-mxisd:8090"
+matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container: "localhost:8090"
+
 matrix_nginx_proxy_systemd_wanted_services_list: |
   {{
     (['matrix-synapse.service'])

roles/matrix-nginx-proxy/defaults/main.yml

@@ -19,10 +19,25 @@ matrix_nginx_proxy_proxy_riot_hostname: "{{ hostname_riot }}"
 matrix_nginx_proxy_proxy_matrix_enabled: false
 matrix_nginx_proxy_proxy_matrix_hostname: "{{ hostname_matrix }}"
 
+# Controls whether proxying for the matrix-corporal API (`/_matrix/corporal`) should be done (on the matrix domain)
+matrix_nginx_proxy_proxy_matrix_corporal_api_enabled: false
+matrix_nginx_proxy_proxy_matrix_corporal_api_addr_with_container: "matrix-corporal:41081"
+matrix_nginx_proxy_proxy_matrix_corporal_api_addr_sans_container: "localhost:41081"
+
+# Controls whether proxying for the Identity API (`/_matrix/identity`) should be done (on the matrix domain)
+matrix_nginx_proxy_proxy_matrix_identity_api_enabled: false
+matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container: "matrix-mxisd:8090"
+matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container: "localhost:8090"
+
 # The addresses where the Matrix Client API is.
 # Certain extensions (like matrix-corporal) may override this in order to capture all traffic.
-matrix_nginx_proxy_matrix_client_api_addr_with_proxy_container: "matrix-synapse:8008"
+matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container: "matrix-synapse:8008"
-matrix_nginx_proxy_matrix_client_api_addr_sans_proxy_container: "localhost:8008"
+matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container: "localhost:8008"
+# This needs to be equal or higher than the maximum upload size accepted by Synapse.
+matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size: "25M"
+
+# A list of strings containing additional configuration blocks to add to the matrix domain's server configuration.
+matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: []
 
 # Specifies when to reload the matrix-nginx-proxy service so that
 # a new SSL certificate could go into effect.

roles/matrix-nginx-proxy/tasks/main.yml

@@ -2,6 +2,15 @@
   tags:
     - always
 
+# Always validating the configuration, even if `matrix_nginx_proxy: false`.
+# This role performs actions even if the role is disabled, so we need
+# to ensure there's a valid configuration in any case.
+- import_tasks: "{{ role_path }}/tasks/validate_config.yml"
+  when: run_setup
+  tags:
+    - setup-all
+    - setup-nginx-proxy
+
 - import_tasks: "{{ role_path }}/tasks/ssl/main.yml"
   when: run_setup
   tags:
@@ -21,3 +30,9 @@
   when: run_self_check
   tags:
     - self-check
+
+- name: Mark matrix-nginx-proxy role as executed
+  set_fact:
+    matrix_nginx_proxy_role_executed: true
+  tags:
+   - always

roles/matrix-nginx-proxy/tasks/validate_config.yml

@@ -0,0 +1,12 @@
+---
+
+- name: (Deprecation) Catch and report renamed settings
+  fail:
+    msg: >
+      Your configuration contains a variable, which now has a different name.
+      Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`).
+  when: "item.old in vars"
+  with_items:
+    - {'old': 'matrix_nginx_proxy_matrix_client_api_addr_with_proxy_container', 'new': 'matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container'}
+    - {'old': 'matrix_nginx_proxy_matrix_client_api_addr_sans_proxy_container', 'new': 'matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container'}
+

roles/matrix-nginx-proxy/templates/nginx-conf.d/matrix-synapse.conf.j2

@@ -46,63 +46,53 @@ server {
 		add_header Access-Control-Allow-Origin *;
 	}
 
-	{% if matrix_corporal_enabled and matrix_corporal_http_api_enabled %}
+	{% if matrix_nginx_proxy_proxy_matrix_corporal_api_enabled %}
 	location /_matrix/corporal {
 		{% if matrix_nginx_proxy_enabled %}
 			{# Use the embedded DNS resolver in Docker containers to discover the service #}
 			resolver 127.0.0.11 valid=5s;
-			set $backend "matrix-corporal:41081";
+			set $backend "{{ matrix_nginx_proxy_proxy_matrix_corporal_api_addr_with_container }}";
 			proxy_pass http://$backend;
 		{% else %}
 			{# Generic configuration for use outside of our container setup #}
-			proxy_pass http://localhost:41081;
+			proxy_pass http://{{ matrix_nginx_proxy_proxy_matrix_corporal_api_addr_sans_container }};
 		{% endif %}
 	}
 	{% endif %}
 
-	{% if matrix_mxisd_enabled %}
+	{% if matrix_nginx_proxy_proxy_matrix_identity_api_enabled %}
 	location /_matrix/identity {
 		{% if matrix_nginx_proxy_enabled %}
 			{# Use the embedded DNS resolver in Docker containers to discover the service #}
 			resolver 127.0.0.11 valid=5s;
-			set $backend "matrix-mxisd:8090";
+			set $backend "{{ matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container }}";
 			proxy_pass http://$backend;
 		{% else %}
 			{# Generic configuration for use outside of our container setup #}
-			proxy_pass http://localhost:8090;
+			proxy_pass http://{{ matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container }};
 		{% endif %}
 	}
 	{% endif %}
 
-	{% if matrix_mautrix_telegram_enabled %}
+	{% for configuration_block in matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks %}
-	location {{ matrix_mautrix_telegram_public_endpoint }} {
+		{{- configuration_block }}
-		{% if matrix_nginx_proxy_enabled %}
+	{% endfor %}
-			{# Use the embedded DNS resolver in Docker containers to discover the service #}
-			resolver 127.0.0.11 valid=5s;
-			set $backend "matrix-mautrix-telegram:8080";
-			proxy_pass http://$backend;
-		{% else %}
-			{# Generic configuration for use outside of our container setup #}
-			proxy_pass http://localhost:8080;
-		{% endif %}
-	}
-	{% endif %}
 
 	location /_matrix {
 		{% if matrix_nginx_proxy_enabled %}
 			{# Use the embedded DNS resolver in Docker containers to discover the service #}
 			resolver 127.0.0.11 valid=5s;
-			set $backend "{{ matrix_nginx_proxy_matrix_client_api_addr_with_proxy_container }}";
+			set $backend "{{ matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container }}";
 			proxy_pass http://$backend;
 		{% else %}
 			{# Generic configuration for use outside of our container setup #}
-			proxy_pass http://{{ matrix_nginx_proxy_matrix_client_api_addr_sans_proxy_container }};
+			proxy_pass http://{{ matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container }};
 		{% endif %}
 
 		proxy_set_header X-Forwarded-For $remote_addr;
 
 		client_body_buffer_size 25M;
-		client_max_body_size {{ matrix_synapse_max_upload_size_mb }}M;
+		client_max_body_size {{ matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size }};
 		proxy_max_temp_file_size 0;
 	}
 

roles/matrix-nginx-proxy/vars/main.yml

@@ -0,0 +1,4 @@
+---
+
+# Tells whether this role had executed or not. Toggled to `true` during runtime.
+matrix_nginx_proxy_role_executed: false

roles/matrix-synapse/tasks/ext/mautrix-telegram/setup.yml

@@ -58,6 +58,53 @@
       {{ ["{{ matrix_synapse_app_service_config_file_mautrix_telegram }}"] | to_nice_json  }}
   when: "matrix_mautrix_telegram_enabled"
 
+- block:
+  - name: Fail if matrix-nginx-proxy role already executed
+    fail:
+      msg: >
+        Trying to append Mautrix Telegram's reverse-proxying configuration to matrix-nginx-proxy,
+        but it's pointless since the matrix-nginx-proxy role had already executed.
+        To fix this, please change the order of roles in your plabook,
+        so that the matrix-nginx-proxy role would run after the matrix-synapse role.
+    when: "matrix_nginx_proxy_role_executed"
+
+  - name: Generate Mautrix Telegram proxying configuration for matrix-nginx-proxy
+    set_fact:
+      matrix_mautrix_telegram_matrix_nginx_proxy_configuration: |
+        location {{ matrix_mautrix_telegram_public_endpoint }} {
+        {% if matrix_nginx_proxy_enabled %}
+        	{# Use the embedded DNS resolver in Docker containers to discover the service #}
+        	resolver 127.0.0.11 valid=5s;
+        	set $backend "matrix-mautrix-telegram:8080";
+        	proxy_pass http://$backend;
+        {% else %}
+        	{# Generic configuration for use outside of our container setup #}
+        	proxy_pass http://localhost:8080;
+        {% endif %}
+        }
+
+  - name: Register Mautrix Telegram proxying configuration with matrix-nginx-proxy
+    set_fact:
+      matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: |
+        {{
+          matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks
+          +
+          [matrix_mautrix_telegram_matrix_nginx_proxy_configuration]
+        }}
+
+  when: "matrix_mautrix_telegram_enabled and matrix_nginx_proxy_enabled|default(False)"
+  tags:
+   - always
+
+- name: Warn about reverse-proxying if matrix-nginx-proxy not used
+  debug:
+    msg: >
+      NOTE: You've enabled the Mautrix Telegram bridge but are not using the matrix-nginx-proxy
+      reverse proxy.
+      Please make sure that you're proxying the `{{ matrix_mautrix_telegram_public_endpoint }}`
+      URL endpoint to the matrix-mautrix-telegram container.
+  when: "matrix_mautrix_telegram_enabled and matrix_nginx_proxy_enabled is not defined"
+
 #
 # Tasks related to getting rid of matrix-mautrix-telegram (if it was previously enabled)
 #