Make addons communicate with the homeserver via a new internal Traefik entrypoint

This also adds labels for Synapse. Support for other homeservers and
components will be added later.

docs/configuring-playbook-bot-draupnir.md

@@ -32,7 +32,7 @@ Refer to the documentation on [how to obtain an access token](obtaining-access-t
 
 You will need to prevent Synapse from rate limiting the bot's account. This is not an optional step. If you do not do this step draupnir will crash. This can be done using Synapse's [admin API](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#override-ratelimiting-for-users). Please ask for help if you are uncomfortable with these steps or run into issues.
 
-If your Synapse Admin API is exposed to the internet for some reason like running the Synapse Admin Role [Link](/docs/configuring-playbook-synapse-admin.md) or running `matrix_synapse_container_labels_client_synapse_admin_api_enabled: true` in your playbook config. If your API is not externally exposed you should still be able to on the local host for your synapse run these commands.
+If your Synapse Admin API is exposed to the internet for some reason like running the Synapse Admin Role [Link](/docs/configuring-playbook-synapse-admin.md) or running `matrix_synapse_container_labels_public_client_synapse_admin_api_enabled: true` in your playbook config. If your API is not externally exposed you should still be able to on the local host for your synapse run these commands.
 
 The following command works on semi up to date Windows 10 installs and All Windows 11 installations and other systems that ship curl. `curl --header "Authorization: Bearer <access_token>" -X POST https://matrix.example.com/_synapse/admin/v1/users/@example:example.com/override_ratelimit` Replace `@example:example.com` with the MXID of your Draupnir and example.com with your homeserver domain. You can easily obtain an access token for a homeserver admin account the same way you can obtain an access token for Draupnir it self. If you made Draupnir Admin you can just use the Draupnir token.
 

docs/configuring-playbook-bot-mjolnir.md

@@ -31,7 +31,7 @@ Refer to the documentation on [how to obtain an access token](obtaining-access-t
 
 You will need to prevent Synapse from rate limiting the bot's account. This is not an optional step. If you do not do this step Mjolnir will crash. This can be done using Synapse's [admin API](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#override-ratelimiting-for-users). Please ask for help if you are uncomfortable with these steps or run into issues.
 
-If your Synapse Admin API is exposed to the internet for some reason like running the Synapse Admin Role [Link](/docs/configuring-playbook-synapse-admin.md) or running `matrix_synapse_container_labels_client_synapse_admin_api_enabled: true` in your playbook config. If your API is not externally exposed you should still be able to on the local host for your synapse run these commands.
+If your Synapse Admin API is exposed to the internet for some reason like running the Synapse Admin Role [Link](/docs/configuring-playbook-synapse-admin.md) or running `matrix_synapse_container_labels_public_client_synapse_admin_api_enabled: true` in your playbook config. If your API is not externally exposed you should still be able to on the local host for your synapse run these commands.
 
 The following command works on semi up to date Windows 10 installs and All Windows 11 installations and other systems that ship curl. `curl --header "Authorization: Bearer <access_token>" -X POST https://matrix.example.com/_synapse/admin/v1/users/@example:example.com/override_ratelimit` Replace `@example:example.com` with the MXID of your Mjolnir and example.com with your homeserver domain. You can easily obtain an access token for a homeserver admin account the same way you can obtain an access token for Mjolnir it self. If you made Mjolnir Admin you can just use the Mjolnir token.
 

docs/configuring-playbook-nginx.md

@@ -48,7 +48,7 @@ For more information about these variables, check the `roles/custom/matrix-nginx
 If you want to use OpenID Connect as an SSO provider (as per the [Synapse OpenID docs](https://github.com/matrix-org/synapse/blob/develop/docs/openid.md)), you need to use the following configuration (in your `vars.yml` file) to instruct nginx to forward `/_synapse/oidc` to Synapse:
 
 ```yaml
-matrix_synapse_container_labels_client_synapse_oidc_api_enabled: true
+matrix_synapse_container_labels_public_client_synapse_oidc_api_enabled: true
 ```
 
 ## Disable Nginx access logs

docs/configuring-playbook-own-webserver.md

@@ -8,6 +8,8 @@ If that's alright, you can skip this.
 
 [Traefik](https://traefik.io/) is the default reverse-proxy for the playbook since [2023-02-26](../CHANGELOG.md/#2023-02-26).
 
+Besides serving public traffic, Traefik is also used for internal communication between addon services (briges, bots, etc.) and the homeserver.
+
 There are 2 ways to use Traefik with this playbook, as described below.
 
 ### Traefik managed by the playbook
@@ -29,7 +31,11 @@ Traefik will manage SSL certificates for all services seamlessly.
 ```yaml
 matrix_playbook_reverse_proxy_type: other-traefik-container
 
-matrix_playbook_reverse_proxyable_services_additional_network: your-traefik-network
+# Uncomment and adjust if your Traefik container is on another network
+# matrix_playbook_reverse_proxy_container_network: traefik
+
+# Adjust to point to your Traefik container
+matrix_playbook_reverse_proxy_hostname: name-of-your-traefik-container
 
 devture_traefik_certs_dumper_ssl_dir_path: "/path/to/your/traefiks/acme.json/directory"
 
@@ -40,13 +46,18 @@ devture_traefik_certs_dumper_ssl_dir_path: "/path/to/your/traefiks/acme.json/dir
 
 In this mode all roles will still have Traefik labels attached. You will, however, need to configure your Traefik instance and its entrypoints.
 
-By default, the playbook congiures services use a `web-secure` (443) and `matrix-federation` (8448) entrypoints, as well as a `default` certificate resolver.
+By default, the playbook configured services use a `web-secure` (443) and `matrix-federation` (8448) entrypoints, as well as a `default` certificate resolver.
 
-You need to configure 3 entrypoints for your Traefik server: `web` (TCP port `80`), `web-secure` (TCP port `443`) and `matrix-federation` (TCP port `8448`).
+You need to configure 4 entrypoints for your Traefik server:
+
+- `web` (TCP port `80`) - used for redirecting to HTTPS (`web-secure`)
+- `web-secure` (TCP port `443`) - used for exposing the Matrix Client-Server API and all other services
+- `matrix-federation` (TCP port `8448`) - used for exposing the Matrix Federation API
+- `matrix-internal-matrix-client-api` (TCP port `8008`) - used internally for addon services (bridges, bots) to communicate with the homserver
 
 Below is some configuration for running Traefik yourself, although we recommend using [Traefik managed by the playbook](#traefik-managed-by-the-playbook).
 
-Note that this configuration on its own does **not** redirect traffic on port 80 (plain HTTP) to port 443 for HTTPS, which may cause some issues, since the built-in Nginx proxy usually does this. If you are not already doing this in Traefik, it can be added to Traefik in a [file provider](https://docs.traefik.io/v2.0/providers/file/) as follows:
+Note that this configuration on its own does **not** redirect traffic on port 80 (plain HTTP) to port 443 for HTTPS. If you are not already doing this in Traefik, it can be added to Traefik in a [file provider](https://docs.traefik.io/v2.0/providers/file/) as follows:
 
 ```toml
 [http]
@@ -86,6 +97,7 @@ services:
       - "--providers.docker.exposedbydefault=false"
       - "--entrypoints.web-secure.address=:443"
       - "--entrypoints.matrix-federation.address=:8448"
+      - "--entrypoints.matrix-internal-matrix-client-api.address=:8008"
       - "--certificatesresolvers.default.acme.tlschallenge=true"
       - "--certificatesresolvers.default.acme.email=YOUR EMAIL"
       - "--certificatesresolvers.default.acme.storage=/letsencrypt/acme.json"
@@ -109,9 +121,9 @@ Doing this is possible, but requires manual work.
 
 There are 2 ways to go about it:
 
-- (recommended) [Fronting the integrated reverse-proxy webserver with another reverse-proxy](#fronting-the-integrated-reverse-proxy-webserver-with-another-reverse-proxy) - using a playbook-managed reverse-proxy (either `matrix-nginx-proxy` or Traefik), disabling SSL termination for it, exposing this reverse-proxy on a few local ports (e.g. `127.0.0.1:81`, etc.) and forwarding traffic from your own webserver to those few ports
+- (recommended) [Fronting the integrated reverse-proxy webserver with another reverse-proxy](#fronting-the-integrated-reverse-proxy-webserver-with-another-reverse-proxy) - using the playbook-managed reverse-proxy (Traefik), but disabling SSL termination for it, exposing this reverse-proxy on a few local ports (e.g. `127.0.0.1:81`, etc.) and forwarding traffic from your own webserver to those few ports
 
-- (difficult) [Using no reverse-proxy on the Matrix side at all](#using-no-reverse-proxy-on-the-matrix-side-at-all) disabling all playbook-managed reverse-proxies (no `matrix-nginx-proxy`, no Traefik)
+- (difficult) [Using no reverse-proxy on the Matrix side at all](#using-no-reverse-proxy-on-the-matrix-side-at-all) disabling the playbook-managed reverse-proxy (Traefik), exposing services one by one using `_host_bind_port` variables and forwarding traffic from your own webserver to those ports
 
 
 ### Fronting the integrated reverse-proxy webserver with another reverse-proxy
@@ -173,7 +185,7 @@ Instead of [Fronting the integrated reverse-proxy webserver with another reverse
 
 This is more difficult, as you would need to handle the configuration for each service manually. Enabling additional services would come with extra manual work you need to do.
 
-If your webserver is on the same machine, sure your web server user (something like `http`, `apache`, `www-data`, `nginx`) is part of the `matrix` group. You should run something like this: `usermod -a -G matrix nginx`. This allows your webserver user to access files owned by the `matrix` group. When using an external nginx webserver, this allows it to read configuration files from `/matrix/nginx-proxy/conf.d`. When using another server, it would make other files, such as `/matrix/static-files/.well-known`, accessible to it.
+If your webserver is on the same machine, ensure your web server user (something like `http`, `apache`, `www-data`, `nginx`) is part of the `matrix` group. You should run something like this: `usermod -a -G matrix nginx`. This allows your webserver user to access files owned by the `matrix` group, so that it can serve static files from `/matrix/static-files`.
 
 #### Using your own nginx reverse-proxy running on the same machine
 

docs/configuring-playbook-synapse-admin.md

@@ -17,7 +17,7 @@ matrix_synapse_admin_enabled: true
 
 **Note**: Synapse Admin requires Synapse's [Admin APIs](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html) to function. Access to them is restricted with a valid access token, so exposing them publicly should not be a real security concern. Still, for additional security, we normally leave them unexposed, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). Because Synapse Admin needs these APIs to function, when installing Synapse Admin, the playbook **automatically** exposes the Synapse Admin API publicly for you. Depending on the homeserver implementation you're using (Synapse, Dendrite), this is equivalent to:
 
-- for Synapse (our default homeserver implementation): `matrix_synapse_container_labels_client_synapse_admin_api_enabled: true`
+- for Synapse (our default homeserver implementation): `matrix_synapse_container_labels_public_client_synapse_admin_api_enabled: true`
 - for [Dendrite](./configuring-playbook-dendrite.md): `matrix_dendrite_container_labels_client_synapse_admin_api_enabled: true`
 
 

docs/howto-srv-server-delegation.md

@@ -27,7 +27,7 @@ Also, all instructions below are from an older version of the playbook and may n
 
 ```yaml
 # To serve the federation from any domain, as long as the path matches
-matrix_synapse_container_labels_federation_api_traefik_rule: PathPrefix(`/_matrix/federation`)
+matrix_synapse_container_labels_public_federation_api_traefik_rule: PathPrefix(`/_matrix/federation`)
 ```
 
 This is because with SRV federation, some servers / tools (one of which being the federation tester) try to access the federation API using the resolved IP address instead of the domain name (or they are not using SNI). This change will make Traefik route all traffic for which the path match this rule go to the federation endpoint.
@@ -142,7 +142,7 @@ matrix_playbook_reverse_proxy_type: playbook-managed-traefik
 devture_traefik_config_certificatesResolvers_acme_email: redacted@example.com
 
 # To serve the federation from any domain, as long as the path matches
-matrix_synapse_container_labels_federation_api_traefik_rule: PathPrefix(`/_matrix/federation`)
+matrix_synapse_container_labels_public_federation_api_traefik_rule: PathPrefix(`/_matrix/federation`)
 
 # To let Traefik know which domains' certificates to serve
 matrix_synapse_container_labels_additional_labels: |

docs/maintenance-synapse.md

@@ -18,7 +18,7 @@ You can use the **[Purge History API](https://github.com/matrix-org/synapse/blob
 
 To make use of this Synapse Admin API, **you'll need an admin access token** first. Refer to the documentation on [how to obtain an access token](obtaining-access-tokens.md).
 
-Synapse's Admin API is not exposed to the internet by default, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). To expose it you will need to add `matrix_synapse_container_labels_client_synapse_admin_api_enabled: true` to your `vars.yml` file.
+Synapse's Admin API is not exposed to the internet by default, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). To expose it you will need to add `matrix_synapse_container_labels_public_client_synapse_admin_api_enabled: true` to your `vars.yml` file.
 
 Follow the [Purge History API](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.md) documentation page for the actual purging instructions.