finallycoffee/matrix-docker-ansible-deploy
finallycoffee/matrix-docker-ansible-deploy
5
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,936 Commits 3 Branches 0 Tags 171 MiB
111fa65e44
Commit Graph

848 Commits

Author SHA1 Message Date
Charles Wright
025a7e5c66
Merge branch 'spantaleev:master' into cvwright/room-workers-v2 2024-01-17 08:02:47 -06:00
Slavi Pantaleev
042c74f90c Remove some useless oidc variables and /_synapse/oidc route handling 
After some checking, it seems like there's `/_synapse/client/oidc`,
but no such thing as `/_synapse/oidc`.

I'm not sure why we've been reverse-proxying these paths for so long
(even in as far back as the `matrix-nginx-proxy` days), but it's time we
put a stop to it.

The OIDC docs have been simplified. There's no need to ask people to
expose the useless `/_synapse/oidc` endpoint. OIDC requires
`/_synapse/client/oidc` and `/_synapse/client` is exposed by default
already.
 2024-01-17 14:45:19 +02:00
Slavi Pantaleev
0bf8aec8f3 Adjust service priorities to better reflect our new dependencies 
Traefik also serves an internal entrypoint that all addon services
(bridges, bots, etc.) depend on, so it makes sense to have it be
available early on. It is injected as a systemd `required` dependency
for all services, so it would have been pulled earlier anyway (despite
the priority). Nevertheless, it's better to make the playbook-defined
priotities for services match, so that services are explicitly asked to
start in a more correct order.

With these changes in place now, all "start service" tasks executed by
Ansible cause a "change", indicating that all these services are started
in the correct order and none of them is unintentionally started as a
dependency for another.
 2024-01-17 11:52:46 +02:00
Slavi Pantaleev
17859eccca Put matrix-static-files in matrix_playbook_reverse_proxy_container_network unless matrix_playbook_reverse_proxy_type is "none" 
We likely weren't handling the `matrix_playbook_reverse_proxy_type: other-traefik-container`
case well before. Now, we should be.
 2024-01-17 08:46:48 +02:00
Slavi Pantaleev
da1f570db6 Make sure matrix-static-files is connected to the (other Traefik) reverse-proxy network 2024-01-17 07:23:42 +02:00
Slavi Pantaleev
0315d03cdb Make sure prometheus-postgres-exporter is connected to the Postgres network (if necessary) 
Supersedes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3110
 2024-01-17 07:17:39 +02:00
Charles Wright
db70230ae1 Add room-workers as a new preset, with new room workers, sync workers, client readers, and federation readers. Based on https://tcpipuk.github.io/synapse/index.html 2024-01-16 09:17:24 -06:00
Slavi Pantaleev
b1e08db01d Fix incorrect assumption for matrix_playbook_reverse_proxy_type == "other-traefik-container" setups 
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3095
 2024-01-15 22:29:23 +02:00
Slavi Pantaleev
0b7657396b Fix reference to unknown variable (matrix_well_known_ident) 
This also supposedly improves the default container network for
`matrix-static-files` for the `other-traefik-container` reverse-proxy
type.
 2024-01-15 22:04:22 +02:00
Slavi Pantaleev
b91ad453be Adjust TLS variables for homeservers to follow devture_traefik_config_entrypoint_web_secure_enabled (via matrix_federation_traefik_entrypoint_tls) 2024-01-15 09:39:36 +02:00
Slavi Pantaleev
3fa21d19be Wire matrix_bot_maubot_hostname via group vars 2024-01-14 21:33:09 +02:00
Slavi Pantaleev
25697861d7 Fix some variable typos in matrix-prometheus-nginxlog-exporter 2024-01-14 21:32:02 +02:00
Slavi Pantaleev
4f9b7ba656 Add missing container label wiring for mautrix-googlechat and mautrix-hangouts 2024-01-14 21:22:08 +02:00
Slavi Pantaleev
f4f3d57520 Remove all traces of matrix-nginx-proxy, add validation & uninstallation tasks 2024-01-14 18:42:14 +02:00
Slavi Pantaleev
bdc573d1b1 Wire some matrix-synapse-reverse-proxy-companion label variables based on matrix-synapse variables 2024-01-14 12:31:05 +02:00
Slavi Pantaleev
038c63888a Remove definition of old variable (matrix_synapse_admin_nginx_proxy_integration_enabled) 2024-01-14 12:12:15 +02:00
Slavi Pantaleev
69ca30d1b1 Add support for the internal Traefik entrypoint to matrix-media-repo 2024-01-14 11:57:51 +02:00
Slavi Pantaleev
6b5f42fa81 Indirectly make use of matrix_homeserver_federation_enabled in matrix-media-repo and add some comments around Traefik labels 2024-01-14 11:54:02 +02:00
Slavi Pantaleev
c238978ac8 Add new global variable for controlling federation regardless of homeserver implementation 
The old variables still work. The global lets us avoid
auto-detection logic like we're currently doing for
`matrix_nginx_proxy_proxy_matrix_federation_api_enabled`.

In the future, we'd just be able to reference
`matrix_homeserver_federation_enabled` and know the up-to-date value
regardless of homeserver.
 2024-01-14 11:52:40 +02:00
Slavi Pantaleev
df5d8bfc04 Remove matrix-homeserver-proxy role in favor of the new internal Traefik entrypoint 
This was meant to serve as an intermediary for services needing to reach
the homeserver. It was used like that for a while in this
`bye-bye-nginx-proxy` branch, but was never actually public.

It has recently been superseded by homeserver-like services injecting
themselves into a new internal Traefik entrypoint
(see `matrix_playbook_internal_matrix_client_api_traefik_entrypoint_*`),
so `matrix-homeserver-proxy` is no longer necessary.

---

This is probably a good moment to share some benchmarks and reasons
for going with the internal Traefik entrypoint as opposed to this nginx
service.

1. (1400 rps) Directly to Synapse (`ab -n 1000 -c 100 http://matrix-synapse:8008/_matrix/client/versions`
2. (~900 rps) Via `matrix-homeserver-proxy` (nginx) proxying to Synapse (`ab -n 1000 -c 100 http://matrix-homeserver-proxy:8008/_matrix/client/versions`)
3. (~1200 rps) Via the new internal entrypoint of Traefik (`matrix-internal-matrix-client-api`) proxying to Synapse (`ab -n 1000 -c 100 http://matrix-traefik:8008/_matrix/client/versions`)

Besides Traefik being quicker for some reason, there are also other
benefits to not having this `matrix-homeserver-proxy` component:

- we can reuse what we have in terms of labels. Services can register a few extra labels on the new Traefik entrypoint
- we don't need services (like `matrix-media-repo`) to inject custom nginx configs into `matrix-homeserver-proxy`. They just need to register labels, like they do already.
- Traefik seems faster than nginx on this benchmark for some reason, which is a nice bonus
- no need to run one extra container (`matrix-homeserver-proxy`) and execute one extra Ansible role
- no need to maintain a setup where some people run the `matrix-homeserver-proxy` component (because they have route-stealing services like `matrix-media-repo` enabled) and others run an optimized setup without this component and everything needs to be rewired to talk to the homeserver directly. Now, everyone can go through Traefik and we can all run an identical setup

Downsides of the new Traefik entrypoint setup are that:

- all addon services that need to talk to the homeserver now depend on Traefik
- people running their own Traefik setup will be inconvenienced - they
  need to manage one additional entrypoint
 2024-01-14 10:53:14 +02:00
Slavi Pantaleev
17c9e3f168 Add support for the internal Traefik entrypoint to synapse-reverse-proxy-companion 2024-01-14 10:48:55 +02:00
Slavi Pantaleev
4d66c14fd5 Add support for the internal Traefik entrypoint to Conduit 2024-01-14 10:48:55 +02:00
Slavi Pantaleev
ee0eb59dc6 Add support for the internal Traefik entrypoint to Dendrite 2024-01-14 10:48:54 +02:00
Slavi Pantaleev
b2aeb8cde9 Rename label-related variables for homeservers 
We'd be adding integration with an internal Traefik entrypoint
(`matrix_playbook_internal_matrix_client_api_traefik_entrypoint`),
so renaming helps disambiguate things.

There's no need for deperecation tasks, because the old names
have only been part of this `bye-bye-nginx-proxy` branch and not used by
anyone publicly.
 2024-01-14 10:48:54 +02:00
Slavi Pantaleev
39bddefd39 Make addons communicate with the homeserver via a new internal Traefik entrypoint 
This also adds labels for Synapse. Support for other homeservers and
components will be added later.
 2024-01-14 10:48:54 +02:00
Slavi Pantaleev
533dc711ad Merge branch 'master' into bye-bye-nginx-proxy 2024-01-14 09:23:43 +02:00
Slavi Pantaleev
95e5a5c62e Deprecate direct usage of devture_traefik_additional_entrypoints_auto 2024-01-14 09:23:36 +02:00
Slavi Pantaleev
bfd93adb20 Fix variable name typo 2024-01-13 20:11:43 +02:00
Slavi Pantaleev
d7b5b65b0c Connect postgres-backup directly to Postgres network, if integrated Postgres is used 
This saves us one container network in the ideal case.
 2024-01-13 20:10:41 +02:00
Slavi Pantaleev
d48a70b052 Connect matrix-synapse-auto-compressor directly to Postgres network, if integrated Postgres is used 
This saves us one container network in the ideal case.
 2024-01-13 20:01:06 +02:00
Slavi Pantaleev
130f9ad0a3 Move prometheus to matrix_monitoring_container_network 2024-01-13 19:55:27 +02:00
Slavi Pantaleev
17d80cb9e8 Move wsproxy to the matrix-addons network and adjust its Postgres connectivity 
This is a bit of a compatibility break.
The role was defaulting the Postgres password to `some-password` and we
auto-generate it now.

However, rebuilding both Postgres and this service should unify the
database credentials and the service configs to the new value.
 2024-01-13 18:13:06 +02:00
Slavi Pantaleev
ed63068e22 Make maubot talk to the homeserver via matrix_addons_homeserver_client_api_url 2024-01-13 18:04:21 +02:00
Slavi Pantaleev
c79f354dce Move Dimension to the addons network and connect to Homeserver via matrix_addons_homeserver_client_api_url 2024-01-13 17:58:41 +02:00
Slavi Pantaleev
0ceea3895e Move all monitoring-related services to their own container network (matrix_monitoring_container_network) 2024-01-13 17:46:52 +02:00
Slavi Pantaleev
782f1f5b1c Run postgres-backup in its own container network (not in matrix_docker_network) 2024-01-13 17:42:01 +02:00
Slavi Pantaleev
cdf28c39d3 Move matrix-user-verification service to its own container network 2024-01-13 17:31:03 +02:00
Slavi Pantaleev
0921087a21 Make Rageshake use its own container network 2024-01-13 17:29:14 +02:00
Slavi Pantaleev
c96a0156c0 Make matrix-dynamic-dns use its own container network 2024-01-13 17:18:22 +02:00
Slavi Pantaleev
c86cff2708 Fix NeDB to Postgres importing task for matrix-bridge-appservice-slack 
Same as 250b91a40968e, but for Slack
 2024-01-13 17:18:22 +02:00
Slavi Pantaleev
6b73073012 Fix NeDB to Postgres importing task for matrix-bridge-appservice-irc 
Postgres is not in `matrix_docker_network` anymore, so what we had
before could not possibly work anymore.
 2024-01-13 17:18:22 +02:00
Slavi Pantaleev
3f212feb1f Move matrix-email2matrix to its own container network 2024-01-13 17:18:22 +02:00
Slavi Pantaleev
e2157517af Hook matrix-homeserver-proxy to matrix-prometheus-nginxlog-exporter 2024-01-13 16:51:09 +02:00
Slavi Pantaleev
262caf0d59 Add native Traefik support to matrix-prometheus-nginxlog-exporter 2024-01-13 16:50:44 +02:00
Slavi Pantaleev
5d76b91dc2 Restore matrix-corporal functionality when matrix-nginx-proxy is not involved 2024-01-13 15:29:47 +02:00
Slavi Pantaleev
c23022ff86 Merge branch 'master' into bye-bye-nginx-proxy 2024-01-13 15:07:07 +02:00
Slavi Pantaleev
71e0022d9a Upgrade prometheus-postgres-exporter (v0.14.0-2 -> v0.14.0-3) and stop using prometheus_postgres_exporter_server_fqn 2024-01-13 15:06:29 +02:00
Slavi Pantaleev
48e6344c9e Merge branch 'master' into bye-bye-nginx-proxy 2024-01-13 10:25:35 +02:00
Slavi Pantaleev
22dce1d4cc Upgrade matrix-reminder-bot and lock it down via the new allowlist setting 2024-01-13 10:22:06 +02:00
Slavi Pantaleev
48311bb96a Stop using deprecated variable name (prometheus_node_exporter_server_fqn) 2024-01-13 09:05:43 +02:00
Slavi Pantaleev
3c81d0b06a Only expose prometheus-node-exporter/prometheus-postgres-exporter metrics publicly if matrix_metrics_exposure_enabled 2024-01-12 17:58:11 +02:00
Slavi Pantaleev
c468a860f8 Switch to exposing prometheus-postgres-exporter via native Traefik labels, not via matrix-prometheus-services-proxy-connect.. and remove matrix-prometheus-services-proxy-connect role 
This requires at least `v0.14.0-2` of the `prometheus-postgres-exporter`
Ansible role.
 2024-01-12 17:54:54 +02:00
Slavi Pantaleev
beb0f2387d Switch to exposing prometheus-node-exporter via native Traefik labels, not via matrix-prometheus-services-proxy-connect 
This requires at least `v1.7.0-2` of the `prometheus-node-exporter`
Ansible role.
 2024-01-12 17:41:54 +02:00
Slavi Pantaleev
7fba83924c Remove etherpad-proxy-connect role 2024-01-12 17:22:46 +02:00
Slavi Pantaleev
4018aa38b3 Move matrix-registration service to its own network and add native Traefik support 2024-01-12 17:17:12 +02:00
Slavi Pantaleev
41a52945d6 Add support for exposing metrics for Synapse workers 2024-01-12 12:16:06 +02:00
Slavi Pantaleev
22f5f0ba75 Add support for exposing metrics for Synapse (without workers) 2024-01-12 12:15:57 +02:00
Slavi Pantaleev
18254cd0b2 Remvoe all Traefik labels from matrix-nginx-proxy and update docs for delegation via SRV 2024-01-11 12:31:56 +02:00
Slavi Pantaleev
c4d6144bb9 Add metrics-exposure support for Dendrite 2024-01-11 12:02:15 +02:00
Slavi Pantaleev
e902214070 Automatically expose /_synapse/admin for Dendrite when synapse-admin is enabled 
This is what we do for Synapse as well.
 2024-01-11 11:31:12 +02:00
Slavi Pantaleev
d8eb768e03 Add native Traefik support to matrix-dendrite 2024-01-11 11:30:42 +02:00
Slavi Pantaleev
f78adfde47 Remove Synapse support from matrix-nginx-proxy 2024-01-11 09:24:01 +02:00
Slavi Pantaleev
030e8065e4 Remove Conduit support from matrix-nginx-proxy 2024-01-11 09:21:00 +02:00
Slavi Pantaleev
9ae8ccac36 Add matrix_conduit_hostname 2024-01-11 09:17:13 +02:00
Slavi Pantaleev
4639eebf12 Add native Traefik support to matrix-conduit 2024-01-11 08:56:51 +02:00
Slavi Pantaleev
53b5d8286f Merge branch 'master' into bye-bye-nginx-proxy 2024-01-11 08:35:53 +02:00
Slavi Pantaleev
6766216fcb Wire Conduit to advertise usage of the Coturn TURN server 
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3089
 2024-01-11 07:52:48 +02:00
Slavi Pantaleev
f54b68956d Adapt matrix-media-repo to new container network setup, etc. 2024-01-09 18:52:38 +02:00
Slavi Pantaleev
db272ab995 Move ma1sd out matrix-addons and into matrix-homeserver container network 
Such a core service probably belongs better when it's in the homeserver network
 2024-01-09 18:51:25 +02:00
Slavi Pantaleev
fc79afadd1 Merge branch 'master' into bye-bye-nginx-proxy 2024-01-09 16:07:54 +02:00
Slavi Pantaleev
3e19c8b102 Define matrix_media_repo_homeservers_auto in group vars 
This is mostly so as to avoid referring to variables from other roles,
like `matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container`.
 2024-01-09 16:07:23 +02:00
Slavi Pantaleev
c7a637bfde Merge branch 'master' into bye-bye-nginx-proxy 2024-01-09 16:03:01 +02:00
Slavi Pantaleev
883afa11dc Do not hardcode devture_postgres_identifier in matrix-media-repo role 
This should come (and already does) from group_vars/matrix_servers
 2024-01-09 16:02:31 +02:00
Slavi Pantaleev
aea66442a1 Move matrix-ma1sd to its own container network and add native Traefik support 2024-01-09 15:27:13 +02:00
Slavi Pantaleev
61216d51cc Move matrix-ldap-registration-proxy to its own container network and add native Traefik support 
This also makes it handle the `/_matrix/client/v3/register` endpoint,
not just `/_matrix/client/r0/register`
 2024-01-09 11:28:20 +02:00
Slavi Pantaleev
998e9ce655 Revert "Auto-generate matrix_bot_matrix_registration_bot_bot_password via group vars" 
This reverts commit bf95ad2235.

This was a bad idea.
It's better to have people manually define the password.

Otherwise, `matrix_homeserver_generic_secret_key` changing some day in
the future would break the bot and one would have to figure out how to
reset its password manually.

Using an explicit password is more stable.
 2024-01-09 10:22:20 +02:00
Slavi Pantaleev
bf95ad2235 Auto-generate matrix_bot_matrix_registration_bot_bot_password via group vars 2024-01-09 10:19:57 +02:00
Slavi Pantaleev
fce84a2b3c Rename variable for consistency (matrix_homeserver_proxy_ident -> matrix_homeserver_proxy_identifier) 2024-01-09 09:54:42 +02:00
Slavi Pantaleev
2f27a57d00 Rename variable for consistency (matrix_static_files_ident -> matrix_static_files_identifier) 2024-01-09 09:54:00 +02:00
Slavi Pantaleev
ea992496a3 Add matrix-cactus-comments-client role 
This is split out from matrix-cactus-comments (see 241779b583),
but also heavily inspired by `matrix-static-files`.
 2024-01-09 09:53:01 +02:00
Slavi Pantaleev
14b252c5f0 Merge branch 'master' into bye-bye-nginx-proxy 2024-01-08 20:00:10 +02:00
Slavi Pantaleev
7c5cbecd78 Enable self-building for cactus-comments on non-amd64 architectures 
The container image has only ever been available for amd64,
so not enabling self-building for the other architectures was a mistake
that orignally landed in:
https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2089
 2024-01-08 19:58:41 +02:00
Slavi Pantaleev
241779b583 Initial work on moving matrix-cactus-comments to its own container network and splitting cactus-client out of it 2024-01-08 19:57:18 +02:00
Slavi Pantaleev
30d82cc651 Merge branch 'master' into bye-bye-nginx-proxy 2024-01-08 18:18:34 +02:00
Slavi Pantaleev
b6916d3adc Add public_address to mautrix-discord 
Related to https://github.com/mautrix/discord/issues/95
 2024-01-08 18:16:02 +02:00
Slavi Pantaleev
594e6d9679 Move matrix-sms-bridge to its own container network and add support for non-Synapse homeservers 2024-01-08 18:10:38 +02:00
Slavi Pantaleev
8e8c9cc03b Move matrix-bridge-mx-puppet-twitter to its own container network and add native Traefik support 2024-01-08 17:56:37 +02:00
Slavi Pantaleev
1e19fee772 Move matrix-bridge-mx-puppet-steam to its own container network 2024-01-08 17:56:12 +02:00
Slavi Pantaleev
3c099541a7 Move matrix-bridge-mx-puppet-slack to its own container network and add native Traefik support 2024-01-08 17:56:12 +02:00
Slavi Pantaleev
150a40ec26 Move matrix-bridge-mx-puppet-instagram to its own container network 2024-01-08 17:16:50 +02:00
Slavi Pantaleev
f94f2b9823 Move matrix-bridge-mx-puppet-groupme to its own container network 2024-01-08 17:16:50 +02:00
Slavi Pantaleev
82de4581e3 Add support for disabling presence on matrix-bridge-mx-puppet-discord 2024-01-08 17:06:38 +02:00
Slavi Pantaleev
6d0ecb0269 Move matrix-bridge-mx-puppet-discord to its own container network 2024-01-08 17:03:48 +02:00
Slavi Pantaleev
8b28f8e122 Move matrix-bridge-mautrix-twitter to its own container network and add native Traefik support 2024-01-07 17:54:46 +02:00
Slavi Pantaleev
f9b4ae8241 Move matrix-bridge-mautrix-telegram to its own container network and add native Traefik support 2024-01-07 17:35:10 +02:00
Slavi Pantaleev
0f89156e94 Move matrix-bridge-mautrix-slack to its own container network 2024-01-07 17:22:43 +02:00
Slavi Pantaleev
d6911503a0 Move matrix-bridge-mautrix-signal to its own container network and add native Traefik support 2024-01-07 17:16:38 +02:00
Slavi Pantaleev
7ec6fd3dfe Make bridges/bots use matrix_addons_homeserver_client_api_url (instead of matrix_homeserver_container_url) 2024-01-07 17:04:23 +02:00
Slavi Pantaleev
142de83b41 Move matrix-bridge-mautrix-hangouts to its own container network 2024-01-07 15:37:39 +02:00
Slavi Pantaleev
f8f3318bb2 Move matrix-bridge-mautrix-googlechat to its own container network 2024-01-07 15:24:11 +02:00