Commit Graph

179 Commits

Author SHA1 Message Date
Michael
5a6bdb0c3d merge upstream 2021-03-16 21:52:26 +08:00
Michael
33ec5710d9 0.2.1 revision 2021-02-28 22:21:40 +08:00
Slavi Pantaleev
d94d0e2ca5
Merge pull request #456 from eMPee584/synapse-workers
Synapse workers
2021-02-19 11:40:36 +02:00
Slavi Pantaleev
9dc87bb948 Add Synapse worker presets for easier configuration
Adding more presets in the future would be nice.
2021-02-19 11:38:47 +02:00
Slavi Pantaleev
217b4a8808 Release Synapse v1.27.0 to ARM32 via self-building
Related to: https://matrix.org/blog/2021/02/18/synapse-1-27-0-released#dropping-armv7-docker-images
2021-02-19 09:10:16 +02:00
Slavi Pantaleev
d6c4d41c2b Define instanceId property on workers
This give us the possibility to run multiple instances of
workers that that don't expose a port.

Right now, we don't support that, but in the future we could
run multiple `federation_sender` or `pusher` workers, without
them fighting over naming (previously, they'd all be named
something like `matrix-synapse-worker-pusher-0`, because
they'd all define `port` as `0`).
2021-02-18 18:19:51 +02:00
Slavi Pantaleev
d33483b8ce Document that Synapse pusher worker instances are shardable
Related to:
- https://github.com/matrix-org/synapse/pull/9407
- https://github.com/matrix-org/synapse/pull/7855
2021-02-16 17:45:41 +02:00
Slavi Pantaleev
daae74b074 Merge branch 'master' into synapse-workers 2021-02-16 17:31:40 +02:00
Slavi Pantaleev
521160c12f Upgrade Synapse (v1.26.0 -> v1.27.0) 2021-02-16 17:30:48 +02:00
Slavi Pantaleev
61e427d690 Do not let people enable more than 1 federation_sender worker 2021-02-15 11:37:03 +02:00
Slavi Pantaleev
85a05f38e8 Allow Synapse worker list to be generated dynamically
This leads to much easier management and potential safety
features (validation). In the future, we could try to avoid port
conflicts as well, but it didn't seem worth the effort to do it now.
Our port ranges seem large enough.

This can also pave the way for a "presets" feature
(similar to `matrix_nginx_proxy_ssl_presets`) which makes it even easier
for people to configure worker counts.
2021-02-15 11:25:35 +02:00
Slavi Pantaleev
5cfeae806b Merge branch 'master' into synapse-workers 2021-02-14 13:00:57 +02:00
Slavi Pantaleev
7e8e95a09a Make S3-mounting path configurable
This will make data migration easier.
2021-02-09 22:05:07 +02:00
Slavi Pantaleev
1cd2a218de Merge branch 'master' into synapse-workers 2021-01-27 21:41:54 +02:00
Slavi Pantaleev
c6feb0b99e Upgrade Synapse (v1.25.0 -> v1.26.0) 2021-01-27 21:41:47 +02:00
Slavi Pantaleev
66cdc7bf5a Clean up worker.yaml generation a bit and make it more flexible 2021-01-25 13:02:01 +02:00
Slavi Pantaleev
63301b0ef1 Improvements around Synapse worker/metrics ports exposure
There was a `matrix_nginx_proxy_enabled|default(False)` check, but:
- it didn't seem to work reliably for some reason (hmm)
- referring to a `matrix_nginx_proxy_*` variable from within the
  `matrix-synapse` role is not ideal
- exposing always happened on `127.0.0.1`, which may not be good enough
  for some rarer setups (where the own webserver is external to the host)
2021-01-25 08:25:43 +02:00
Marcel Partap
183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00
Slavi Pantaleev
95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
Slavi Pantaleev
b15da29ebb Bump Synapse to v1.25.0 for ARM 2021-01-14 10:41:47 +02:00
Slavi Pantaleev
d5945c6e78 Upgrade Synapse (v1.24.0 -> v1.25.0) for amd64 2021-01-13 13:02:49 +02:00
Marcel Partap
cd8100544b Merge remote-tracking branch 'origin/master' into synapse-workers
Sync with upstream
2021-01-08 20:58:50 +01:00
Slavi Pantaleev
245b749946 Upgrade Synapse for ARM (v1.23.0 -> v1.24.0)
Continuation of aa86e0dac6, now that ARM images are out.
2020-12-09 20:54:18 +02:00
Slavi Pantaleev
aa86e0dac6 Upgrade Synapse (v1.23.0 -> v1.24.0)
Because the ARM images are not pushed yet, we hold back to v1.23.0
for now.
2020-12-09 13:31:10 +02:00
Slavi Pantaleev
c07c927d9f Automatically enable openid listeners when ma1sd enabled
ma1sd requires the openid endpoints for certain functionality.
Example: 90b2b5301c/src/main/java/io/kamax/mxisd/auth/AccountManager.java (L67-L99)

If federation is disabled, we still need to expose these openid APIs on the
federation port.

Previously, we were doing similar magic for Dimension.
As per its documentation, when running unfederated, one is to enable
the openid listener as well. As per their recommendation, people
are advised to do enable it on the Client-Server API port
and use the `federationUrl` variable to override where the federation
port is (making federation requests go to the Client-Server API).

Because ma1sd always uses the federation port (unless you do some
DNS overwriting magic using its configuration -- which we'd rather not
do), it's better if we just default to putting the `openid` listener
where it belongs - on the federation port.

With this commit, we retain the "automatically enable openid APIs" thing
we've been doing for Dimension, but move it to the federation port instead.
We also now do the same thing when ma1sd is enabled.
2020-12-08 16:59:20 +02:00
Marcel Partap
f201bca519 synapse workers: define and expose METRICS port for each worker
As seen on TV:
https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md#monitoring-workers
2020-12-01 22:49:15 +01:00
Marcel Partap
af08f18779 synapse workers default config: disable user_dir worker for now
(until https://github.com/matrix-org/synapse/issues/8787 is resolved)
2020-12-01 22:22:04 +01:00
Marcel Partap
b73ac965ac Merge remote-tracking branch 'origin/master' into synapse-workers 2020-12-01 21:24:26 +01:00
Slavi Pantaleev
be5263f397 Move self-building git repository URLs to variables (stop hardcoding) 2020-11-28 21:34:14 +02:00
c58a7e03c7
synapse: update to 1.23.0 2020-11-18 14:16:46 +01:00
Slavi Pantaleev
5eed874199 Improve self-building experience (avoid conflict with pullable images)
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/716

This patch makes us use more fully-qualified container image names
(either prefixed with docker.io/ or with localhost/).

The latter happens when self-building is enabled.

We've recently had issues where if an image was removed manually
and the service was restarted (making `docker run` fetch it from Docker Hub, etc.),
we'd end up with a pulled image, even though we're aiming for a self-built one.
Re-running the playbook would then not do a rebuild, because:
- the image with that name already exists (even though it's something
else)
- we sometimes had conditional logic where we'd build only if the git
repo changed

By explicitly changing the name of the images (prefixing with localhost/),
we avoid such confusion and the possibility that we'd automatically pul something
which is not what we expect.

Also, I've removed that condition where building would happen on git
changes only. We now always build (unless an image with that name
already exists). We just force-build when the git repo changes.
2020-11-14 23:00:49 +02:00
Marcel Partap
4678c5d7bd Merge remote-tracking branch 'origin/master' into synapse-workers
Also, replace vague FIXME by a proper NOTE on the complete
story of the user_dir endpoints..
2020-11-11 21:26:08 +01:00
Slavi Pantaleev
5c91e56898 Upgrade Synapse (v1.22.0 -> v1.22.1) 2020-10-30 19:35:55 +02:00
Marcel Partap
2d1b9f2dbf synapse workers: reworkings + get endpoints from upstream docs via awk
(yes, a bit awkward and brittle… xD)
2020-10-28 07:13:19 +01:00
Slavi Pantaleev
70f0b97a0a Upgrade Synapse (v1.21.2 -> v1.22.0) 2020-10-27 14:24:02 +02:00
Marcel Partap
87bd64ce9e Merge remote-tracking branch 'origin/master' into synapse-workers 2020-10-23 23:45:07 +02:00
Slavi Pantaleev
f7ecc7a2a5 Upgrade Synapse (v1.21.1 -> v1.21.2) 2020-10-15 17:42:52 +03:00
Slavi Pantaleev
5abd511368 Upgrade Synapse (v1.21.0 -> v1.21.1) 2020-10-13 13:08:25 +03:00
Aaron Raimist
78529cbd47
Upgrade Synapse (v1.20.1 -> v1.21.0) 2020-10-12 23:59:34 -05:00
Marcel Partap
36e9be6092 matrix_synapse_workers_{avail,enabled}_list: sort non-generic workers
.. alphabetically and put those not documented as multi-instance
capable on ports ending on zero.
2020-10-11 21:44:42 +02:00
Marcel Partap
93a8ea7e4a Merge remote-tracking branch 'master' into feature/add-worker-support 2020-10-11 20:59:05 +02:00
Slavi Pantaleev
e68450f094 Upgrade Synapse (v1.20.0 -> v1.20.1) 2020-09-24 18:43:54 +03:00
Slavi Pantaleev
dd217137b6 Upgrade Synapse (v1.19.3 -> v1.20.0) 2020-09-22 19:28:07 +03:00
Slavi Pantaleev
65e22a6888 Upgrade Synapse (v1.19.2 -> v1.19.3) 2020-09-18 17:37:04 +03:00
Slavi Pantaleev
e10e3e354d Upgrade Synapse (v1.19.1 -> v1.19.2) 2020-09-16 16:35:17 +03:00
Max Klenk
4fdfc0a34f
add missing ratelimiting options required for load testing 2020-09-11 09:46:20 +02:00
Max Klenk
a25a429a52
add redis support 2020-09-10 13:39:00 +02:00
Max Klenk
06bc430c7c
refactor to use new workers and routes they serve 2020-08-28 13:53:39 +02:00
Max Klenk
53ccc783b7
remove duplicated key 2020-08-27 15:26:46 +02:00
Max Klenk
567d0318b0
Merge branch 'synapse-workers' into feature/add-worker-support 2020-08-27 15:22:12 +02:00