Add support for stream writer Synapse workers

As stream writer workers are also powered by the `generic_worker`
Synapse app, this necessitated that we provide means for distinguishing
between them and regular `generic_workers`.

I've also taken the time to optimize nginx configuration generation
(more Jinja2 macro usage, less duplication).

Worker names have also changed.
Workers are now named sequentially like this:
- `matrix-synapse-worker-0-generic`
- `matrix-synapse-worker-1-stream-writer-typing`
- `matrix-synapse-worker-2-pusher`

instead of `matrix-synapse-worker_generic_worker-18111` (indexed with a
port number).

People who modify `matrix_synapse_workers_enabled_list` directly will
need to adjust their configuration.
This commit is contained in:
Slavi Pantaleev
2022-09-15 07:05:25 +03:00
parent 99f4f5edc7
commit 226c550ffa
19 changed files with 449 additions and 121 deletions

View File

@ -352,13 +352,13 @@ worker_app: synapse.app.homeserver
# thx https://oznetnerd.com/2017/04/18/jinja2-selectattr-filter/
# reduce the main worker's offerings to core homeserver business
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list | length > 0 %}
send_federation: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list | length > 0 %}
enable_media_repo: false
{% endif %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list %}
{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list | length > 0 %}
start_pushers: false
{% endif %}
@ -2870,6 +2870,7 @@ opentracing:
# worker1:
# host: localhost
# port: 8034
instance_map: {{ matrix_synapse_instance_map | to_json }}
# Experimental: When using workers you can define which workers should
# handle event persistence and typing notifications. Any worker
@ -2878,6 +2879,7 @@ opentracing:
#stream_writers:
# events: worker1
# typing: worker1
stream_writers: {{ matrix_synapse_stream_writers | to_json }}
# The worker that is used to run background tasks (e.g. cleaning up expired
# data). If not provided this defaults to the main process.