Add support for multiple pusher workers

CHANGELOG.md

@@ -6,6 +6,7 @@ People who are interested in running a Synapse worker setup should know that **o
 
 - we've added support for [Stream writers](#stream-writers-support)
 - we've added support for [multiple federation sender workers](#multiple-federation-sender-workers-support)
+- we've added support for [multiple pusher workers](#multiple-pusher-workers-support)
 - see the [Potential Backward Incompatibilities after these Synapse worker changes](#potential-backward-incompatibilities-after-these-synapse-worker-changes)
 
 ### Stream writers support
@@ -32,6 +33,12 @@ matrix_synapse_workers_stream_writer_presence_stream_workers_count: 1
 Until now, we only supported a single `federation_sender` worker (`matrix_synapse_workers_federation_sender_workers_count` could either be `0` or `1`).
 From now on, you can have as many as you want to help with your federation traffic.
 
+### Multiple pusher workers support
+
+Until now, we only supported a single `pusher` worker (`matrix_synapse_workers_pusher_workers_count` could either be `0` or `1`).
+From now on, you can have as many as you want to help with pushing notifications out.
+
+
 ### Potential Backward Incompatibilities after these Synapse worker changes
 
 Below we'll discuss **potential backward incompatibilities**.

roles/matrix-synapse/defaults/main.yml

@@ -498,12 +498,21 @@ matrix_synapse_workers_stream_writer_workers_http_port_range_start: 20011
 matrix_synapse_workers_stream_writer_workers_replication_port_range_start: 25011
 matrix_synapse_workers_stream_writer_workers_metrics_range_start: 19111
 
-# matrix_synapse_workers_pusher_workers_count can only be 0 or 1 for now.
+# matrix_synapse_workers_pusher_workers_count controls the number of pusher workers (workers who push out notifications) to spawn.
-# More instances are not supported due to a playbook limitation having to do with keeping `pusher_instances` in `homeserver.yaml` updated.
+# See https://matrix-org.github.io/synapse/latest/workers.html#synapseapppusher
-# See https://github.com/matrix-org/synapse/commit/ddfdf945064925eba761ae3748e38f3a1c73c328
 matrix_synapse_workers_pusher_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['pusher_workers_count'] }}"
 matrix_synapse_workers_pusher_workers_metrics_range_start: 19200
 
+# matrix_synapse_federation_pusher_instances populates the `pusher_instances` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
+# What you see below is an initial default value which will be adjusted at runtime based on the value of `matrix_synapse_workers_pusher_workers_count` or `matrix_synapse_workers_enabled_list`.
+# Adjusting this value manually is generally not necessary.
+matrix_synapse_federation_pusher_instances: []
+
+# matrix_synapse_start_pushers controls if theh main Synapse process should push out notifications or if it should be left to pusher workers (see `matrix_synapse_federation_pusher_instances`).
+# This is allowed if workers are disabled, or if there are no pusher workers.
+# Adjusting this value manually is generally not necessary.
+matrix_synapse_start_pushers: "{{ not matrix_synapse_workers_enabled or (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list | length > 0) }}"
+
 # matrix_synapse_workers_appservice_workers_count can only be 0 or 1. More instances are not supported.
 # appservice workers are deprecated since Synapse v1.59. This will be removed.
 # See: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
@@ -511,7 +520,7 @@ matrix_synapse_workers_appservice_workers_count: 0
 matrix_synapse_workers_appservice_workers_metrics_range_start: 19300
 
 # matrix_synapse_workers_federation_sender_workers_count controls the number of federation sender workers to spawn.
-# See https://github.com/matrix-org/synapse/blob/master/docs/workers.md#synapseappfederation_sender
+# See https://matrix-org.github.io/synapse/latest/workers.html#synapseappfederation_sender
 matrix_synapse_workers_federation_sender_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['federation_sender_workers_count'] }}"
 matrix_synapse_workers_federation_sender_workers_metrics_range_start: 19400
 
@@ -520,7 +529,7 @@ matrix_synapse_workers_federation_sender_workers_metrics_range_start: 19400
 # Adjusting this value manually is generally not necessary.
 matrix_synapse_federation_sender_instances: []
 
-# matrix_synapse_send_federation controls if theh main Synapse process should send federation traffic.
+# matrix_synapse_send_federation controls if theh main Synapse process should send federation traffic or if it should be left to federation_sender workers (see `matrix_synapse_federation_sender_instances`).
 # This is allowed if workers are disabled, or if there are no federation sender workers.
 # Adjusting this value manually is generally not necessary.
 matrix_synapse_send_federation: "{{ not matrix_synapse_workers_enabled or (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'federation_sender') | list | length > 0) }}"

roles/matrix-synapse/tasks/validate_config.yml

@@ -12,7 +12,7 @@
     - "matrix_synapse_database_password"
     - "matrix_synapse_database_database"
 
-- name: Fail if asking to configure deprecaed workers (appservice, userdir)
+- name: Fail if asking to configure deprecated workers (appservice, userdir)
   ansible.builtin.fail:
     msg: >-
       `{{ item }}` cannot be more than 0.
@@ -30,7 +30,6 @@
       `{{ item }}` cannot be more than 1. This is a single-instance worker.
   when: "vars[item]|int > 1"
   with_items:
-    - "matrix_synapse_workers_pusher_workers_count"
     - "matrix_synapse_workers_stream_writer_typing_stream_workers_count"
     - "matrix_synapse_workers_stream_writer_to_device_stream_workers_count"
     - "matrix_synapse_workers_stream_writer_account_data_stream_workers_count"

roles/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -355,9 +355,6 @@ worker_app: synapse.app.homeserver
 {% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'media_repository') | list | length > 0  %}
 enable_media_repo: false
 {% endif %}
-{% if matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'pusher') | list | length > 0  %}
-start_pushers: false
-{% endif %}
 
 daemonize: false
 {% endif %}
@@ -2862,6 +2859,9 @@ send_federation: {{ matrix_synapse_send_federation | to_json }}
 #  - federation_sender1
 federation_sender_instances: {{ matrix_synapse_federation_sender_instances | to_json }}
 
+pusher_instances: {{ matrix_synapse_federation_pusher_instances | to_json }}
+start_pushers: {{ matrix_synapse_start_pushers | to_json }}
+
 # When using workers this should be a map from `worker_name` to the
 # HTTP replication listener of the worker, if configured.
 #