Fixup for https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
This reverts 1cd82cf068 and also multiplies results by `1024`
so as to pass bytes to Synapse, not KB (as done before).
1cd82cf068 was correctly documenting what we were doing (passing KB values),
but that's incorrect.
Synapse's Config Conventions
(https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#config-conventions)
are supposed to clear it up, but they don't currently state what happens when you pass a plain number (without a unit suffix).
Thankfully, the source code tells us:
bc1db16086/synapse/config/_base.py (L181-L206)
> If an integer is provided it is treated as bytes and is unchanged.
>
> String byte sizes can have a suffix of ...
> No suffix is understood as a plain byte count.
We were previously passing strings, but that has been improved in 3d73ec887a.
Regardless, non-suffixed values seem to be treated as bytes by Synapse,
so this patch changes the variables to use bytes.
Moreover, we're moving from `matrix_synapse_memtotal_kb` to
`matrix_synapse_cache_size_calculations_memtotal_bytes` as working with
the base unit everywhere is preferrable.
Here, we also introduce 2 new variables to allow for the caps to be
tweaked:
- `matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes`
- `matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes`
We're casting everything it `int`, but since Jinja templates are
involved, these values end up as strings anyway.
Doing `| int | to_json` is good, but we should only cast numbers to
integer, not empty strings, as that (0) may be interpreted differently
by Synapse.
To turn of auto-tuning, one is possibly supposed to pass empty strings:
> This option defaults to off, enable it by providing values for the sub-options listed below.
It could be that `0` is also considered "no value provided", but I
haven't verified that.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
* Modify Synapse Cache Factor to use Auto Tune
Synapse has the ability to as it calls in its config auto tune caches.
This ability lets us set very high cache factors and then instead limit our resource use.
Defaults for this commit are 1/10th of what Element apparently runs for EMS stuff and matrix.org on Cache Factor and upstream documentation defaults for auto tune.
* Add vars to Synapse main.yml to control cache related config
This commit adds various cache related vars to main.yml for Synapse.
Some are auto tune and some are just adding explicit ways to control upstream vars.
* Updated Auto Tune figures
Autotuned figures have been bumped in consultation with other community members as to a reasonable level. Please note these defaults are more on the one of each workers side than they are on the monolith Side.
* Fix YML Error
The playbook is not happy with the previous state of this patch so this commit hopefully fixes it
* Add to_json to various Synapse tuning related configs
* Fix incorrect indication in homeserver.yaml.j2
* Minor cleanups
* Synapse Cache Autotuning Documentation
* Upgrade Synapse Cache Autotune to auto configure memory use
* Update Synapse Tuning docs to reflect automatic memory use configuration
* Fix Linting errors in synapses main.yml
* Rename variables for consistency (matrix_synapse_caches_autotuning_* -> matrix_synapse_cache_autotuning_*)
* Remove FIX ME comment about Synapse's `cache_autotuning`
`docs/maintenance-synapse.md` and `roles/custom/matrix-synapse/defaults/main.yml`
already contains documentation about these variables and the default values we set.
* Improve "Tuning caches and cache autotuning" documentation for Synapse
* Announce larger Synapse caches and cache auto-tuning
---------
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
`matrix_synapse_federation_port_enabled` is defined like this:
```
matrix_synapse_federation_port_enabled: "{{ matrix_synapse_federation_enabled or matrix_synapse_federation_port_openid_resource_required }}"
```
Previously, people that disabled federation, but needed the `openid`
listener were running without these federation-related labels.
In this patch, we're also dropping the `not matrix_synapse_workers_enabled` condition,
because.. none of the Matrix-related labels would be applied anyway when
workers are enabled, thanks to `matrix_synapse_container_labels_matrix_related_labels_enabled`.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3127
This is a break in backward-compatibility for people disabling
`index.html` creation via the playbook but are managing their static
website files in another way (AUX role, etc).
* Fix s3-storage migrate and shell: container needs attachment to postgres network also
* Connect to s3-storage-provider migrate to multiple networks in multiple steps
Multiple `--network` calls lead to:
> docker: Error response from daemon: Container cannot be connected to network endpoints: NETWORK_1 NETWORK_2.
* Connect to s3-storage-provider shell to multiple networks in multiple steps
---------
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
Most addons live in the same network by default (matrix-addons) right now,
so this network would have usually been created by some other addon.
Howevre, if this is the only addon someone uses, it may have remained
uncreated causing a problem.
I believe `specialized-workers` is a better name than `room-workers`,
because when enabled, 4 different types of specialized workers are
created:
- Room workers
- Sync workers
- Client readers
- Federation readers
Only one of these is called room-workers.
In the future, more specialized workers may be added, making the
`room-workers` preset name an even poorer choice.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
Until now, the validation check would only get tripped up
if generic workers are used, combined with at least one EACH
other type of specialized workers.
This means that someone doing this:
```
matrix_synapse_workers_preset: one-of-each
matrix_synapse_workers_client_reader_workers_count: 5
```
.. would not have triggered this safety check.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
Hookshot wants a trailing slash for this route.
If we let Hookshot redirect, it goes to `/widgetapi/v1/static/`,
instead of `/hookshot/widgetapi/v1/static/`, so we take this matter into our
own hands.
Public URLs are like: `/hookshot/widgetapi/v1/static/`
.. which get translated to requests for: `/widgetapi/v1/static/`
Previously, we were stripping the whole `/hookshot/widgetapi` prefix,
which is wrong.
Most of these files were defining a service, usually toward the end.
These lines have been moved upward.
Some components (mautrix-signal, mautrix-gmessages, etc.) were defining
a service conditionally (only if metrics are exposed, etc). This was
causing issues like these in the Traefik logs:
> level=error msg="service \"matrix-mautrix-twitter\" error: port is missing" providerName=docker container=matrix-mautrix-twitter-..
This changes the behavior of
`matrix_playbook_migration_matrix_nginx_proxy_uninstallation_enabled`
and is against what we initially described in the changelog entry,
but I've discovered some problems when the `matrix-nginx-proxy` service
and container remain running. They need to go.
After some checking, it seems like there's `/_synapse/client/oidc`,
but no such thing as `/_synapse/oidc`.
I'm not sure why we've been reverse-proxying these paths for so long
(even in as far back as the `matrix-nginx-proxy` days), but it's time we
put a stop to it.
The OIDC docs have been simplified. There's no need to ask people to
expose the useless `/_synapse/oidc` endpoint. OIDC requires
`/_synapse/client/oidc` and `/_synapse/client` is exposed by default
already.
Issues and Pull Requests were not migrated to the new
organization/repository, so `matrix-org/synapse/pull` and
`matrix-org/synapse/issues` references were kept as-is.
`matrix-org/synapse-s3-storage-provider` references were also kept,
as that module still continues living under the `matrix-org` organization.
This patch mainly aims to change documentation-related things, not actual
usage in full yet. For polish that, another more comprehensive patch is coming later.
