finallycoffee/matrix-docker-ansible-deploy
finallycoffee/matrix-docker-ansible-deploy
5
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Switch postgres/postgres-backup Ansible role sources and adjust variable names (devture_postgres_ -> postgres_)

Browse Source
This commit is contained in:
Slavi Pantaleev 2024-09-27 09:37:24 +03:00
parent 62d66cc196
commit 8445843562
42 changed files with 346 additions and 324 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGELOG.md
View File

@ -711,7 +711,7 @@ From now on, the [Postgres Ansible role](https://github.com/devture/com.devture.
Our [Tuning PostgreSQL](docs/maintenance-postgres.md#tuning-postgresql) documentation page has details about how you can turn auto-tuning off or adjust the automatically-determined Postgres configuration parameters manually.
People who [enable load-balancing with Synapse workers](docs/configuring-playbook-synapse.md#load-balancing-with-workers) no longer need to increase the maximum number of Postgres connections manually (previously done via `devture_postgres_process_extra_arguments`). There's a new variable (`devture_postgres_max_connections`) for controlling this number and the playbook automatically raises its value from `200` to `500` for setups which enable workers.
People who [enable load-balancing with Synapse workers](docs/configuring-playbook-synapse.md#load-balancing-with-workers) no longer need to increase the maximum number of Postgres connections manually (previously done via `postgres_process_extra_arguments`). There's a new variable (`postgres_max_connections`) for controlling this number and the playbook automatically raises its value from `200` to `500` for setups which enable workers.
# 2023-08-31
@ -1232,14 +1232,14 @@ See our [Setting up matrix-bot-chatgpt](docs/configuring-playbook-bot-chatgpt.md
Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix.
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `postgres_backup` prefix.
# 2022-11-28
## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role
**TLDR**: the tasks that install the integrated Postgres server now live in an external role - [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres). You'll need to run `make roles` to install it, and to also rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix (e.g. `matrix_postgres_connection_password` -> `devture_postgres_connection_password`). All your data will still be there! Some scripts have moved (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`).
**TLDR**: the tasks that install the integrated Postgres server now live in an external role - [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres). You'll need to run `make roles` to install it, and to also rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix (e.g. `matrix_postgres_connection_password` -> `postgres_connection_password`). All your data will still be there! Some scripts have moved (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`).
The `matrix-postgres` role that has been part of the playbook for a long time has been replaced with the [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres) role. This was done as part of our work to [use external roles for some things](#the-playbook-now-uses-external-roles-for-some-things) for better code re-use and maintainability.

2
docs/configuring-playbook-external-postgres.md
View File

@ -10,7 +10,7 @@ If you'd like to use an external PostgreSQL server that you manage, you can edit
If you'd like to use an external Postgres server, use a custom `vars.yml` configuration like this:
```yaml
devture_postgres_enabled: false
postgres_enabled: false
# Rewire Synapse to use your external Postgres server
matrix_synapse_database_host: "your-postgres-server-hostname"

8
docs/configuring-playbook-matrix-media-repo.md
View File

@ -123,7 +123,7 @@ To import the Synapse media store, you're supposed to invoke the `import_synapse
This guide here is adapted from the [upstream documentation about the import_synapse script](https://github.com/turt2live/matrix-media-repo#importing-media-from-synapse).
Run the following command on the server (after replacing `devture_postgres_connection_password` in it with the value found in your `vars.yml` file):
Run the following command on the server (after replacing `postgres_connection_password` in it with the value found in your `vars.yml` file):
```sh
docker exec -it matrix-media-repo \
@ -132,7 +132,7 @@ docker exec -it matrix-media-repo \
-dbHost matrix-postgres \
-dbPort 5432 \
-dbUsername matrix \
-dbPassword devture_postgres_connection_password
-dbPassword postgres_connection_password
```
Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing.
@ -145,7 +145,7 @@ If you're using the [Dendrite](configuring-playbook-dendrite.md) homeserver inst
To import the Dendrite media store, you're supposed to invoke the `import_dendrite` tool which is part of the matrix-media-repo container image. Your Dendrite database is called `dendrite_mediaapi` by default, unless you've changed it by modifying `matrix_dendrite_media_api_database`.
Run the following command on the server (after replacing `devture_postgres_connection_password` in it with the value found in your `vars.yml` file):
Run the following command on the server (after replacing `postgres_connection_password` in it with the value found in your `vars.yml` file):
```sh
docker exec -it matrix-media-repo \
@ -154,7 +154,7 @@ docker exec -it matrix-media-repo \
-dbHost matrix-postgres \
-dbPort 5432 \
-dbUsername matrix \
-dbPassword devture_postgres_connection_password
-dbPassword postgres_connection_password
```
Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing.

16
docs/configuring-playbook-postgres-backup.md
View File

@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
```yaml
devture_postgres_backup_enabled: true
postgres_backup_enabled: true
```
Refer to the table below for additional configuration variables and their default values.
@ -18,13 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
| Name | Default value | Description |
| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
|`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`devture_postgres_backup_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_backup_data_path` |
|`devture_postgres_backup_data_path` | `"{{ devture_postgres_backup_base_path }}/data"` | Storage path for postgres-backup database backups |
|`postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`postgres_backup_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `postgres_backup_data_path` |
|`postgres_backup_data_path` | `"{{ postgres_backup_base_path }}/data"` | Storage path for postgres-backup database backups |
## Installing

2
docs/configuring-playbook-synapse.md
View File

@ -75,7 +75,7 @@ The only thing you **cannot** do is mix [generic workers](#generic-workers) and
#### Effect of enabling workers on the rest of your server
When Synapse workers are enabled, the integrated [Postgres database is tuned](maintenance-postgres.md#tuning-postgresql), so that the maximum number of Postgres connections are increased from `200` to `500`. If you need to decrease or increase the number of maximum Postgres connections further, use the `devture_postgres_max_connections` variable.
When Synapse workers are enabled, the integrated [Postgres database is tuned](maintenance-postgres.md#tuning-postgresql), so that the maximum number of Postgres connections are increased from `200` to `500`. If you need to decrease or increase the number of maximum Postgres connections further, use the `postgres_max_connections` variable.
A separate Ansible role (`matrix-synapse-reverse-proxy-companion`) and component handles load-balancing for workers. This role/component is automatically enabled when you enable workers. Make sure to use the `setup-all` tag (not `install-all`!) during the playbook's [installation](./installing.md) process, especially if you're disabling workers, so that components may be installed/uninstalled correctly.

8
docs/maintenance-postgres.md
View File

@ -107,12 +107,12 @@ Example: `--extra-vars="postgres_dump_name=matrix-postgres-dump.sql"`
PostgreSQL can be [tuned](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server) to make it run faster. This is done by passing extra arguments to the Postgres process.
The [Postgres Ansible role](https://github.com/devture/com.devture.ansible.role.postgres) **already does some tuning by default**, which matches the [tuning logic](https://github.com/le0pard/pgtune/blob/master/src/features/configuration/configurationSlice.js) done by websites like https://pgtune.leopard.in.ua/.
You can manually influence some of the tuning variables . These parameters (variables) are injected via the `devture_postgres_postgres_process_extra_arguments_auto` variable.
You can manually influence some of the tuning variables . These parameters (variables) are injected via the `postgres_postgres_process_extra_arguments_auto` variable.
Most users should be fine with the automatically-done tuning. However, you may wish to:
- **adjust the automatically-determined tuning parameters manually**: change the values for the tuning variables defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) (see `devture_postgres_max_connections`, `devture_postgres_data_storage` etc). These variables are ultimately passed to Postgres via a `devture_postgres_postgres_process_extra_arguments_auto` variable
- **adjust the automatically-determined tuning parameters manually**: change the values for the tuning variables defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) (see `postgres_max_connections`, `postgres_data_storage` etc). These variables are ultimately passed to Postgres via a `postgres_postgres_process_extra_arguments_auto` variable
- **turn automatically-performed tuning off**: override it like this: `devture_postgres_postgres_process_extra_arguments_auto: []`
- **turn automatically-performed tuning off**: override it like this: `postgres_postgres_process_extra_arguments_auto: []`
- **add additional tuning parameters**: define your additional Postgres configuration parameters in `devture_postgres_postgres_process_extra_arguments_custom`. See `devture_postgres_postgres_process_extra_arguments_auto` defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) for inspiration
- **add additional tuning parameters**: define your additional Postgres configuration parameters in `postgres_postgres_process_extra_arguments_custom`. See `postgres_postgres_process_extra_arguments_auto` defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) for inspiration

2
examples/vars.yml
View File

@ -40,7 +40,7 @@ devture_traefik_config_certificatesResolvers_acme_email: ''
#
# The playbook creates additional Postgres users and databases (one for each enabled service)
# using this superuser account.
devture_postgres_connection_password: ''
postgres_connection_password: ''
# By default, we configure Coturn's external IP address using the value specified for `ansible_host` in your `inventory/hosts` file.
# If this value is an external IP address, you can skip this section.

504
group_vars/matrix_servers
View File

File diff suppressed because it is too large Load Diff

8
requirements.yml
View File

@ -42,11 +42,11 @@
- src: git+https://github.com/devture/com.devture.ansible.role.playbook_state_preserver.git
version: ff2fd42e1c1a9e28e3312bbd725395f9c2fc7f16
name: playbook_state_preserver
- src: git+https://github.com/devture/com.devture.ansible.role.postgres.git
version: v16.4-0
- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-postgres.git
version: v17.0-0
name: postgres
- src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git
version: ccfd8db07fd8725119f0e06ba5144b8f58a67890
- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-postgres-backup.git
version: v16-0
name: postgres_backup
- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus.git
version: v2.54.1-1

2
roles/custom/matrix-bot-buscarron/tasks/setup_install.yml
View File

@ -15,7 +15,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_bot_buscarron_sqlite_database_path_local }}"
dst: "{{ matrix_bot_buscarron_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bot-honoroit/tasks/setup_install.yml
View File

@ -15,7 +15,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_bot_honoroit_sqlite_database_path_local }}"
dst: "{{ matrix_bot_honoroit_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bot-matrix-reminder-bot/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}"
dst: "{{ matrix_bot_matrix_reminder_bot_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bot-postmoogle/tasks/setup_install.yml
View File

@ -12,7 +12,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_bot_postmoogle_sqlite_database_path_local }}"
dst: "{{ matrix_bot_postmoogle_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-appservice-discord/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_appservice_discord_sqlite_database_path_local }}"
dst: "{{ matrix_appservice_discord_database_connString }}"
caller: "{{ role_path | basename }}"

6
roles/custom/matrix-bridge-appservice-irc/tasks/migrate_nedb_to_postgres.yml
View File

@ -2,8 +2,8 @@
- name: Fail if Postgres not enabled
ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot migrate."
when: "not devture_postgres_enabled | bool"
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot migrate."
when: "not postgres_enabled | bool"
# Defaults
@ -16,7 +16,7 @@
- name: Ensure Postgres is started
ansible.builtin.service:
name: "{{ devture_postgres_identifier }}"
name: "{{ postgres_identifier }}"
state: started
daemon_reload: true
register: postgres_service_start_result

6
roles/custom/matrix-bridge-appservice-slack/tasks/migrate_nedb_to_postgres.yml
View File

@ -2,8 +2,8 @@
- name: Fail if Postgres not enabled
ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot migrate."
when: "not devture_postgres_enabled | bool"
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot migrate."
when: "not postgres_enabled | bool"
# Defaults
@ -16,7 +16,7 @@
- name: Ensure Postgres is started
ansible.builtin.service:
name: "{{ devture_postgres_identifier }}"
name: "{{ postgres_identifier }}"
state: started
daemon_reload: true
register: postgres_service_start_result

2
roles/custom/matrix-bridge-go-skype-bridge/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_go_skype_bridge_sqlite_database_path_local }}"
dst: "{{ matrix_go_skype_bridge_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-discord/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_discord_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_discord_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-facebook/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_facebook_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_facebook_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-gmessages/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_gmessages_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_gmessages_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-googlechat/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_googlechat_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_googlechat_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-hangouts/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_hangouts_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_hangouts_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-meta-instagram/tasks/install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_meta_instagram_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_meta_instagram_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-meta-messenger/tasks/install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_meta_messenger_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_meta_messenger_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-signal/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_signal_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_signal_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-slack/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_slack_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_slack_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-telegram/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_telegram_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_telegram_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mautrix_whatsapp_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_whatsapp_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-discord/tasks/setup_install.yml
View File

@ -50,7 +50,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_discord_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_discord_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-groupme/tasks/setup_install.yml
View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_groupme_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_groupme_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-instagram/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_instagram_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_instagram_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-slack/tasks/setup_install.yml
View File

@ -44,7 +44,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_slack_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_slack_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-steam/tasks/setup_install.yml
View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_steam_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_steam_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-bridge-mx-puppet-twitter/tasks/setup_install.yml
View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_mx_puppet_twitter_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_twitter_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-dendrite/templates/systemd/matrix-dendrite.service.j2
View File

@ -16,7 +16,7 @@ Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} matrix-dendrite 2>/dev/null || true'
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-dendrite 2>/dev/null || true'
{% if (devture_postgres_identifier + '.service') in matrix_dendrite_systemd_required_services_list %}
{% if (postgres_identifier + '.service') in matrix_dendrite_systemd_required_services_list %}
# Dendrite is too quick to start in relation to its Postgres dependency.
# Delay Dendrite startup to avoid failing with: "failed to connect to accounts db" ("pq: the database system is starting up").
ExecStartPre={{ matrix_host_command_sleep }} 5

2
roles/custom/matrix-dimension/tasks/setup_install.yml
View File

@ -55,7 +55,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_dimension_sqlite_database_path_local }}"
dst: "{{ matrix_dimension_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-ma1sd/tasks/setup_install.yml
View File

@ -31,7 +31,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_ma1sd_sqlite_database_path_local }}"
dst: "{{ matrix_ma1sd_database_connection_string }}"
caller: "{{ role_path | basename }}"

2
roles/custom/matrix-registration/tasks/setup_install.yml
View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres
vars:
devture_postgres_db_migration_request:
postgres_db_migration_request:
src: "{{ matrix_registration_sqlite_database_path_local }}"
dst: "{{ matrix_registration_database_connection_string }}"
caller: "{{ role_path | basename }}"

12
roles/custom/matrix-synapse/tasks/rust-synapse-compress-state/compress_room.yml
View File

@ -9,7 +9,7 @@
{{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-compress-room
--user={{ matrix_synapse_uid }}:{{ matrix_synapse_gid }}
--cap-drop=ALL
--network={{ devture_postgres_container_network }}
--network={{ postgres_container_network }}
--mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work
{{ matrix_synapse_rust_synapse_compress_state_docker_image }}
{{ matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path }} -t -o /work/state-compressor.sql
@ -31,15 +31,15 @@
ansible.builtin.set_fact:
matrix_synapse_rust_synapse_compress_state_psql_import_command: >-
{{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-psql-import
--user={{ devture_postgres_uid }}:{{ devture_postgres_gid }}
--user={{ postgres_uid }}:{{ postgres_gid }}
--cap-drop=ALL
--network={{ devture_postgres_container_network }}
--env-file={{ devture_postgres_base_path }}/env-postgres-psql
--network={{ postgres_container_network }}
--env-file={{ postgres_base_path }}/env-postgres-psql
--mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work,ro
--entrypoint=/bin/sh
{{ devture_postgres_container_image_latest }}
{{ postgres_container_image_latest }}
-c "cat /work/state-compressor.sql |
psql -v ON_ERROR_STOP=1 -h {{ devture_postgres_connection_hostname }} -d {{ matrix_synapse_database_database }}"
psql -v ON_ERROR_STOP=1 -h {{ postgres_connection_hostname }} -d {{ matrix_synapse_database_database }}"
- name: Import compression SQL into Postgres
ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_psql_import_command }}"

6
roles/custom/matrix-synapse/tasks/rust-synapse-compress-state/main.yml
View File

@ -3,8 +3,8 @@
- name: Fail if Postgres not enabled
ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot use rust-synapse-compress-state."
when: "not devture_postgres_enabled | bool"
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot use rust-synapse-compress-state."
when: "not postgres_enabled | bool"
# Defaults
@ -57,7 +57,7 @@
- name: Generate rust-synapse-compress-state room find SQL command
ansible.builtin.set_fact:
matrix_synapse_rust_synapse_compress_state_find_rooms_command: >-
{{ devture_postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "SELECT array_to_json(array_agg(row_to_json (r))) FROM (SELECT room_id, count(*) AS count FROM state_groups_state GROUP BY room_id HAVING count(*) > {{ matrix_synapse_rust_synapse_compress_state_min_state_groups_required }} ORDER BY count DESC) r;"
{{ postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "SELECT array_to_json(array_agg(row_to_json (r))) FROM (SELECT room_id, count(*) AS count FROM state_groups_state GROUP BY room_id HAVING count(*) > {{ matrix_synapse_rust_synapse_compress_state_min_state_groups_required }} ORDER BY count DESC) r;"
- name: Find rooms eligible for compression with rust-synapse-compress-state
ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_find_rooms_command }}"

6
roles/custom/matrix-synapse/tasks/update_user_password.yml
View File

@ -13,11 +13,11 @@
- name: Fail if not using integrated Postgres database
ansible.builtin.fail:
msg: "This command is working only when Postgres is installed via the the integrated com.devture.ansible.role.postgres role"
when: "not devture_postgres_enabled | bool"
when: "not postgres_enabled | bool"
- name: Ensure Postgres is started
ansible.builtin.service:
name: "{{ devture_postgres_identifier }}"
name: "{{ postgres_identifier }}"
state: started
daemon_reload: true
register: postgres_start_result
@ -42,7 +42,7 @@
- name: Generate user password-change SQL command
ansible.builtin.set_fact:
matrix_synapse_user_password_change_command: >-
{{ devture_postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "UPDATE users SET password_hash='{{ password_hash.stdout }}' WHERE name = '@{{ username }}:{{ matrix_domain }}'"
{{ postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "UPDATE users SET password_hash='{{ password_hash.stdout }}' WHERE name = '@{{ username }}:{{ matrix_domain }}'"
- name: Update user password hash
ansible.builtin.command:

24
roles/custom/matrix_playbook_migration/tasks/validate_config.yml
View File

@ -72,7 +72,7 @@
The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `devture_postgres_*`).
Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"
@ -362,3 +362,25 @@
msg: >-
We found usage of the following variables which are now removed: {{ matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars.keys() | join(', ') }}
when: "matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars | length > 0"
- name: (Deprecation) Catch and report devture_postgres_backup variables
ansible.builtin.fail:
msg: |-
The postgres-backup role in the playbook now lives under the MASH organization (https://github.com/mother-of-all-self-hosting/ansible-role-postgres-backup).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `devture_postgres_backup_`-prefixed variables (`devture_postgres_backup_*` -> `postgres_backup_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'devture_postgres_backup_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'devture_postgres_backup_.*') | list | items2dict"
- name: (Deprecation) Catch and report devture_postgres variables
ansible.builtin.fail:
msg: |-
The postgres role in the playbook now lives under the MASH organization (https://github.com/mother-of-all-self-hosting/ansible-role-postgres).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `devture_postgres_`-prefixed variables (`devture_postgres_*` -> `devture_postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'devture_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'devture_postgres_.*') | list | items2dict"