Switch postgres/postgres-backup Ansible role sources and adjust variable names (devture_postgres_ -> postgres_)

This commit is contained in:
Slavi Pantaleev 2024-09-27 09:37:24 +03:00
parent 62d66cc196
commit 8445843562
42 changed files with 346 additions and 324 deletions

View File

@ -711,7 +711,7 @@ From now on, the [Postgres Ansible role](https://github.com/devture/com.devture.
Our [Tuning PostgreSQL](docs/maintenance-postgres.md#tuning-postgresql) documentation page has details about how you can turn auto-tuning off or adjust the automatically-determined Postgres configuration parameters manually. Our [Tuning PostgreSQL](docs/maintenance-postgres.md#tuning-postgresql) documentation page has details about how you can turn auto-tuning off or adjust the automatically-determined Postgres configuration parameters manually.
People who [enable load-balancing with Synapse workers](docs/configuring-playbook-synapse.md#load-balancing-with-workers) no longer need to increase the maximum number of Postgres connections manually (previously done via `devture_postgres_process_extra_arguments`). There's a new variable (`devture_postgres_max_connections`) for controlling this number and the playbook automatically raises its value from `200` to `500` for setups which enable workers. People who [enable load-balancing with Synapse workers](docs/configuring-playbook-synapse.md#load-balancing-with-workers) no longer need to increase the maximum number of Postgres connections manually (previously done via `postgres_process_extra_arguments`). There's a new variable (`postgres_max_connections`) for controlling this number and the playbook automatically raises its value from `200` to `500` for setups which enable workers.
# 2023-08-31 # 2023-08-31
@ -1232,14 +1232,14 @@ See our [Setting up matrix-bot-chatgpt](docs/configuring-playbook-bot-chatgpt.md
Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup). Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix. You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `postgres_backup` prefix.
# 2022-11-28 # 2022-11-28
## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role ## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role
**TLDR**: the tasks that install the integrated Postgres server now live in an external role - [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres). You'll need to run `make roles` to install it, and to also rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix (e.g. `matrix_postgres_connection_password` -> `devture_postgres_connection_password`). All your data will still be there! Some scripts have moved (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`). **TLDR**: the tasks that install the integrated Postgres server now live in an external role - [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres). You'll need to run `make roles` to install it, and to also rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix (e.g. `matrix_postgres_connection_password` -> `postgres_connection_password`). All your data will still be there! Some scripts have moved (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`).
The `matrix-postgres` role that has been part of the playbook for a long time has been replaced with the [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres) role. This was done as part of our work to [use external roles for some things](#the-playbook-now-uses-external-roles-for-some-things) for better code re-use and maintainability. The `matrix-postgres` role that has been part of the playbook for a long time has been replaced with the [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres) role. This was done as part of our work to [use external roles for some things](#the-playbook-now-uses-external-roles-for-some-things) for better code re-use and maintainability.

View File

@ -10,7 +10,7 @@ If you'd like to use an external PostgreSQL server that you manage, you can edit
If you'd like to use an external Postgres server, use a custom `vars.yml` configuration like this: If you'd like to use an external Postgres server, use a custom `vars.yml` configuration like this:
```yaml ```yaml
devture_postgres_enabled: false postgres_enabled: false
# Rewire Synapse to use your external Postgres server # Rewire Synapse to use your external Postgres server
matrix_synapse_database_host: "your-postgres-server-hostname" matrix_synapse_database_host: "your-postgres-server-hostname"

View File

@ -123,7 +123,7 @@ To import the Synapse media store, you're supposed to invoke the `import_synapse
This guide here is adapted from the [upstream documentation about the import_synapse script](https://github.com/turt2live/matrix-media-repo#importing-media-from-synapse). This guide here is adapted from the [upstream documentation about the import_synapse script](https://github.com/turt2live/matrix-media-repo#importing-media-from-synapse).
Run the following command on the server (after replacing `devture_postgres_connection_password` in it with the value found in your `vars.yml` file): Run the following command on the server (after replacing `postgres_connection_password` in it with the value found in your `vars.yml` file):
```sh ```sh
docker exec -it matrix-media-repo \ docker exec -it matrix-media-repo \
@ -132,7 +132,7 @@ docker exec -it matrix-media-repo \
-dbHost matrix-postgres \ -dbHost matrix-postgres \
-dbPort 5432 \ -dbPort 5432 \
-dbUsername matrix \ -dbUsername matrix \
-dbPassword devture_postgres_connection_password -dbPassword postgres_connection_password
``` ```
Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing. Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing.
@ -145,7 +145,7 @@ If you're using the [Dendrite](configuring-playbook-dendrite.md) homeserver inst
To import the Dendrite media store, you're supposed to invoke the `import_dendrite` tool which is part of the matrix-media-repo container image. Your Dendrite database is called `dendrite_mediaapi` by default, unless you've changed it by modifying `matrix_dendrite_media_api_database`. To import the Dendrite media store, you're supposed to invoke the `import_dendrite` tool which is part of the matrix-media-repo container image. Your Dendrite database is called `dendrite_mediaapi` by default, unless you've changed it by modifying `matrix_dendrite_media_api_database`.
Run the following command on the server (after replacing `devture_postgres_connection_password` in it with the value found in your `vars.yml` file): Run the following command on the server (after replacing `postgres_connection_password` in it with the value found in your `vars.yml` file):
```sh ```sh
docker exec -it matrix-media-repo \ docker exec -it matrix-media-repo \
@ -154,7 +154,7 @@ docker exec -it matrix-media-repo \
-dbHost matrix-postgres \ -dbHost matrix-postgres \
-dbPort 5432 \ -dbPort 5432 \
-dbUsername matrix \ -dbUsername matrix \
-dbPassword devture_postgres_connection_password -dbPassword postgres_connection_password
``` ```
Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing. Enter `1` for the Machine ID when prompted (you are not doing any horizontal scaling) unless you know what you're doing.

View File

@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup: Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
```yaml ```yaml
devture_postgres_backup_enabled: true postgres_backup_enabled: true
``` ```
Refer to the table below for additional configuration variables and their default values. Refer to the table below for additional configuration variables and their default values.
@ -18,13 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
| Name | Default value | Description | | Name | Default value | Description |
| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- | | :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
|`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups| |`postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.| |`postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep| |`postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep| |`postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep| |`postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`devture_postgres_backup_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_backup_data_path` | |`postgres_backup_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `postgres_backup_data_path` |
|`devture_postgres_backup_data_path` | `"{{ devture_postgres_backup_base_path }}/data"` | Storage path for postgres-backup database backups | |`postgres_backup_data_path` | `"{{ postgres_backup_base_path }}/data"` | Storage path for postgres-backup database backups |
## Installing ## Installing

View File

@ -75,7 +75,7 @@ The only thing you **cannot** do is mix [generic workers](#generic-workers) and
#### Effect of enabling workers on the rest of your server #### Effect of enabling workers on the rest of your server
When Synapse workers are enabled, the integrated [Postgres database is tuned](maintenance-postgres.md#tuning-postgresql), so that the maximum number of Postgres connections are increased from `200` to `500`. If you need to decrease or increase the number of maximum Postgres connections further, use the `devture_postgres_max_connections` variable. When Synapse workers are enabled, the integrated [Postgres database is tuned](maintenance-postgres.md#tuning-postgresql), so that the maximum number of Postgres connections are increased from `200` to `500`. If you need to decrease or increase the number of maximum Postgres connections further, use the `postgres_max_connections` variable.
A separate Ansible role (`matrix-synapse-reverse-proxy-companion`) and component handles load-balancing for workers. This role/component is automatically enabled when you enable workers. Make sure to use the `setup-all` tag (not `install-all`!) during the playbook's [installation](./installing.md) process, especially if you're disabling workers, so that components may be installed/uninstalled correctly. A separate Ansible role (`matrix-synapse-reverse-proxy-companion`) and component handles load-balancing for workers. This role/component is automatically enabled when you enable workers. Make sure to use the `setup-all` tag (not `install-all`!) during the playbook's [installation](./installing.md) process, especially if you're disabling workers, so that components may be installed/uninstalled correctly.

View File

@ -107,12 +107,12 @@ Example: `--extra-vars="postgres_dump_name=matrix-postgres-dump.sql"`
PostgreSQL can be [tuned](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server) to make it run faster. This is done by passing extra arguments to the Postgres process. PostgreSQL can be [tuned](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server) to make it run faster. This is done by passing extra arguments to the Postgres process.
The [Postgres Ansible role](https://github.com/devture/com.devture.ansible.role.postgres) **already does some tuning by default**, which matches the [tuning logic](https://github.com/le0pard/pgtune/blob/master/src/features/configuration/configurationSlice.js) done by websites like https://pgtune.leopard.in.ua/. The [Postgres Ansible role](https://github.com/devture/com.devture.ansible.role.postgres) **already does some tuning by default**, which matches the [tuning logic](https://github.com/le0pard/pgtune/blob/master/src/features/configuration/configurationSlice.js) done by websites like https://pgtune.leopard.in.ua/.
You can manually influence some of the tuning variables . These parameters (variables) are injected via the `devture_postgres_postgres_process_extra_arguments_auto` variable. You can manually influence some of the tuning variables . These parameters (variables) are injected via the `postgres_postgres_process_extra_arguments_auto` variable.
Most users should be fine with the automatically-done tuning. However, you may wish to: Most users should be fine with the automatically-done tuning. However, you may wish to:
- **adjust the automatically-determined tuning parameters manually**: change the values for the tuning variables defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) (see `devture_postgres_max_connections`, `devture_postgres_data_storage` etc). These variables are ultimately passed to Postgres via a `devture_postgres_postgres_process_extra_arguments_auto` variable - **adjust the automatically-determined tuning parameters manually**: change the values for the tuning variables defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) (see `postgres_max_connections`, `postgres_data_storage` etc). These variables are ultimately passed to Postgres via a `postgres_postgres_process_extra_arguments_auto` variable
- **turn automatically-performed tuning off**: override it like this: `devture_postgres_postgres_process_extra_arguments_auto: []` - **turn automatically-performed tuning off**: override it like this: `postgres_postgres_process_extra_arguments_auto: []`
- **add additional tuning parameters**: define your additional Postgres configuration parameters in `devture_postgres_postgres_process_extra_arguments_custom`. See `devture_postgres_postgres_process_extra_arguments_auto` defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) for inspiration - **add additional tuning parameters**: define your additional Postgres configuration parameters in `postgres_postgres_process_extra_arguments_custom`. See `postgres_postgres_process_extra_arguments_auto` defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) for inspiration

View File

@ -40,7 +40,7 @@ devture_traefik_config_certificatesResolvers_acme_email: ''
# #
# The playbook creates additional Postgres users and databases (one for each enabled service) # The playbook creates additional Postgres users and databases (one for each enabled service)
# using this superuser account. # using this superuser account.
devture_postgres_connection_password: '' postgres_connection_password: ''
# By default, we configure Coturn's external IP address using the value specified for `ansible_host` in your `inventory/hosts` file. # By default, we configure Coturn's external IP address using the value specified for `ansible_host` in your `inventory/hosts` file.
# If this value is an external IP address, you can skip this section. # If this value is an external IP address, you can skip this section.

File diff suppressed because it is too large Load Diff

View File

@ -42,11 +42,11 @@
- src: git+https://github.com/devture/com.devture.ansible.role.playbook_state_preserver.git - src: git+https://github.com/devture/com.devture.ansible.role.playbook_state_preserver.git
version: ff2fd42e1c1a9e28e3312bbd725395f9c2fc7f16 version: ff2fd42e1c1a9e28e3312bbd725395f9c2fc7f16
name: playbook_state_preserver name: playbook_state_preserver
- src: git+https://github.com/devture/com.devture.ansible.role.postgres.git - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-postgres.git
version: v16.4-0 version: v17.0-0
name: postgres name: postgres
- src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-postgres-backup.git
version: ccfd8db07fd8725119f0e06ba5144b8f58a67890 version: v16-0
name: postgres_backup name: postgres_backup
- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus.git - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus.git
version: v2.54.1-1 version: v2.54.1-1

View File

@ -15,7 +15,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_bot_buscarron_sqlite_database_path_local }}" src: "{{ matrix_bot_buscarron_sqlite_database_path_local }}"
dst: "{{ matrix_bot_buscarron_database_connection_string }}" dst: "{{ matrix_bot_buscarron_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -15,7 +15,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_bot_honoroit_sqlite_database_path_local }}" src: "{{ matrix_bot_honoroit_sqlite_database_path_local }}"
dst: "{{ matrix_bot_honoroit_database_connection_string }}" dst: "{{ matrix_bot_honoroit_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}" src: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}"
dst: "{{ matrix_bot_matrix_reminder_bot_database_connection_string }}" dst: "{{ matrix_bot_matrix_reminder_bot_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -12,7 +12,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_bot_postmoogle_sqlite_database_path_local }}" src: "{{ matrix_bot_postmoogle_sqlite_database_path_local }}"
dst: "{{ matrix_bot_postmoogle_database_connection_string }}" dst: "{{ matrix_bot_postmoogle_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_appservice_discord_sqlite_database_path_local }}" src: "{{ matrix_appservice_discord_sqlite_database_path_local }}"
dst: "{{ matrix_appservice_discord_database_connString }}" dst: "{{ matrix_appservice_discord_database_connString }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -2,8 +2,8 @@
- name: Fail if Postgres not enabled - name: Fail if Postgres not enabled
ansible.builtin.fail: ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot migrate." msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot migrate."
when: "not devture_postgres_enabled | bool" when: "not postgres_enabled | bool"
# Defaults # Defaults
@ -16,7 +16,7 @@
- name: Ensure Postgres is started - name: Ensure Postgres is started
ansible.builtin.service: ansible.builtin.service:
name: "{{ devture_postgres_identifier }}" name: "{{ postgres_identifier }}"
state: started state: started
daemon_reload: true daemon_reload: true
register: postgres_service_start_result register: postgres_service_start_result

View File

@ -2,8 +2,8 @@
- name: Fail if Postgres not enabled - name: Fail if Postgres not enabled
ansible.builtin.fail: ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot migrate." msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot migrate."
when: "not devture_postgres_enabled | bool" when: "not postgres_enabled | bool"
# Defaults # Defaults
@ -16,7 +16,7 @@
- name: Ensure Postgres is started - name: Ensure Postgres is started
ansible.builtin.service: ansible.builtin.service:
name: "{{ devture_postgres_identifier }}" name: "{{ postgres_identifier }}"
state: started state: started
daemon_reload: true daemon_reload: true
register: postgres_service_start_result register: postgres_service_start_result

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_go_skype_bridge_sqlite_database_path_local }}" src: "{{ matrix_go_skype_bridge_sqlite_database_path_local }}"
dst: "{{ matrix_go_skype_bridge_database_connection_string }}" dst: "{{ matrix_go_skype_bridge_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_discord_sqlite_database_path_local }}" src: "{{ matrix_mautrix_discord_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_discord_database_connection_string }}" dst: "{{ matrix_mautrix_discord_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_facebook_sqlite_database_path_local }}" src: "{{ matrix_mautrix_facebook_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_facebook_database_connection_string }}" dst: "{{ matrix_mautrix_facebook_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_gmessages_sqlite_database_path_local }}" src: "{{ matrix_mautrix_gmessages_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_gmessages_database_connection_string }}" dst: "{{ matrix_mautrix_gmessages_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_googlechat_sqlite_database_path_local }}" src: "{{ matrix_mautrix_googlechat_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_googlechat_database_connection_string }}" dst: "{{ matrix_mautrix_googlechat_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_hangouts_sqlite_database_path_local }}" src: "{{ matrix_mautrix_hangouts_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_hangouts_database_connection_string }}" dst: "{{ matrix_mautrix_hangouts_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_meta_instagram_sqlite_database_path_local }}" src: "{{ matrix_mautrix_meta_instagram_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_meta_instagram_database_connection_string }}" dst: "{{ matrix_mautrix_meta_instagram_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_meta_messenger_sqlite_database_path_local }}" src: "{{ matrix_mautrix_meta_messenger_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_meta_messenger_database_connection_string }}" dst: "{{ matrix_mautrix_meta_messenger_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_signal_sqlite_database_path_local }}" src: "{{ matrix_mautrix_signal_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_signal_database_connection_string }}" dst: "{{ matrix_mautrix_signal_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_slack_sqlite_database_path_local }}" src: "{{ matrix_mautrix_slack_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_slack_database_connection_string }}" dst: "{{ matrix_mautrix_slack_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_telegram_sqlite_database_path_local }}" src: "{{ matrix_mautrix_telegram_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_telegram_database_connection_string }}" dst: "{{ matrix_mautrix_telegram_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mautrix_whatsapp_sqlite_database_path_local }}" src: "{{ matrix_mautrix_whatsapp_sqlite_database_path_local }}"
dst: "{{ matrix_mautrix_whatsapp_database_connection_string }}" dst: "{{ matrix_mautrix_whatsapp_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -50,7 +50,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_discord_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_discord_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_discord_database_connection_string }}" dst: "{{ matrix_mx_puppet_discord_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_groupme_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_groupme_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_groupme_database_connection_string }}" dst: "{{ matrix_mx_puppet_groupme_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_instagram_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_instagram_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_instagram_database_connection_string }}" dst: "{{ matrix_mx_puppet_instagram_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -44,7 +44,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_slack_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_slack_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_slack_database_connection_string }}" dst: "{{ matrix_mx_puppet_slack_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_steam_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_steam_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_steam_database_connection_string }}" dst: "{{ matrix_mx_puppet_steam_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -51,7 +51,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_mx_puppet_twitter_sqlite_database_path_local }}" src: "{{ matrix_mx_puppet_twitter_sqlite_database_path_local }}"
dst: "{{ matrix_mx_puppet_twitter_database_connection_string }}" dst: "{{ matrix_mx_puppet_twitter_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@ Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} matrix-dendrite 2>/dev/null || true' ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} stop --time={{ devture_systemd_docker_base_container_stop_grace_time_seconds }} matrix-dendrite 2>/dev/null || true'
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-dendrite 2>/dev/null || true' ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-dendrite 2>/dev/null || true'
{% if (devture_postgres_identifier + '.service') in matrix_dendrite_systemd_required_services_list %} {% if (postgres_identifier + '.service') in matrix_dendrite_systemd_required_services_list %}
# Dendrite is too quick to start in relation to its Postgres dependency. # Dendrite is too quick to start in relation to its Postgres dependency.
# Delay Dendrite startup to avoid failing with: "failed to connect to accounts db" ("pq: the database system is starting up"). # Delay Dendrite startup to avoid failing with: "failed to connect to accounts db" ("pq: the database system is starting up").
ExecStartPre={{ matrix_host_command_sleep }} 5 ExecStartPre={{ matrix_host_command_sleep }} 5

View File

@ -55,7 +55,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_dimension_sqlite_database_path_local }}" src: "{{ matrix_dimension_sqlite_database_path_local }}"
dst: "{{ matrix_dimension_database_connection_string }}" dst: "{{ matrix_dimension_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -31,7 +31,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_ma1sd_sqlite_database_path_local }}" src: "{{ matrix_ma1sd_sqlite_database_path_local }}"
dst: "{{ matrix_ma1sd_database_connection_string }}" dst: "{{ matrix_ma1sd_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -16,7 +16,7 @@
name: galaxy/com.devture.ansible.role.postgres name: galaxy/com.devture.ansible.role.postgres
tasks_from: migrate_db_to_postgres tasks_from: migrate_db_to_postgres
vars: vars:
devture_postgres_db_migration_request: postgres_db_migration_request:
src: "{{ matrix_registration_sqlite_database_path_local }}" src: "{{ matrix_registration_sqlite_database_path_local }}"
dst: "{{ matrix_registration_database_connection_string }}" dst: "{{ matrix_registration_database_connection_string }}"
caller: "{{ role_path | basename }}" caller: "{{ role_path | basename }}"

View File

@ -9,7 +9,7 @@
{{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-compress-room {{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-compress-room
--user={{ matrix_synapse_uid }}:{{ matrix_synapse_gid }} --user={{ matrix_synapse_uid }}:{{ matrix_synapse_gid }}
--cap-drop=ALL --cap-drop=ALL
--network={{ devture_postgres_container_network }} --network={{ postgres_container_network }}
--mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work --mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work
{{ matrix_synapse_rust_synapse_compress_state_docker_image }} {{ matrix_synapse_rust_synapse_compress_state_docker_image }}
{{ matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path }} -t -o /work/state-compressor.sql {{ matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path }} -t -o /work/state-compressor.sql
@ -31,15 +31,15 @@
ansible.builtin.set_fact: ansible.builtin.set_fact:
matrix_synapse_rust_synapse_compress_state_psql_import_command: >- matrix_synapse_rust_synapse_compress_state_psql_import_command: >-
{{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-psql-import {{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-psql-import
--user={{ devture_postgres_uid }}:{{ devture_postgres_gid }} --user={{ postgres_uid }}:{{ postgres_gid }}
--cap-drop=ALL --cap-drop=ALL
--network={{ devture_postgres_container_network }} --network={{ postgres_container_network }}
--env-file={{ devture_postgres_base_path }}/env-postgres-psql --env-file={{ postgres_base_path }}/env-postgres-psql
--mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work,ro --mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work,ro
--entrypoint=/bin/sh --entrypoint=/bin/sh
{{ devture_postgres_container_image_latest }} {{ postgres_container_image_latest }}
-c "cat /work/state-compressor.sql | -c "cat /work/state-compressor.sql |
psql -v ON_ERROR_STOP=1 -h {{ devture_postgres_connection_hostname }} -d {{ matrix_synapse_database_database }}" psql -v ON_ERROR_STOP=1 -h {{ postgres_connection_hostname }} -d {{ matrix_synapse_database_database }}"
- name: Import compression SQL into Postgres - name: Import compression SQL into Postgres
ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_psql_import_command }}" ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_psql_import_command }}"

View File

@ -3,8 +3,8 @@
- name: Fail if Postgres not enabled - name: Fail if Postgres not enabled
ansible.builtin.fail: ansible.builtin.fail:
msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`devture_postgres_enabled`). Cannot use rust-synapse-compress-state." msg: "Postgres via the com.devture.ansible.role.postgres role is not enabled (`postgres_enabled`). Cannot use rust-synapse-compress-state."
when: "not devture_postgres_enabled | bool" when: "not postgres_enabled | bool"
# Defaults # Defaults
@ -57,7 +57,7 @@
- name: Generate rust-synapse-compress-state room find SQL command - name: Generate rust-synapse-compress-state room find SQL command
ansible.builtin.set_fact: ansible.builtin.set_fact:
matrix_synapse_rust_synapse_compress_state_find_rooms_command: >- matrix_synapse_rust_synapse_compress_state_find_rooms_command: >-
{{ devture_postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "SELECT array_to_json(array_agg(row_to_json (r))) FROM (SELECT room_id, count(*) AS count FROM state_groups_state GROUP BY room_id HAVING count(*) > {{ matrix_synapse_rust_synapse_compress_state_min_state_groups_required }} ORDER BY count DESC) r;" {{ postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "SELECT array_to_json(array_agg(row_to_json (r))) FROM (SELECT room_id, count(*) AS count FROM state_groups_state GROUP BY room_id HAVING count(*) > {{ matrix_synapse_rust_synapse_compress_state_min_state_groups_required }} ORDER BY count DESC) r;"
- name: Find rooms eligible for compression with rust-synapse-compress-state - name: Find rooms eligible for compression with rust-synapse-compress-state
ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_find_rooms_command }}" ansible.builtin.command: "{{ matrix_synapse_rust_synapse_compress_state_find_rooms_command }}"

View File

@ -13,11 +13,11 @@
- name: Fail if not using integrated Postgres database - name: Fail if not using integrated Postgres database
ansible.builtin.fail: ansible.builtin.fail:
msg: "This command is working only when Postgres is installed via the the integrated com.devture.ansible.role.postgres role" msg: "This command is working only when Postgres is installed via the the integrated com.devture.ansible.role.postgres role"
when: "not devture_postgres_enabled | bool" when: "not postgres_enabled | bool"
- name: Ensure Postgres is started - name: Ensure Postgres is started
ansible.builtin.service: ansible.builtin.service:
name: "{{ devture_postgres_identifier }}" name: "{{ postgres_identifier }}"
state: started state: started
daemon_reload: true daemon_reload: true
register: postgres_start_result register: postgres_start_result
@ -42,7 +42,7 @@
- name: Generate user password-change SQL command - name: Generate user password-change SQL command
ansible.builtin.set_fact: ansible.builtin.set_fact:
matrix_synapse_user_password_change_command: >- matrix_synapse_user_password_change_command: >-
{{ devture_postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "UPDATE users SET password_hash='{{ password_hash.stdout }}' WHERE name = '@{{ username }}:{{ matrix_domain }}'" {{ postgres_bin_path }}/cli-non-interactive --dbname={{ matrix_synapse_database_database | quote }} -c "UPDATE users SET password_hash='{{ password_hash.stdout }}' WHERE name = '@{{ username }}:{{ matrix_domain }}'"
- name: Update user password hash - name: Update user password hash
ansible.builtin.command: ansible.builtin.command:

View File

@ -72,7 +72,7 @@
The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup). The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup).
The new role is pretty much the same, but uses differently named variables. The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `devture_postgres_*`). Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }} The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict" when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"
@ -362,3 +362,25 @@
msg: >- msg: >-
We found usage of the following variables which are now removed: {{ matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars.keys() | join(', ') }} We found usage of the following variables which are now removed: {{ matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars.keys() | join(', ') }}
when: "matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars | length > 0" when: "matrix_playbook_migration_matrix_synapse_reverse_proxy_companion_container_labels_public_client_synapse_oidc_vars | length > 0"
- name: (Deprecation) Catch and report devture_postgres_backup variables
ansible.builtin.fail:
msg: |-
The postgres-backup role in the playbook now lives under the MASH organization (https://github.com/mother-of-all-self-hosting/ansible-role-postgres-backup).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `devture_postgres_backup_`-prefixed variables (`devture_postgres_backup_*` -> `postgres_backup_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'devture_postgres_backup_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'devture_postgres_backup_.*') | list | items2dict"
- name: (Deprecation) Catch and report devture_postgres variables
ansible.builtin.fail:
msg: |-
The postgres role in the playbook now lives under the MASH organization (https://github.com/mother-of-all-self-hosting/ansible-role-postgres).
The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `devture_postgres_`-prefixed variables (`devture_postgres_*` -> `devture_postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'devture_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | selectattr('key', 'match', 'devture_postgres_.*') | list | items2dict"