Commit Graph

84 Commits

Author SHA1 Message Date
28f6091ed4 GoMatrixHosting v0.7.0 2022-02-27 17:40:20 +08:00
7e5b88c3b7 fix: all praise the allmighty yamllinter 2022-02-05 21:32:54 +01:00
c4d2c8394c Abort if on an unsupported Postgres version (v9.6)
Official support ends today (2021-11-11).
Synapse still supports v9.6, but we'd better force users to transition
to newer versions anyway.
2021-11-11 15:42:34 +02:00
735c966ab6 Disable systemd services when stopping to uninstall them
Until now, we were leaving services "enabled"
(symlinks in /etc/systemd/system/multi-user.target.wants/).

We clean these up now. Broken symlinks may still exist in older
installations that enabled/disabled services. We're not taking care
to fix these up. It's just a cosmetic defect anyway.
2021-11-10 17:39:21 +02:00
096c960b84 Add support for Postgres v14 2021-10-01 11:27:40 +03:00
bee14550ab Fix local/bin scripts autocompletion by adding rx perms to everyone
It's mildly annoying when trying to execute these scripts while logged
in as a regular user, as the missing execute permissions will hinder
autocompletion even when trying to use with sudo.

These shell scripts don't contain secrets, but may fail when ran by a
regular user. The failure is due to the lack of access to the /matrix
directory, and does not result in any damage.
2021-05-28 10:39:27 +02:00
1ab507349c Fix self-building for various components on Ansible < 2.8
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1070
2021-05-20 08:43:20 +03:00
a56cb34850 Notify people if /matrix/postgres/data-auto-upgrade-backup exists 2021-01-23 14:14:45 +02:00
a2422c458a Notify of remaining matrix-postgres local data in a better way 2021-01-23 14:04:51 +02:00
f9968b6981 Fix matrix_postgres_connection_password length check 2021-01-22 21:22:58 +02:00
88addd71fc Fix Postgres imports going to the matrix DB by default
Well, they still do go to that DB by default,
but our docs give a better command to users, which would do the right
thing.
2021-01-22 17:39:08 +02:00
95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
05ca9357a8 Add .service suffix to systemd units list
We'll be adding `.timer` units later on, so it's good to be
more explicit.
2021-01-14 23:02:10 +02:00
fb83eccf99 Relocate SQL template file 2021-01-03 00:58:31 +02:00
86da489b9b Never fail when stopping systemd service during (SQLite -> Postgres) migration
We need to suppress systemd service-stopping requests in certain rare
cases like https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/771

That issue seems to describe a case, where a migration from mxisd to
ma1sd was happening (DB files had just been moved), and then we were
attemping to stop `matrix-ma1sd.service` so we could import that database into
Postgres. However, there's neither `matrix-mxisd.service`, nor
`matrix-ma1sd.service` after `migrate_mxisd.yml` had just run, so
stopping `matrix-ma1sd.service` was failing.
2020-12-29 10:31:20 +02:00
21662af3be Archive database only after additional_psql_statements_list had executed 2020-12-23 14:12:11 +02:00
019a4d7dcd Use role relative paths for things 2020-12-23 11:34:48 +00:00
ad1425eee4 Add pgloader self-building support (for ARM) 2020-12-23 09:08:54 +02:00
8675dedbdb Add support for automatic (nedb -> Postgres) migration to matrix-appservice-slack 2020-12-22 19:56:52 +02:00
715bdf2c64 Add support for automatic (nedb -> Postgres) migration to mx-appservice-irc 2020-12-22 19:32:43 +02:00
1bd5c240e5 Add support for executing additional DB migration statements
In cases where pgloader is not enough and we need to do some additional
migration work after it, we can now use
`additional_psql_statements_list` and
`additional_psql_statements_db_name`.

This is to be used when migrating `matrix-registration`'s data at the
very least.
2020-12-15 23:18:29 +02:00
dd797ba6a7 Fix Postgres database importing/upgrading conflicts
We were running into conflicts, because having initialized
the roles (users) and databases, trying to import leads to
errors (role XXX already exists, etc.).

We were previously ignoring the Synapse database (`homeserver`)
when upgrading/importing, because that one gets created by default
whenever the container starts.

For our additional databases, it's a similar situation now.
It's not created by default as soon as Postgres starts with an empty
database, but rather we create it as part of running the playbook.

So we either need to skip those role/database creation statements
while upgrading/importing, or to avoid creating the additional database
and rely on the import for that. I've gone for the former, because
it's already similar to what we were doing and it's simpler
(it lets `setup_postgres.yml` be the same in all scenarios).
2020-12-14 22:28:20 +02:00
13d8a9b39c hint supported automatic migration nedb->postgres 2020-12-14 16:33:40 +01:00
af3ea67bba Add (SQLite + Postgres) support and automatic migration to matrix-ma1sd 2020-12-14 17:16:25 +02:00
c1431b28f0 Make use of matrix_postgres_db_migration_request.caller 2020-12-14 16:13:57 +02:00
dc7850e83c Fix wording and variable names a bit 2020-12-14 16:03:40 +02:00
bc376c2fb2 Add database migration utility to matrix-postgres role 2020-12-14 16:03:40 +02:00
5dba0c038b Make --tags=import-generic-sqlite-db commands not pass a sensitive connection string around
Instead of passing the connection string, we can now pass a name of a
variable, which contains a connection string.

Both are supported for having extra flexibility.
2020-12-14 11:47:00 +02:00
cb969c6ca2 Add --tags=import-generic-sqlite-db (pgloader import)
This can be used by various bridges, etc., to import an SQLite
(or some other supported) database into Postgres.
2020-12-14 02:23:29 +02:00
c66c084027 Merge branch 'master' into postgres-per-default 2020-12-14 01:51:15 +02:00
6e1dfb62f0 Rename some doc files and commands related to importing
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.

Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.

We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
2020-12-14 01:51:00 +02:00
183d2a10db Ensure matrix-postgres.service is started before creating additional users/databases 2020-12-14 00:59:59 +02:00
46a4034d3e Use "password" for additional Postgres databases, not "pass"
Being more explicit sounds better.
2020-12-14 00:43:03 +02:00
3a037a5993 Ensure additional databases contain all the keys that we expect 2020-12-14 00:39:38 +02:00
da4cb2f639 Do not use the postgresql_user/postgresql_db modules
While these modules are really nice and helpful, we can't use them
for at least 2 reasons:

- for us, Postgres runs in a container on a private Docker network
(`--network=matrix`) without usually being exposed to the host.
These modules execute on the host so they won't be able to reach it.

- these modules require `psycopg2`, so we need to install it before
using it. This might or might not be its own can of worms.
2020-12-14 00:31:38 +02:00
bbc09d013b Do not execute additional databases creation code if not necessary
The tasks in `create_additional_databases.yml` will likely
ensure `matrix-postgres.service` is started, etc.

If no additional databases are defined, we'd rather not execute that
file and all these tasks that it may do in the future.
2020-12-13 23:46:05 +02:00
c765ceb270 Prevent weird loop error
> Invalid data passed to 'loop', it requires a list, got this instead: matrix_postgres_additional_databases. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup.

Well, or working around it, as I've done in this commit (which seems
more sane than `wantlist=True` stuff).
2020-12-13 22:56:56 +02:00
e2952f16f7 Determine matrix-postgres IP address without relying on jq
To avoid needing to have `jq` installed on the machine, we could:
- try to run jq in a Docker container using some small image providing
that
- better yet, avoid `jq` altogether
2020-12-13 22:45:48 +02:00
f47e8a97e6 Make use of matrix_host_command_docker instead of hardcoding 2020-12-13 22:38:35 +02:00
0641106370 Allow username of additional Postgres databases to be different
We'll most likely use one that matches the database name, but
it's better to have it configurable.
2020-12-13 22:37:04 +02:00
527d5f57d5 Relocate Postgres additional database creation logic
Moving it above the "uninstalling" set of tasks is better.
Extracting it out to another file at the same time, for readability,
especially given that it will probably have to become more complex in
the future (potentially installing `jq`, etc.)
2020-12-13 22:37:04 +02:00
77a5c7cf3c Merge branch 'master' into postgres-per-default 2020-12-13 21:04:15 +02:00
47613e5a27 Remove synapse-janitor support
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/746
2020-12-11 23:24:42 +02:00
d9f4914e0d WIP: postgres: create databases for all services
If a service is enabled, a database for it is created in postgres with a uniqque password. The service can then use this database for data storage instead of relying on sqlite.
2020-12-10 18:26:22 +01:00
796c752b60 Ensure Postgres passwords are not longer than 99 characters
Complements https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/732
2020-11-26 09:51:48 +02:00
75f9fde7a4 Remove some more -v usage
Continuation of 1fca917ad1.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/722
2020-11-25 10:49:59 +02:00
6a72e3fa54 Try to make importing SQLite from older Synapse version work
If the SQLite database was from an older version of Synapse, it appears
that Synapse would try to run migrations on it first, before importing.
This was failing, because the file wasn't writable.

Hopefully, this fixes the problem.
2020-10-07 08:54:46 +03:00
3a3383fada Add support for postgres 13 2020-09-30 16:50:59 -05:00
a456e3a9e7 Surface certain messages at the end of playbook execution
Fixes #106 (Github Issue).
2020-09-01 13:12:35 +03:00
e3dca2f66f Try to avoid Docker logs growing too much for one-off containers
We recently had a report of the Postgres backup container's log file
growing the size of /var/lib/docker until it ran out of disk space.

Trying to prevent similar problems in the future.
2020-09-01 09:03:48 +03:00