Commit Graph

2484 Commits

Author SHA1 Message Date
0790a7b2a8 Add support for matrix_dimension_systemd_{required,wanted}_services_list
We were referencing them from `group_vars/matrix_servers` since
recently, but there were no such variables and they weren't being put to
use.
2020-12-14 20:31:07 +02:00
374f43735a Separate matrix-dimension install/uninstall tasks 2020-12-14 20:05:31 +02:00
8d74593878 Prepare matrix-registation for (SQLite + Postgres) support
Auto-migration and everything seems to work. It's just that
matrix-registration cannot load the Python modules required
for talking to a Postgres database.

Tracked here: https://github.com/ZerataX/matrix-registration/issues/44

Until this gets fixed, we'll continue default to 'sqlite'.
2020-12-14 18:58:37 +02:00
516ccb2b2b Separate matrix-registration install/uninstall tasks 2020-12-14 18:12:14 +02:00
13d8a9b39c hint supported automatic migration nedb->postgres 2020-12-14 16:33:40 +01:00
af3ea67bba Add (SQLite + Postgres) support and automatic migration to matrix-ma1sd 2020-12-14 17:16:25 +02:00
0ca48f3532 Separate matrix-ma1sd install/uninstall tasks 2020-12-14 16:57:51 +02:00
7248eb3c11 Fix syntax error in roles/matrix-bridge-appservice-irc/defaults/main.yml 2020-12-14 16:25:44 +02:00
cba973d6b5 Enable automatic (SQLite -> Postgres) migration for matrix-appservice-discord 2020-12-14 16:25:22 +02:00
13f84e2ad5 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-whatsapp 2020-12-14 16:21:01 +02:00
86a8091768 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-telegram 2020-12-14 16:19:54 +02:00
3ba8520266 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-hangouts 2020-12-14 16:18:38 +02:00
bbc08722c5 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-facebook 2020-12-14 16:14:23 +02:00
c1431b28f0 Make use of matrix_postgres_db_migration_request.caller 2020-12-14 16:13:57 +02:00
ac37091d01 Enable automatic (SQLite -> Postgres) migration for matrix-reminder-bot 2020-12-14 16:03:40 +02:00
dc7850e83c Fix wording and variable names a bit 2020-12-14 16:03:40 +02:00
bc376c2fb2 Add database migration utility to matrix-postgres role 2020-12-14 16:03:40 +02:00
54da61f81b add postgres support mx-appservice-[slack|irc] with fallback to nedb in role and migration notice 2020-12-14 14:08:35 +01:00
e3a0c9adda Add (Postgres + SQLite) support to matrix-reminder-bot
This has been tested and appears to work.
2020-12-14 15:02:11 +02:00
dde1c9f899 Fix indentation causing YAML syntax error 2020-12-14 14:53:35 +02:00
aa828ff9f6 Separate matrix-reminder-bot install/uninstall tasks 2020-12-14 14:50:04 +02:00
b9a04a7f95 Rename some remaining matrix_*_postgres_* vars back to matrix_*_database_*
Looks like there are some that I missed in 087dbe4ddc
2020-12-14 14:42:18 +02:00
5d70bc1376 add postgres support for mx-puppet-* with fallback to sqlite in role and migration notice 2020-12-14 13:22:58 +01:00
087dbe4ddc Rename matrix_*_postgres_* back to matrix_*_database_*
I was thinking that it makes sense to be more specific,
and using `_postgres_` also separated these variables
from the `_database_` variables that ended up in bridge configuration.

However, @jdreichmann makes a good point
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/740#discussion_r542281102)
that we don't need to be so specific and can allow for other engines (like MySQL) to use these variables.
2020-12-14 13:02:47 +02:00
ce21ea3640 Add (Postgres + SQLite) support to matrix-mautrix-hangouts bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:34:59 +02:00
43d6ff2af8 Fix sqlite usage for mautrix-facebook/mautrix-telegram
Regression since 2d99ade72f and 9bf8ce878e, respectively.

When SQLite is to be used, these bridges expect an `sqlite://`
connection string, and not a plain file name (path), like Appservice
Discord and mautrix-whatsapp do.
2020-12-14 12:30:10 +02:00
6c77eae969 Add (Postgres + SQLite) support to matrix-mautrix-whatsapp bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:24:37 +02:00
9bf8ce878e Add (Postgres + SQLite) support to matrix-mautrix-telegram bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:06:28 +02:00
a3406a182b Move some things around 2020-12-14 12:04:47 +02:00
2d99ade72f Add (Postgres + SQLite) support to matrix-mautrix-facebook bridge 2020-12-14 11:50:42 +02:00
5dba0c038b Make --tags=import-generic-sqlite-db commands not pass a sensitive connection string around
Instead of passing the connection string, we can now pass a name of a
variable, which contains a connection string.

Both are supported for having extra flexibility.
2020-12-14 11:47:00 +02:00
d91aa5a060 Do not introduce sub-variables exposing implementation details 2020-12-14 10:52:07 +02:00
f1e85f7112 Don't mention Postgres roles, just say users 2020-12-14 10:04:37 +02:00
4617984b9f Add (SQLite -> Postgres) migration instructions 2020-12-14 02:24:32 +02:00
cb969c6ca2 Add --tags=import-generic-sqlite-db (pgloader import)
This can be used by various bridges, etc., to import an SQLite
(or some other supported) database into Postgres.
2020-12-14 02:23:29 +02:00
c66c084027 Merge branch 'master' into postgres-per-default 2020-12-14 01:51:15 +02:00
6e1dfb62f0 Rename some doc files and commands related to importing
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.

Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.

We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
2020-12-14 01:51:00 +02:00
b87b754372 Fail if appservice-discord wants Postgres, but has leftover SQLite data 2020-12-14 01:36:15 +02:00
183d2a10db Ensure matrix-postgres.service is started before creating additional users/databases 2020-12-14 00:59:59 +02:00
a374d309c8 Make appservice-discord support both SQLite and Postgres
People can toggle between them now. The playbook also defaults
to using SQLite if an external Postgres server is used.

Ideally, we'd be able to create databases/users in external Postgres
servers as well, but our initialization logic (and `docker run` command,
etc.) hardcode too many things right now.
2020-12-14 00:52:25 +02:00
46a4034d3e Use "password" for additional Postgres databases, not "pass"
Being more explicit sounds better.
2020-12-14 00:43:03 +02:00
3a037a5993 Ensure additional databases contain all the keys that we expect 2020-12-14 00:39:38 +02:00
da4cb2f639 Do not use the postgresql_user/postgresql_db modules
While these modules are really nice and helpful, we can't use them
for at least 2 reasons:

- for us, Postgres runs in a container on a private Docker network
(`--network=matrix`) without usually being exposed to the host.
These modules execute on the host so they won't be able to reach it.

- these modules require `psycopg2`, so we need to install it before
using it. This might or might not be its own can of worms.
2020-12-14 00:31:38 +02:00
bbc09d013b Do not execute additional databases creation code if not necessary
The tasks in `create_additional_databases.yml` will likely
ensure `matrix-postgres.service` is started, etc.

If no additional databases are defined, we'd rather not execute that
file and all these tasks that it may do in the future.
2020-12-13 23:46:05 +02:00
c765ceb270 Prevent weird loop error
> Invalid data passed to 'loop', it requires a list, got this instead: matrix_postgres_additional_databases. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup.

Well, or working around it, as I've done in this commit (which seems
more sane than `wantlist=True` stuff).
2020-12-13 22:56:56 +02:00
e2952f16f7 Determine matrix-postgres IP address without relying on jq
To avoid needing to have `jq` installed on the machine, we could:
- try to run jq in a Docker container using some small image providing
that
- better yet, avoid `jq` altogether
2020-12-13 22:45:48 +02:00
f47e8a97e6 Make use of matrix_host_command_docker instead of hardcoding 2020-12-13 22:38:35 +02:00
0641106370 Allow username of additional Postgres databases to be different
We'll most likely use one that matches the database name, but
it's better to have it configurable.
2020-12-13 22:37:04 +02:00
527d5f57d5 Relocate Postgres additional database creation logic
Moving it above the "uninstalling" set of tasks is better.
Extracting it out to another file at the same time, for readability,
especially given that it will probably have to become more complex in
the future (potentially installing `jq`, etc.)
2020-12-13 22:37:04 +02:00
dac0d3a682 Add default matrix_postgres_additional_databases 2020-12-13 21:07:16 +02:00