Commit Graph

7604 Commits

Author SHA1 Message Date
86a8091768 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-telegram 2020-12-14 16:19:54 +02:00
3ba8520266 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-hangouts 2020-12-14 16:18:38 +02:00
bbc08722c5 Enable automatic (SQLite -> Postgres) migration for matrix-mautrix-facebook 2020-12-14 16:14:23 +02:00
c1431b28f0 Make use of matrix_postgres_db_migration_request.caller 2020-12-14 16:13:57 +02:00
ac37091d01 Enable automatic (SQLite -> Postgres) migration for matrix-reminder-bot 2020-12-14 16:03:40 +02:00
dc7850e83c Fix wording and variable names a bit 2020-12-14 16:03:40 +02:00
bc376c2fb2 Add database migration utility to matrix-postgres role 2020-12-14 16:03:40 +02:00
54da61f81b add postgres support mx-appservice-[slack|irc] with fallback to nedb in role and migration notice 2020-12-14 14:08:35 +01:00
e3a0c9adda Add (Postgres + SQLite) support to matrix-reminder-bot
This has been tested and appears to work.
2020-12-14 15:02:11 +02:00
dde1c9f899 Fix indentation causing YAML syntax error 2020-12-14 14:53:35 +02:00
aa828ff9f6 Separate matrix-reminder-bot install/uninstall tasks 2020-12-14 14:50:04 +02:00
b9a04a7f95 Rename some remaining matrix_*_postgres_* vars back to matrix_*_database_*
Looks like there are some that I missed in 087dbe4ddc
2020-12-14 14:42:18 +02:00
5d70bc1376 add postgres support for mx-puppet-* with fallback to sqlite in role and migration notice 2020-12-14 13:22:58 +01:00
087dbe4ddc Rename matrix_*_postgres_* back to matrix_*_database_*
I was thinking that it makes sense to be more specific,
and using `_postgres_` also separated these variables
from the `_database_` variables that ended up in bridge configuration.

However, @jdreichmann makes a good point
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/740#discussion_r542281102)
that we don't need to be so specific and can allow for other engines (like MySQL) to use these variables.
2020-12-14 13:02:47 +02:00
ce21ea3640 Add (Postgres + SQLite) support to matrix-mautrix-hangouts bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:34:59 +02:00
43d6ff2af8 Fix sqlite usage for mautrix-facebook/mautrix-telegram
Regression since 2d99ade72f and 9bf8ce878e, respectively.

When SQLite is to be used, these bridges expect an `sqlite://`
connection string, and not a plain file name (path), like Appservice
Discord and mautrix-whatsapp do.
2020-12-14 12:30:10 +02:00
6c77eae969 Add (Postgres + SQLite) support to matrix-mautrix-whatsapp bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:24:37 +02:00
2848322461 Remove some remaining _database_password usage in favor of _postgres_pasword
The only one that remains is `matrix_synapse_database_password`, but
that's something old and should be dealt with separately in the future
(unless it remains as it is).
2020-12-14 12:19:35 +02:00
6a2dd8088c Fix typo (matrix_telegram_facebook_postgres_hostname -> matrix_mautrix_telegram_postgres_hostname)
Related to 9bf8ce878e.
2020-12-14 12:13:10 +02:00
05dd091133 Rename _database_hostname to _postgres_hostname
Should have been done during 2a99e84b5b, but I've missed it.

Because of this, 9bf8ce878e was also incomplete and would have caused
an error.
2020-12-14 12:11:41 +02:00
9bf8ce878e Add (Postgres + SQLite) support to matrix-mautrix-telegram bridge
I don't use this bridge, so this is completely untested.
2020-12-14 12:06:28 +02:00
a3406a182b Move some things around 2020-12-14 12:04:47 +02:00
2d99ade72f Add (Postgres + SQLite) support to matrix-mautrix-facebook bridge 2020-12-14 11:50:42 +02:00
5dba0c038b Make --tags=import-generic-sqlite-db commands not pass a sensitive connection string around
Instead of passing the connection string, we can now pass a name of a
variable, which contains a connection string.

Both are supported for having extra flexibility.
2020-12-14 11:47:00 +02:00
2a99e84b5b Rename Postgres-related variables (matrix*database_* -> matrix*postgres_*)
`matrix_SERVICE_database_engine` remains as it is - the only one with a
`_database_` infix. Postgres-related ones move to `_postgres_`.
2020-12-14 11:19:18 +02:00
c0edacbefd Fix matrix_postgres_additional_databases syntax problems 2020-12-14 11:17:12 +02:00
d91aa5a060 Do not introduce sub-variables exposing implementation details 2020-12-14 10:52:07 +02:00
f1e85f7112 Don't mention Postgres roles, just say users 2020-12-14 10:04:37 +02:00
b217a2ed69 postgres: set default in playbook to postgres but fallback to sqlite 2020-12-14 07:54:24 +01:00
4617984b9f Add (SQLite -> Postgres) migration instructions 2020-12-14 02:24:32 +02:00
cb969c6ca2 Add --tags=import-generic-sqlite-db (pgloader import)
This can be used by various bridges, etc., to import an SQLite
(or some other supported) database into Postgres.
2020-12-14 02:23:29 +02:00
c66c084027 Merge branch 'master' into postgres-per-default 2020-12-14 01:51:15 +02:00
6e1dfb62f0 Rename some doc files and commands related to importing
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.

Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.

We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
2020-12-14 01:51:00 +02:00
b87b754372 Fail if appservice-discord wants Postgres, but has leftover SQLite data 2020-12-14 01:36:15 +02:00
dd994995bc Simplify password for additional Postgres databases
Using the result of `password_hash` works for creating them,
but authentication seems to be failing with some tools like pgloader.

It's possible that we're not escaping things properly somewhere.
Ideally, it'd be nice to solve that. But the easier (and still
relatively safe/good) solution is to just turn that password hash
into a UUID that's safe for passing around without worrying about
escaping.
2020-12-14 01:22:23 +02:00
183d2a10db Ensure matrix-postgres.service is started before creating additional users/databases 2020-12-14 00:59:59 +02:00
a374d309c8 Make appservice-discord support both SQLite and Postgres
People can toggle between them now. The playbook also defaults
to using SQLite if an external Postgres server is used.

Ideally, we'd be able to create databases/users in external Postgres
servers as well, but our initialization logic (and `docker run` command,
etc.) hardcode too many things right now.
2020-12-14 00:52:25 +02:00
46a4034d3e Use "password" for additional Postgres databases, not "pass"
Being more explicit sounds better.
2020-12-14 00:43:03 +02:00
3a037a5993 Ensure additional databases contain all the keys that we expect 2020-12-14 00:39:38 +02:00
da4cb2f639 Do not use the postgresql_user/postgresql_db modules
While these modules are really nice and helpful, we can't use them
for at least 2 reasons:

- for us, Postgres runs in a container on a private Docker network
(`--network=matrix`) without usually being exposed to the host.
These modules execute on the host so they won't be able to reach it.

- these modules require `psycopg2`, so we need to install it before
using it. This might or might not be its own can of worms.
2020-12-14 00:31:38 +02:00
bbc09d013b Do not execute additional databases creation code if not necessary
The tasks in `create_additional_databases.yml` will likely
ensure `matrix-postgres.service` is started, etc.

If no additional databases are defined, we'd rather not execute that
file and all these tasks that it may do in the future.
2020-12-13 23:46:05 +02:00
c765ceb270 Prevent weird loop error
> Invalid data passed to 'loop', it requires a list, got this instead: matrix_postgres_additional_databases. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup.

Well, or working around it, as I've done in this commit (which seems
more sane than `wantlist=True` stuff).
2020-12-13 22:56:56 +02:00
e2952f16f7 Determine matrix-postgres IP address without relying on jq
To avoid needing to have `jq` installed on the machine, we could:
- try to run jq in a Docker container using some small image providing
that
- better yet, avoid `jq` altogether
2020-12-13 22:45:48 +02:00
f47e8a97e6 Make use of matrix_host_command_docker instead of hardcoding 2020-12-13 22:38:35 +02:00
d251764c16 Fix syntax issues in matrix_postgres_additional_databases
Quotes are necessary around dictionary field names.

There was a missing `}` as well.
2020-12-13 22:37:04 +02:00
0641106370 Allow username of additional Postgres databases to be different
We'll most likely use one that matches the database name, but
it's better to have it configurable.
2020-12-13 22:37:04 +02:00
527d5f57d5 Relocate Postgres additional database creation logic
Moving it above the "uninstalling" set of tasks is better.
Extracting it out to another file at the same time, for readability,
especially given that it will probably have to become more complex in
the future (potentially installing `jq`, etc.)
2020-12-13 22:37:04 +02:00
dac0d3a682 Add default matrix_postgres_additional_databases 2020-12-13 21:07:16 +02:00
77a5c7cf3c Merge branch 'master' into postgres-per-default 2020-12-13 21:04:15 +02:00
9dc023c558 Merge pull request #750 from chagai95/patch-1
Adding jitsi as well
2020-12-12 01:44:50 +02:00