Commit Graph

1448 Commits

Author SHA1 Message Date
Slavi Pantaleev
95346f3117 Reorganize Postgres access (breaking change)
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.

We also no longer have a default Postgres superuser password,
which improves security.

The changelog explains more as to why this was done
and how to proceed from here.
2021-01-22 13:26:12 +02:00
throwawayay
a30ef0cc29
Update element-web (1.7.16 -> 1.7.17) 2021-01-20 08:35:07 -05:00
Slavi Pantaleev
024a23ed17 Upgrade mautrix-facebook to the new Postgres-only version
I had intentionally held it back in 39ea3496a4
until:
- it received more testing (there were a few bugs during the
migration, but now it seems OK)
- this migration guide was written
2021-01-20 10:12:51 +02:00
pushytoxin
d51ea25219 When validating LE certs, do not wait for a random time
While administering we will occasionally invoke this script interactively with the "non-interactive" switch still there, yet still sit at the desk waiting for 300 seconds for this timer to run out.

The systemd-timer already uses a 3h randomized delay for automatic renewals, which serves this purpose well.
2021-01-19 18:41:45 +01:00
Slavi Pantaleev
39ea3496a4 Downgrade/lock mautrix-facebook to pre-mobile times
The `mobile` branch got merged to `master`, which ends up becoming
`:latest`. It's a "rewrite" of the bridge's backend and only
supports a Postgres database.

We'd like to go back (well, forward) to `:latest`, but that will take
a little longer, because:
- we need to handle and document things for people still on SQLite
(especially those with external Postgres, who are likely on SQLite for
bridges)
- I'd rather test the new builds (and migration) a bit before
releasing it to others and possibly breaking their bridge

Brave ones who are already using the bridge with Postgres
can jump on `:latest` and report their experience.
2021-01-19 18:44:15 +02:00
Slavi Pantaleev
c9d96d8135 Fix mautrix-telegram paths creation bug 2021-01-19 09:15:34 +02:00
Slavi Pantaleev
56c54d5cc7 Upgrade matrix-corporal (2.0.1 -> 2.1.0) 2021-01-18 18:23:17 +02:00
Slavi Pantaleev
c1008fde44 Upgrade matrix-coturn (4.5.1.3 -> 4.5.2) 2021-01-18 00:41:47 +02:00
Slavi Pantaleev
cf06f84608 Upgrade matrix-corporal (2.0.0 -> 2.0.1) 2021-01-17 22:05:26 +02:00
Slavi Pantaleev
d95cbe38d7 Rename configuration setting 2021-01-17 18:29:26 +02:00
Slavi Pantaleev
28d86e3aaa Initial work on support for matrix-corporal v2 2021-01-16 23:47:14 +02:00
Slavi Pantaleev
8549926395 Attempt to fix mautrix-whatsapp DB migration user table conflict
Discussed in https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/791
2021-01-15 17:13:47 +02:00
Slavi Pantaleev
1692a28fe4 Work around annoying Docker warning about undefined $HOME
> WARNING: Error loading config file: .dockercfg: $HOME is not defined

.. which appeared in Docker 20.10.
2021-01-15 00:23:01 +02:00
Slavi Pantaleev
26f0bbfdef Fix self-building for matrix-ma1sd on non-version tag/branch
Building `master` or something like this was failing.
2021-01-14 23:57:38 +02:00
Slavi Pantaleev
9e936e45ad Use BuildKit for ma1sd Docker building
Newer versions (`master`) use things like `--platform=...`,
which are not supported unless we enable the new BuildKit building
backend.
2021-01-14 23:48:30 +02:00
Slavi Pantaleev
e1690722f7 Replace cronjobs with systemd timers
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/756

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/737

I feel like timers are somewhat more complicated and dirty (compared to
cronjobs), but they come with these benefits:

- log output goes to journald
- on newer systemd distros, you can see when the timer fired, when it
will fire, etc.
- we don't need to rely on cron (reducing our dependencies to just
systemd + Docker)

Cronjobs work well, but it's one more dependency that needs to be
installed. We were even asking people to install it manually
(in `docs/prerequisites.md`), which could have gone unnoticed.

Once in a while someone says "my SSL certificates didn't renew"
and it's likely because they forgot to install a cron daemon.

Switching to systemd timers means that installation is simpler
and more unified.
2021-01-14 23:35:50 +02:00
Slavi Pantaleev
05ca9357a8 Add .service suffix to systemd units list
We'll be adding `.timer` units later on, so it's good to be
more explicit.
2021-01-14 23:02:10 +02:00
Slavi Pantaleev
653d1d7924 Revert "Don't self-build ma1sd every time unless git sources changed"
This reverts commit 2a25b63bb6.

Looking at other roles, we trigger building regardless of this.
It's better to always trigger it, because it's less fragile.
If the build fails and we only trigger it on "git changes"
then we won't trigger it for a while. That's not good.

Triggering it each and every time may seem like a waste,
but it supposedly runs quickly due to Docker caching.
2021-01-14 22:20:51 +02:00
Slavi Pantaleev
6f5aaad48d Split install/uninstall tasks in matrix-coturn 2021-01-14 22:11:38 +02:00
Slavi Pantaleev
57ea43d8b0 Remove unused variable
This variable has been useless since 2019-01-08.
We probably don't need to check for its usage anymore,
given how much time has passed since then, but ..
2021-01-14 17:47:13 +02:00
Slavi Pantaleev
7a90eb6d4f Relocate some validation tasks 2021-01-14 17:00:46 +02:00
Slavi Pantaleev
67dc5237c5
Merge pull request #794 from drpaneas/appservice_slack_rebuild
Selfbuild appservice-slack bridge
2021-01-14 10:47:31 +02:00
Slavi Pantaleev
862a6276a0
Do not pull appservice-slack when self-building 2021-01-14 10:47:23 +02:00
Slavi Pantaleev
b15da29ebb Bump Synapse to v1.25.0 for ARM 2021-01-14 10:41:47 +02:00
Panagiotis Georgiadis
a66a604e53
Selfbuild appservice-slack bridge 2021-01-14 01:29:11 +01:00
Slavi Pantaleev
2a25b63bb6 Don't self-build ma1sd every time unless git sources changed 2021-01-13 20:14:47 +02:00
Slavi Pantaleev
a5a44a9d3f
Merge pull request #786 from drpaneas/rebuild_telegram
Local rebuild for Telegram
2021-01-13 18:01:15 +02:00
Slavi Pantaleev
52fa7e576b
Fix path typo 2021-01-13 18:00:32 +02:00
Slavi Pantaleev
5fa30cdfcb
Ensure matrix_mautrix_facebook_docker_src_files_path created
Before we potentially clone to that path, we'd better make sure it exists.

We also simplify `when` statements a bit.
Given that we're in `setup_install.yml`, we know that the bridge is enabled,
so there's no need to check for that.
2021-01-13 17:59:46 +02:00
Slavi Pantaleev
568cb3d86f Upgrade matrix-mailer (4.93-r0 -> 4.93-r1)
This is a bit misleading, because the old Docker image
was tagged as `4.93.1`. There hasn't been a `4.93.1` version yet though.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/792
2021-01-13 17:37:31 +02:00
Slavi Pantaleev
24100342e1 Tell people that federation_ip_range_blacklist is gone
Related to d5945c6e78
2021-01-13 13:47:51 +02:00
Slavi Pantaleev
d5945c6e78 Upgrade Synapse (v1.24.0 -> v1.25.0) for amd64 2021-01-13 13:02:49 +02:00
Panagiotis Georgiadis
999fd2596f
Local rebuild for Telegram 2021-01-12 19:29:50 +01:00
Slavi Pantaleev
0b260a133f Add matrix-aux role to help with managing auxiliary files/directories 2021-01-11 22:32:52 +02:00
Will
5b0761bf40
Create list_tokens.yml 2021-01-09 08:52:02 -08:00
Will
1468010194
Update main.yml 2021-01-09 08:50:34 -08:00
Slavi Pantaleev
f7ae050eaf Remove useless quotes around ssl_ciphers value
Not sure if it breaks with them or not, but no other directive
uses quotes and the nginx docs show examples without quotes,
so we're being consistent with all of that.
2021-01-08 21:22:44 +02:00
Slavi Pantaleev
5822ba0c01 Use a more natural if statement 2021-01-08 21:21:33 +02:00
Slavi Pantaleev
de6ecd8818
Update inaccurate comments 2021-01-08 21:15:14 +02:00
Agustin Ferrario
5156c63a76 Clean up code
Code was clean up and simplified to make it simpler and easier to
maintain. No features were modified.
2021-01-08 18:35:27 +01:00
Agustin Ferrario
25d423e6b6 Fix errors per spantaleev suggestions
The different configurations are now all lower case, for consistent
naming.

`matrix_nginx_proxy_ssl_config` is now called
`matrix_nginx_proxy_ssl_preset`. The different options for "modern",
"intermediate" and "old" are stored in the main.yml file, instead of
being hardcoded in the configuration files. This will improve the
maintainability of the code.

The "custom" preset was removed. Now if one of the variables is set, it
will use it instead of the preset. This will allow to mix and match more
easily, for example using all the intermediate options but only
supporting TLSv1.2. This will also provide better backward
compatibility.
2021-01-08 11:32:10 +01:00
Agustin Ferrario
3cb71e7e84 Merge branch 'master' of https://github.com/spantaleev/matrix-docker-ansible-deploy 2021-01-03 13:18:21 +01:00
Slavi Pantaleev
6cce5383bc Fix Ansible 2.9.6 check
Fixup for https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/769
2021-01-03 08:55:30 +02:00
Slavi Pantaleev
2c09111a3a Actually enforce that we run on Ansible >= 2.7.1
Related to 6e652e10ad
2021-01-03 08:54:17 +02:00
Slavi Pantaleev
8710883064
Merge pull request #743 from pushytoxin/docker_network
Drop the old workaround for an Ansible bug that has been fixed three years ago
2021-01-03 08:49:09 +02:00
Slavi Pantaleev
cd2d2f594a
Merge pull request #686 from laszabine/signal
Added a role for the bridge mautrix-signal
2021-01-03 08:25:01 +02:00
Slavi Pantaleev
3b524ee815 Make mautrix-signal bridge not log to files
We try to only use console logging (going to journald) for everything,
instead of logging things twice (or more).
2021-01-03 08:20:43 +02:00
Slavi Pantaleev
274f23f668 Make matrix-mautrix-signal-daemon.service depend on docker.service 2021-01-03 08:16:49 +02:00
Slavi Pantaleev
da2a6682b3 Get rid of matrix_mautrix_signal_configuration_permissions
While it's kind of nice having it, it's also somewhat raw
and unnecessary.

Having a good default and not even mentioning it seems better
for most users.

People who need a more exposed bridge (rare) can use
override the default configuration using
`matrix_mautrix_signal_configuration_extension_yaml`.
2021-01-03 08:06:32 +02:00
Slavi Pantaleev
df8d9cfd34 Remove some TODOs
The answer to these is: it's good to have them in both places.
The role defines the obvious things it depends on (not knowing
what setup it will find itself into), and then
`group_vars/matrix_servers` "extends" it based on everything else it
knows (the homeserver being Synapse, whether or not the internal
Postgres server is being used, etc.)
2021-01-03 07:46:55 +02:00
Slavi Pantaleev
4805637181 Add support for custom ma1sd view sesion templates 2021-01-03 07:36:09 +02:00
Slavi Pantaleev
f84c69c164 Relocate custom ma1sd threepid email templates to config/
We used to store them in data/, but that seems inappropriate,
since it's just static configuration that the playbook can recreate.
2021-01-03 07:35:13 +02:00
Slavi Pantaleev
b5812b539b Rename ma1sd custom email template variable
Keeps up with a1f64f5159 (diff-0ccf69eb4d59a7645eb4d0a0b077e693948edb33ad06df043bba3fb30122879b)
2021-01-03 00:58:31 +02:00
Slavi Pantaleev
fb83eccf99 Relocate SQL template file 2021-01-03 00:58:31 +02:00
Sabine Laszakovits
84cac25c11 added config data_dir (else in ~, which isn't set) 2021-01-02 19:01:21 +01:00
Sabine Laszakovits
56af2b1a8c small fixes 2021-01-02 00:56:45 +01:00
Sabine Laszakovits
89f7f3c3b8 added log level configuration 2021-01-02 00:55:55 +01:00
Sabine Laszakovits
ffb837d4bc made the bridge use the default postgres db 2021-01-02 00:39:11 +01:00
Sabine Laszakovits
a06c58c753 Merge branch 'master' into signal 2021-01-01 21:05:00 +01:00
Slavi Pantaleev
1ed991e25c
Merge pull request #769 from aaronraimist/check-for-buggy-ansible
Check for buggy version of Ansible that Ubuntu 20.04 provides
2020-12-29 11:19:37 +02:00
Slavi Pantaleev
86da489b9b Never fail when stopping systemd service during (SQLite -> Postgres) migration
We need to suppress systemd service-stopping requests in certain rare
cases like https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/771

That issue seems to describe a case, where a migration from mxisd to
ma1sd was happening (DB files had just been moved), and then we were
attemping to stop `matrix-ma1sd.service` so we could import that database into
Postgres. However, there's neither `matrix-mxisd.service`, nor
`matrix-ma1sd.service` after `migrate_mxisd.yml` had just run, so
stopping `matrix-ma1sd.service` was failing.
2020-12-29 10:31:20 +02:00
Aaron Raimist
8827a49e21
Check equality properly 2020-12-26 20:20:00 -06:00
Aaron Raimist
3dd0517f04
Check for buggy version of Ansible that Ubuntu 20.04 provides 2020-12-26 20:13:49 -06:00
Slavi Pantaleev
a2a4218e95 Make mautrix-python-based bridges E2EE happier
Fixes a problem like this:
> File "/usr/lib/python3.8/site-packages/mautrix/bridge/e2ee.py", line 79, in __init__
> raise RuntimeError("Unsupported database scheme")

mautrix-python's e2ee.py module expects to find `postgres://` instead of
`postgresql://`.
2020-12-23 15:39:12 +02:00
Slavi Pantaleev
80c72615c7 Fixup all Dimension boolean fields after pgloader import
This is 8b6174786b done right. There were many more fields
that we had to account for.
2020-12-23 14:12:11 +02:00
Slavi Pantaleev
21662af3be Archive database only after additional_psql_statements_list had executed 2020-12-23 14:12:11 +02:00
Stuart Mumford
019a4d7dcd Use role relative paths for things 2020-12-23 11:34:48 +00:00
Slavi Pantaleev
be0c599565 Feed more slashes to mautrix bridges when using SQLite
This makes the `sqlite://` URI match what we were using before
and what the config expects.
2020-12-23 13:33:25 +02:00
Slavi Pantaleev
8b6174786b Fixup Dimension database schema a bit after pgloader import 2020-12-23 12:57:43 +02:00
Slavi Pantaleev
c5f8b1f61b Fix mautrix-whatsapp Postgres connection string to not use SSL by default 2020-12-23 11:40:22 +02:00
Slavi Pantaleev
f19b29846d
Merge pull request #740 from jdreichmann/postgres-per-default
postgres: create databases for all services
2020-12-23 11:00:41 +02:00
Slavi Pantaleev
ad1425eee4 Add pgloader self-building support (for ARM) 2020-12-23 09:08:54 +02:00
Slavi Pantaleev
8675dedbdb Add support for automatic (nedb -> Postgres) migration to matrix-appservice-slack 2020-12-22 19:56:52 +02:00
Slavi Pantaleev
9b95e1937c Auto-restart matrix-appservice-irc after (nedb -> Postgres) migration 2020-12-22 19:34:08 +02:00
Slavi Pantaleev
715bdf2c64 Add support for automatic (nedb -> Postgres) migration to mx-appservice-irc 2020-12-22 19:32:43 +02:00
Slavi Pantaleev
15f4cc924d Rename variables (_database_db_name -> _database_name) 2020-12-22 17:10:02 +02:00
Slavi Pantaleev
ab6563ce4e Add support for automatic (Postgres -> SQLite) migration to mx-puppet-twitter 2020-12-22 17:09:08 +02:00
Slavi Pantaleev
69cc2145d2 Add support for automatic (Postgres -> SQLite) migration to mx-puppet-steam 2020-12-22 16:51:59 +02:00
Slavi Pantaleev
262a25f997 Add support for automatic (Postgres -> SQLite) migration to mx-puppet-slack 2020-12-22 16:39:21 +02:00
Slavi Pantaleev
e49eb078a2 Add support for automatic (Postgres -> SQLite) migration to mx-puppet-skype 2020-12-22 16:29:47 +02:00
Dan Arnfield
c3b63c6c97 Update element-web (1.7.15 -> 1.7.16) 2020-12-22 08:29:37 -06:00
Dan Arnfield
10e0fa17ad Update nginx (1.19.5 -> 1.19.6) 2020-12-22 08:23:37 -06:00
Slavi Pantaleev
d135cd9cd3 Ensure mx-puppet-discord directories are created before attempting migration
Our old (base-path -> data-path) SQLite migration can't work otherwise.

It's probably not necessary to keep it anymore, but since we still do,
at least we should take care to ensure it works.
2020-12-22 13:44:36 +02:00
Slavi Pantaleev
44c9f4daca Add support for automatic (Postgres -> SQLite) migration to mx-puppet-instagram 2020-12-22 13:30:52 +02:00
Slavi Pantaleev
e64758c119 Add missing restart task
Should have been part of 149872e00c
2020-12-22 13:24:53 +02:00
Slavi Pantaleev
149872e00c Add support for automatic (Postgres -> SQLite) migration to mx-puppet-discord 2020-12-22 11:10:10 +02:00
Slavi Pantaleev
9b4bf73587 Fix undefined variable reference 2020-12-22 11:08:07 +02:00
Slavi Pantaleev
6488e11d69 Relocate some tasks 2020-12-22 10:52:36 +02:00
Slavi Pantaleev
ca066217d1
Merge pull request #757 from 0x46616c6b/disable-nginx-logging-option
add option to disable nginx access log
2020-12-21 22:30:25 +02:00
louis
dcd4716636 add option to disable nginx access log 2020-12-21 21:26:49 +01:00
Slavi Pantaleev
d0ee86e0a5 Fix matrix_corporal_docker_image_name_prefix referencing matrix_synapse_ stuff 2020-12-21 15:44:14 +02:00
Agustin Ferrario
a06feba281 Merge branch 'master' of https://github.com/spantaleev/matrix-docker-ansible-deploy 2020-12-18 10:22:43 +01:00
Slavi Pantaleev
8748f3d443 Move python{,3}-docker installation to another task
This also adds support for installing python3-docker (not python-docker)
in systems that run Python 3.
2020-12-17 11:49:56 +02:00
Slavi Pantaleev
349fbb6434 Do not hardcode armhf for Raspbian
Raspbian doesn't seem to support arm64, so this is somewhat pointless
right now.

However, they might in the future. Doing this should also unify us
some more with `setup_debian.yml` with the ultimate goal of
eliminating `setup_raspbian.yml`.
2020-12-17 11:47:34 +02:00
Slavi Pantaleev
a09ed58892 Ensure gnupg installed on Raspbian
It's likely installed by default, but it doesn't hurt to specify it.
It also makes us more the same with `setup_debian.yml`.
2020-12-17 11:45:32 +02:00
Slavi Pantaleev
f545de53f7 Do not hardcode "ubuntu" for the Docker APT key URL
Well, `ubuntu` or `debian`, the same key is served right now,
so it doesn't really matter.

This seems cleaner and less prone to breakage though.
2020-12-17 11:39:18 +02:00
Slavi Pantaleev
55f252a6ed Do not hardcode amd64 in setup_debian.yml
Until now, we've only supported non-amd64 on Raspbian.

Seems like there are now people running Debian/Ubuntu on ARM,
so we were forcing them into amd64 Docker packages.

I've gotten a report that this change fixes support
for Ubuntu Server 20.04 on RPi 4B.
2020-12-17 11:37:30 +02:00
Slavi Pantaleev
ed159cc742 Move matrix_architecture to matrix-base
We were only defining this in `group_vars/matrix_servers`, which is
inconsistent with how we normally do things.
2020-12-17 11:33:18 +02:00
Agustin Ferrario
2082242499 Add matrix_nginx_proxy_ssl_config
A new variable called `matrix_nginx_proxy_ssl_config` is created for
configuring how the nginx proxy configures SSL. Also a new configuration
validation option and other auxiliary variables are created.

A new variable configuration called `matrix_nginx_proxy_ssl_config` is
created. This allow to set the SSL configuration easily using the
default options proposed by Mozilla. The default configuration is set to
"Intermediate", removing the weak ciphers used in the old
configurations.

The new variable can also be set to "Custom" for a more granular control.
This allows to set another three variables called:

- `matrix_nginx_proxy_ssl_protocols`,
- `matrix_nginx_proxy_ssl_prefer_server_ciphers`
- `matrix_nginx_proxy_ssl_ciphers`

Also a new task is added to validate the SSL configuration variable.
2020-12-16 10:35:37 +01:00
Slavi Pantaleev
0f4649a45c Merge branch 'master' into postgres-per-default 2020-12-16 03:35:39 +02:00