Run Synapse workers in their own containers

This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.

We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
This commit is contained in:
Slavi Pantaleev
2021-01-25 12:14:46 +02:00
parent da50fb27a0
commit 70796703d3
12 changed files with 160 additions and 190 deletions

View File

@ -1,30 +0,0 @@
#!/bin/bash
# Find a synapse worker's PID and write it to a file so systemd can manage it as a service
# example invocation:
# matrix-synapse-worker-write-pid user_dir:18700 /run/matrix-synapse-worker.user_dir:18700.pid
docker_api_call() { curl --silent --unix-socket /var/run/docker.sock ${@}; }
TARGETCONTAINER=matrix-synapse
TARGETWORKER=${1}
PIDFILE=${2}
# get ID list of subprocesses executed in $TARGETCONTAINER, and for each..
for EXECID in $(docker_api_call http://localhost/containers/${TARGETCONTAINER}/json | jq --raw-output '.ExecIDs[]')
do
# fetch detailed process info
EXECINFO=$(docker_api_call http://localhost/exec/${EXECID}/json)
# extract config file path from last command argument
WORKERCONFIGFILE=$(echo ${EXECINFO} | jq --raw-output .ProcessConfig.arguments[-1])
# reconstruct worker name
WORKERNAME=${WORKERCONFIGFILE#*/worker.}
WORKERNAME=${WORKERNAME%.yaml}
# if name matches the target worker: write out most recent PID & quit
[ "${WORKERNAME}" = "${TARGETWORKER}" ] \
&& echo ${EXECINFO} | jq --raw-output .Pid > ${PIDFILE} \
&& exit 0
done