finallycoffee/matrix-docker-ansible-deploy
5
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity

add prometheus-nginxlog-exporter role (#2315)

Browse Source 
* add prometheus-nginxlog-exporter role

* Rename matrix_prometheus_nginxlog_exporter_container_url to matrix_prometheus_nginxlog_exporter_container_hostname

* avoid referencing variables from other roles, handover info using group_vars/matrix_servers

* fix: stop service when uninstalling

fix: typo

move available arch's into a var

fix: text

* fix: prometheus enabled condition

Co-authored-by: ikkemaniac <ikkemaniac@localhost>
This commit is contained in:
ikkemaniac
2022-12-07 15:58:36 +01:00
committed by GitHub
parent 48d601008c
commit 8ebf18a885
18 changed files with 2139 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/configuring-playbook-prometheus-grafana.md
View File

@ -15,6 +15,9 @@ matrix_prometheus_node_exporter_enabled: true
# You can remove this, if unnecessary.
matrix_prometheus_postgres_exporter_enabled: true
# You can remove this, if unnecessary.
matrix_prometheus_nginxlog_exporter_enabled: true
matrix_grafana_enabled: true
matrix_grafana_anonymous_access: false
@ -39,6 +42,7 @@ Name | Description
`matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
`matrix_prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
`matrix_prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus.
`matrix_prometheus_nginxlog_exporter_enabled`|[NGINX Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose NGINX logs to Prometheus.
`matrix_grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.<your-domain>` subdomain) the dashboards with the graphs that we're interested in
`matrix_grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
`matrix_grafana_default_admin_user`<br>`matrix_grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
@ -73,6 +77,7 @@ Name | Description
`matrix_prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network)
`matrix_prometheus_node_exporter_metrics_proxying_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.DOMAIN/metrics/node-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network)
`matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [NGINX Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network)
`matrix_prometheus_postgres_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.DOMAIN/metrics/postgres-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network)
`matrix_bridge_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.DOMAIN/metrics/hookshot` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)

59
docs/configuring-playbook-prometheus-nginxlog.md Normal file
View File

@ -0,0 +1,59 @@
# Enabling metrics and graphs for NginX logs (optional)
It can be useful to have some (visual) insight into NignX logs.
This adds [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) to your matrix deployment.
It will provide a prometheus 'metrics' endpoint exposing data from both the `matrix-nginx-proxy` and `matrix-synapse-reverse-proxy-companion` logs and automatically aggregates the data with prometheus.
Optionally it visualizes the data, if [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) is enabled, by means of a dedicated Grafana dashboard named `NGINX PROXY`
You can enable this role by adding the following settings in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_prometheus_nginxlog_exporter_enabled: true
# required depency
matrix_prometheus_enabled: true
# optional for visualization
matrix_grafana_enabled: true
```
x | Prerequisites | Variable | Description
|:--:|:--:|:--:|:--|
**REQUIRED** | `matrix-prometheus`| `matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
_Optional_ | [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) | [`matrix_grafana_enabled`](configuring-playbook-prometheus-grafana.md)|[Grafana](https://grafana.com) is the visual component. It shows (on the `stats.<your-domain>` subdomain) graphs that we're interested in. When enabled the `NGINX PROXY` dashboard is automatically added.
## Docker Image Compatibility
At the moment of writing only images for `amd64` and `arm64` architectures are available
The playbook currently does not support building an image.
You can however use a custom-build image by setting
```yaml
matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false
matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag
```
## Security and privacy
Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access.
Please make sure you change the default Grafana password.
## Save metrics on an external Prometheus server
The playbook will automatically integrate the metrics into the Prometheus server provided with this playbook. You can choose to save data on an external Prometheus instance.
The metrics of this role will be exposed on `https://matrix.DOMAIN/metrics/nginxlog` when setting
```yaml
matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true
# required dependency
matrix_nginx_proxy_proxy_matrix_metrics_enabled: true
```
The playbook can provide a single endpoint (`https://matrix.DOMAIN/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, etc). To enable this `/metrics/*` feature, use `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled`.
The following variables may be of interest:
Name | Description
-----|----------
`matrix_nginx_proxy_proxy_matrix_metrics_enabled`|Set this to `true` to enable metrics exposure for various services on `https://matrix.DOMAIN/metrics/*`. Refer to the individual `matrix_SERVICE_metrics_proxying_enabled` variables below for exposing metrics for each individual service.