From f8ad4384fac19c5a9ff3fb1b4a2d87e32b0fd4c8 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sat, 28 Dec 2024 08:48:59 -0500 Subject: [PATCH 1/6] Update docs/configuring-playbook-prometheus-nginxlog.md: add the common section "Extending the configuration" Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-nginxlog.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 224061c07..af8cf3306 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -20,6 +20,14 @@ Add the following configuration to your `inventory/host_vars/matrix.example.com/ matrix_prometheus_nginxlog_exporter_enabled: true ``` +### Extending the configuration + +There are some additional things you may wish to configure about the component. + +Take a look at: + +- `roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml` for some variables that you can customize via your `vars.yml` file + ## Installing After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below: From 421f13f6c531572549bf35eed82ec8875605e9b9 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Thu, 30 Jan 2025 13:40:56 +0900 Subject: [PATCH 2/6] Update docs/configuring-playbook-prometheus-nginxlog.md: add the common section "Troubleshooting" Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-nginxlog.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index af8cf3306..74e10be30 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -65,3 +65,7 @@ You can either use `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled Whichever way you go with, this service will expose its metrics endpoint **without password-protection** at `https://matrix.example.com/metrics/nginxlog` by default. For password-protection, use (`matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users`) or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). + +## Troubleshooting + +As with all other services, you can find the logs in [systemd-journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html) by logging in to the server with SSH and running `journalctl -fu matrix-prometheus-nginxlog-exporter`. From ac926fb6b78522ad3a3306a4fced9bb9bbaf0d94 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Thu, 30 Jan 2025 13:48:47 +0900 Subject: [PATCH 3/6] Update docs/configuring-playbook-prometheus-nginxlog.md: adopt the common format for the introduction Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-nginxlog.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 74e10be30..4b68ad3bc 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -1,13 +1,15 @@ # Enabling metrics and graphs for NginX logs (optional) -It can be useful to have some (visual) insight into [nginx](https://nginx.org/) logs. - -This adds [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) to your Matrix deployment. +The playbook can install and configure the [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) service for you, in order to make it possible to have some (visual) insight into [nginx](https://nginx.org/) logs. It will collect access logs from various nginx reverse-proxies which may be used internally (e.g. `matrix-synapse-reverse-proxy-companion`, if Synapse workers are enabled) and will make them available at a Prometheus-compatible `/metrics` endpoint. +See the project's [documentation](https://github.com/martin-helmich/prometheus-nginxlog-exporter/blob/master/README.adoc) to learn what it does and why it might be useful to you. + **Note**: nginx is only used internally by this Ansible playbook. With Traefik being our default reverse-proxy, collecting nginx metrics is less relevant. +## Prerequisite + To make use of this, you need to install [Prometheus](./configuring-playbook-prometheus-grafana.md) either via the playbook or externally. When using an external Prometheus, configuration adjustments are necessary — see [Save metrics on an external Prometheus server](#save-metrics-on-an-external-prometheus-server). If your setup includes [Grafana](./configuring-playbook-prometheus-grafana.md), a dedicated `NGINX PROXY` Grafana dashboard will be created. From 57bc9f46cc3d01341fe6b97d4721e7b33bbdb733 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Thu, 30 Jan 2025 13:50:40 +0900 Subject: [PATCH 4/6] Update docs/configuring-playbook-prometheus-nginxlog.md: move sections related to configuration to the dedicated one Signed-off-by: Suguru Hirahara --- ...onfiguring-playbook-prometheus-nginxlog.md | 50 +++++++++---------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 4b68ad3bc..9218a0553 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -22,6 +22,31 @@ Add the following configuration to your `inventory/host_vars/matrix.example.com/ matrix_prometheus_nginxlog_exporter_enabled: true ``` +### Docker Image Compatibility (optional) + +At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: + +```yaml +matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false +matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag +``` + +## Security and privacy + +Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. Please make sure you change the default Grafana password. + +### Save metrics on an external Prometheus server (optional) + +The playbook will automatically integrate the metrics into the [Prometheus](./configuring-playbook-prometheus-grafana.md) server provided with this playbook (if enabled). In such cases, the metrics endpoint is not exposed publicly — it's only available on the container network. + +When using an external Prometheus server, you'll need to expose metrics publicly. See [Collecting metrics to an external Prometheus server](./configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). + +You can either use `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true` to expose just this one service, or `matrix_metrics_exposure_enabled: true` to expose all services. + +Whichever way you go with, this service will expose its metrics endpoint **without password-protection** at `https://matrix.example.com/metrics/nginxlog` by default. + +For password-protection, use (`matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users`) or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). + ### Extending the configuration There are some additional things you may wish to configure about the component. @@ -43,31 +68,6 @@ The shortcut commands with the [`just` program](just.md) are also available: `ju `just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too. -## Docker Image Compatibility - -At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: - -```yaml -matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false -matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag -``` - -## Security and privacy - -Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. Please make sure you change the default Grafana password. - -## Save metrics on an external Prometheus server - -The playbook will automatically integrate the metrics into the [Prometheus](./configuring-playbook-prometheus-grafana.md) server provided with this playbook (if enabled). In such cases, the metrics endpoint is not exposed publicly — it's only available on the container network. - -When using an external Prometheus server, you'll need to expose metrics publicly. See [Collecting metrics to an external Prometheus server](./configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). - -You can either use `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true` to expose just this one service, or `matrix_metrics_exposure_enabled: true` to expose all services. - -Whichever way you go with, this service will expose its metrics endpoint **without password-protection** at `https://matrix.example.com/metrics/nginxlog` by default. - -For password-protection, use (`matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users`) or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). - ## Troubleshooting As with all other services, you can find the logs in [systemd-journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html) by logging in to the server with SSH and running `journalctl -fu matrix-prometheus-nginxlog-exporter`. From a6364cc6a9f320838aeedb8f0488a905dc82e9f6 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Thu, 30 Jan 2025 14:05:12 +0900 Subject: [PATCH 5/6] Update docs/configuring-playbook-prometheus-nginxlog.md: change the section "Security and privacy" into the warning blockquote Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-nginxlog.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 9218a0553..eb46f1b54 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -31,12 +31,11 @@ matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag ``` -## Security and privacy - -Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. Please make sure you change the default Grafana password. - ### Save metrics on an external Prometheus server (optional) +> [!WARNING] +> Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. Please make sure you change the default Grafana password. + The playbook will automatically integrate the metrics into the [Prometheus](./configuring-playbook-prometheus-grafana.md) server provided with this playbook (if enabled). In such cases, the metrics endpoint is not exposed publicly — it's only available on the container network. When using an external Prometheus server, you'll need to expose metrics publicly. See [Collecting metrics to an external Prometheus server](./configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). From 7f90dda4fcec32bf2a0e798e60684b2a187e6eea Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Thu, 30 Jan 2025 14:12:19 +0900 Subject: [PATCH 6/6] Update docs/configuring-playbook-prometheus-nginxlog.md: move down the section "Docker Image Compatibility" The section seems to be less relevant than the one for saving metrics on a Prometheus server. Signed-off-by: Suguru Hirahara --- ...configuring-playbook-prometheus-nginxlog.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index eb46f1b54..5d04d905b 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -22,15 +22,6 @@ Add the following configuration to your `inventory/host_vars/matrix.example.com/ matrix_prometheus_nginxlog_exporter_enabled: true ``` -### Docker Image Compatibility (optional) - -At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: - -```yaml -matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false -matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag -``` - ### Save metrics on an external Prometheus server (optional) > [!WARNING] @@ -46,6 +37,15 @@ Whichever way you go with, this service will expose its metrics endpoint **witho For password-protection, use (`matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users`) or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). +### Docker Image Compatibility (optional) + +At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: + +```yaml +matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false +matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag +``` + ### Extending the configuration There are some additional things you may wish to configure about the component.