app/vmauth: add real_ip_header to ip_filters
Some checks are pending
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
CodeQL JS/TS / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
main / integration-test (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run

This commit allows vmauth to obtain client IP address from HTTP Headers.
Main scenario for it is vmauth located behind reverse-proxy.

 It adds both global and per user configuration settings: -httpRealIPHeader and `real_ip_header` config option.

vmauth try to obtain IP from header if this setting is set. If header is not exists, vmauth fallbacks to `remoteAddress`.

Commit also updates incorrect benchmarks and align test package naming for ip_filters

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

Signed-off-by: f41gh7 <nik@victoriametrics.com>
This commit is contained in:
Nikolay 2024-11-28 14:53:21 +01:00 committed by f41gh7
parent dec9a2f023
commit 92512cbe54
No known key found for this signature in database
GPG Key ID: 4558311CF775EC72
2 changed files with 35 additions and 1 deletions

View File

@ -25,9 +25,9 @@ See also [LTS releases](https://docs.victoriametrics.com/lts-releases/).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add support for template alias in predefined panels. This allows creating more readable metric names in the legend using constructions like `{{label_name}}`, where `label_name` is the name of the label. [See this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/116101da78a4dee8bd7c4ba0e66458fd05a10469#diff-95141489b32468cf852d2705d96eaa48c50a8b1cdd0424a29e7ca289912a6dcbR140-R151) * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add support for template alias in predefined panels. This allows creating more readable metric names in the legend using constructions like `{{label_name}}`, where `label_name` is the name of the label. [See this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/116101da78a4dee8bd7c4ba0e66458fd05a10469#diff-95141489b32468cf852d2705d96eaa48c50a8b1cdd0424a29e7ca289912a6dcbR140-R151)
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): add `ignore_first_sample_interval` param to [aggregation config](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config). It allows users to control the time interval when aggregation skips sending aggregated samples to avoid unexpected spikes in values. By default, this interval is set to x2 of `staleness_interval`. The new setting is applicable only to `total`, `total_prometheus`, `increase`, `increase_prometheus` and `histogram_bucket` outputs. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7116) for details. Thanks to @iyuroch for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7313). * FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): add `ignore_first_sample_interval` param to [aggregation config](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config). It allows users to control the time interval when aggregation skips sending aggregated samples to avoid unexpected spikes in values. By default, this interval is set to x2 of `staleness_interval`. The new setting is applicable only to `total`, `total_prometheus`, `increase`, `increase_prometheus` and `histogram_bucket` outputs. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7116) for details. Thanks to @iyuroch for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7313).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add `dump_request_on_errors` bool setting to [auth config](https://docs.victoriametrics.com/vmauth/#auth-config) for debugging HTTP requests that missed routing rules. This should improve debugability of vmauth settings. * FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add `dump_request_on_errors` bool setting to [auth config](https://docs.victoriametrics.com/vmauth/#auth-config) for debugging HTTP requests that missed routing rules. This should improve debugability of vmauth settings.
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add `real_ip_header` setting to [ip_filters](https://docs.victoriametrics.com/vmauth/#ip-filters) and corresponding global flag `httpRealIPHeader`. It allows `vmauth` obtain client IP from HTTP headers for filtering. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883) for details.
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add `dryRun` flag to validate configuration. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7505) for details. * FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add `dryRun` flag to validate configuration. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7505) for details.
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): Properly return `200 OK` HTTP status code when importing data via [Pushgateway protocol](https://docs.victoriametrics.com/#how-to-import-data-in-prometheus-exposition-format) using [multitenant URL format](https://docs.victoriametrics.com/cluster-victoriametrics/#url-format). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636) and [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7571). * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): Properly return `200 OK` HTTP status code when importing data via [Pushgateway protocol](https://docs.victoriametrics.com/#how-to-import-data-in-prometheus-exposition-format) using [multitenant URL format](https://docs.victoriametrics.com/cluster-victoriametrics/#url-format). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636) and [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7571).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): Properly set `TCP` connection timeout for `Kubernetes API server` connection for metric scrapping with `kubernetes_sd_configs`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7127). * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): Properly set `TCP` connection timeout for `Kubernetes API server` connection for metric scrapping with `kubernetes_sd_configs`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7127).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): fix the `resource_group` filter for Azure service discovery on virtual machine scale sets. Previously, this filter did not apply to virtual machine scale sets, causing all virtual machines to be discovered. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7630). * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent): fix the `resource_group` filter for Azure service discovery on virtual machine scale sets. Previously, this filter did not apply to virtual machine scale sets, causing all virtual machines to be discovered. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7630).

View File

@ -827,6 +827,38 @@ users:
allow_list: [127.0.0.1] allow_list: [127.0.0.1]
``` ```
By default, the client's TCP address is utilized for IP filtering. In scenarios where `vmauth` operates behind a reverse proxy, it is advisable to configure `vmauth` to retrieve the client IP address from an [HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For) (e.g., `X-Forwarded-For`) {{% available_from "#" %}} or via the [Proxy Protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) for TCP load balancers. This can be achieved using the global configuration flags:
* `-httpRealIPHeader=X-Forwarded-For` {{% available_from "#" %}}
* `-httpListenAddr.useProxyProtocol=true`
### Security Considerations
**HTTP headers are inherently untrustworthy.** It is strongly recommended to implement additional security measures, such as:
* Dropping `X-Forwarded-For` headers at the internet-facing reverse proxy (e.g., before traffic reaches `vmauth`).
* Do not use `-httpRealIPHeader` at internet-facing `vmauth`.
See additional recommendations at [link](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For#security_and_privacy_concerns)
### Per-User Configuration
The values of `httpRealIPHeader` {{% available_from "#" %}} can be changed on a per-user basis within the user-specific configuration.
```yaml
users:
- username: "foobar"
password: "***"
url_prefix: "http://localhost:8428"
ip_filters:
allow_list: [127.0.0.1]
real_ip_header: X-Forwarded-For
- username: "foobar"
password: "***"
url_prefix: "http://localhost:8428"
ip_filters:
allow_list: [127.0.0.1]
real_ip_header: CF-Connecting-IP
```
See config example of using IP filters [here](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmauth/example_config_ent.yml). See config example of using IP filters [here](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmauth/example_config_ent.yml).
## Reading auth tokens from other HTTP headers ## Reading auth tokens from other HTTP headers
@ -1253,6 +1285,8 @@ See the docs at https://docs.victoriametrics.com/vmauth/ .
Whether to use proxy protocol for connections accepted at the corresponding -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing Whether to use proxy protocol for connections accepted at the corresponding -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing
Supports array of values separated by comma or specified via multiple flags. Supports array of values separated by comma or specified via multiple flags.
Empty values are set to false. Empty values are set to false.
-httpRealIPHeader string
HTTP request header to use for obtaining IP address of client for applying 'ip_filters'. By default vmauth uses IP address of TCP the client. Useful if vmauth is behind reverse-proxy
-idleConnTimeout duration -idleConnTimeout duration
The timeout for HTTP keep-alive connections to backend services. It is recommended setting this value to values smaller than -http.idleConnTimeout set at backend services (default 50s) The timeout for HTTP keep-alive connections to backend services. It is recommended setting this value to values smaller than -http.idleConnTimeout set at backend services (default 50s)
-internStringCacheExpireDuration duration -internStringCacheExpireDuration duration