mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-23 12:31:07 +01:00
lib/promrelabel: support action: graphite relabeling
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737
This commit is contained in:
Aliaksandr Valialkin
parent
45c1e27937
commit
c18f8cccfa
@ -478,6 +478,8 @@ The `/api/v1/export` endpoint should return the following response:
|
||||
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
|
||||
```
|
||||
|
||||
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
|
||||
|
||||
## Querying Graphite data
|
||||
|
||||
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
|
||||
@ -492,6 +494,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
|
||||
|
||||
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
|
||||
|
||||
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
|
||||
|
||||
|
||||
## How to send data from OpenTSDB-compatible agents
|
||||
|
||||
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
|
||||
@ -1132,7 +1137,9 @@ Example contents for `-relabelConfig` file:
|
||||
regex: true
|
||||
```
|
||||
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
|
||||
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
|
||||
|
||||
|
||||
## Federation
|
||||
|
||||
|
||||
@ -252,12 +252,13 @@ Labels can be added to metrics by the following mechanisms:
|
||||
VictoriaMetrics components (including `vmagent`) support Prometheus-compatible relabeling.
|
||||
They provide the following additional actions on top of actions from the [Prometheus relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config):
|
||||
|
||||
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`.
|
||||
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`.
|
||||
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal.
|
||||
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal.
|
||||
* `keep_metrics`: keeps all the metrics with names matching the given `regex`.
|
||||
* `drop_metrics`: drops all the metrics with names matching the given `regex`.
|
||||
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`
|
||||
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`
|
||||
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal
|
||||
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal
|
||||
* `keep_metrics`: keeps all the metrics with names matching the given `regex`
|
||||
* `drop_metrics`: drops all the metrics with names matching the given `regex`
|
||||
* `graphite`: applies Graphite-style relabeling to metric name. See [these docs](#graphite-relabeling)
|
||||
|
||||
The `regex` value can be split into multiple lines for improved readability and maintainability. These lines are automatically joined with `|` char when parsed. For example, the following configs are equivalent:
|
||||
|
||||
@ -305,6 +306,38 @@ You can read more about relabeling in the following articles:
|
||||
* [Extracting labels from legacy metric names](https://www.robustperception.io/extracting-labels-from-legacy-metric-names)
|
||||
* [relabel_configs vs metric_relabel_configs](https://www.robustperception.io/relabel_configs-vs-metric_relabel_configs)
|
||||
|
||||
## Graphite relabeling
|
||||
|
||||
VictoriaMetrics components support `action: graphite` relabeling rules, which allow extracting various parts from Graphite-style metrics
|
||||
into the configued labels with the syntax similar to [Glob matching in statsd_exporter](https://github.com/prometheus/statsd_exporter#glob-matching).
|
||||
Note that the `name` field must be substituted with explicit `__name__` option under `labels` section.
|
||||
If `__name__` option is missing under `labels` section, then the original Graphite-style metric name is left unchanged.
|
||||
|
||||
For example, the following relabeling rule generates `requests_total{job="app42",instance="host124:8080"}` metric
|
||||
from "app42.host123.requests.total" Graphite-style metric:
|
||||
|
||||
```yaml
|
||||
- action: graphite
|
||||
match: "*.*.*.total"
|
||||
labels:
|
||||
__name__: "${3}_total"
|
||||
job: "$1"
|
||||
instance: "${2}:8080"
|
||||
```
|
||||
|
||||
Important notes about `action: graphite` relabeling rules:
|
||||
|
||||
- The relabeling rule is applied only to metrics, which match the given `match` expression. Other metrics remain unchanged.
|
||||
- The `*` matches the maximum possible number of chars until the next dot or until the next part of the `match` expression whichever comes first.
|
||||
It may match zero chars if the next char is `.`.
|
||||
For example, `match: "app*foo.bar"` matches `app42foo.bar` and `42` becomes available to use at `labels` section via `$1` capture group.
|
||||
- The `$0` capture group matches the original metric name.
|
||||
- The relabeling rules are executed in order defined in the original config.
|
||||
|
||||
The `action: graphite` relabeling rules are easier to write and maintain than `action: replace` for labels extraction from Graphite-style metric names.
|
||||
Additionally, the `action: graphite` relabeling rules usually work much faster than the equivalent `action: replace` rules.
|
||||
|
||||
|
||||
## Prometheus staleness markers
|
||||
|
||||
`vmagent` sends [Prometheus staleness markers](https://www.robustperception.io/staleness-and-promql) to `-remoteWrite.url` in the following cases:
|
||||
|
||||
@ -34,6 +34,7 @@ The following tip changes can be tested by building VictoriaMetrics components f
|
||||
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): expose `/api/v1/status/config` endpoint in the same way as Prometheus does. See [these docs](https://prometheus.io/docs/prometheus/latest/querying/api/#config).
|
||||
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `-promscrape.suppressScrapeErrorsDelay` command-line flag, which can be used for delaying and aggregating the logging of per-target scrape errors. This may reduce the amounts of logs when `vmagent` scrapes many unreliable targets. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575). Thanks to @jelmd for [the initial implementation](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576).
|
||||
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `-promscrape.cluster.name` command-line flag, which allows proper data de-duplication when the same target is scraped from multiple [vmagent clusters](https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679).
|
||||
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `action: graphite` relabeling rules optimized for extracting labels from Graphite-style metric names. See [these docs](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737).
|
||||
* FEATURE: [VictoriaMetrics enterprise](https://victoriametrics.com/products/enterprise/): expose `vm_downsampling_partitions_scheduled` and `vm_downsampling_partitions_scheduled_size_bytes` metrics, which can be used for tracking the progress of initial [downsampling](https://docs.victoriametrics.com/#downsampling) for historical data. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2612).
|
||||
|
||||
* BUGFIX: support for data ingestion in [DataDog format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-datadog-agent) from legacy clients / agents. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2670). Thanks to @elProxy for the fix.
|
||||
|
||||
@ -268,7 +268,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
||||
VictoriaMetrics provides an ability to explore time series cardinality at `cardinality` tab in [vmui](#vmui) in the following ways:
|
||||
|
||||
- To identify metric names with the highest number of series.
|
||||
- To idnetify labels with the highest number of series.
|
||||
- To identify labels with the highest number of series.
|
||||
- To identify values with the highest number of series for the selected label (aka `focusLabel`).
|
||||
- To identify label=name pairs with the highest number of series.
|
||||
- To identify labels with the highest number of unique values.
|
||||
|
||||
@ -477,6 +478,8 @@ The `/api/v1/export` endpoint should return the following response:
|
||||
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
|
||||
```
|
||||
|
||||
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
|
||||
|
||||
## Querying Graphite data
|
||||
|
||||
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
|
||||
@ -491,6 +494,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
|
||||
|
||||
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
|
||||
|
||||
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
|
||||
|
||||
|
||||
## How to send data from OpenTSDB-compatible agents
|
||||
|
||||
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
|
||||
@ -1131,7 +1137,9 @@ Example contents for `-relabelConfig` file:
|
||||
regex: true
|
||||
```
|
||||
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
|
||||
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
|
||||
|
||||
|
||||
## Federation
|
||||
|
||||
@ -1441,6 +1449,7 @@ VictoriaMetrics returns TSDB stats at `/api/v1/status/tsdb` page in the way simi
|
||||
|
||||
* `topN=N` where `N` is the number of top entries to return in the response. By default top 10 entries are returned.
|
||||
* `date=YYYY-MM-DD` where `YYYY-MM-DD` is the date for collecting the stats. By default the stats is collected for the current day. Pass `date=1970-01-01` in order to collect global stats across all the days.
|
||||
* `focusLabel=LABEL_NAME` returns label values with the highest number of time series for the given `LABEL_NAME` in the `seriesCountByFocusLabelValue` list.
|
||||
* `match[]=SELECTOR` where `SELECTOR` is an arbitrary [time series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors) for series to take into account during stats calculation. By default all the series are taken into account.
|
||||
* `extra_label=LABEL=VALUE`. See [these docs](#prometheus-querying-api-enhancements) for more details.
|
||||
|
||||
|
||||
@ -482,6 +482,8 @@ The `/api/v1/export` endpoint should return the following response:
|
||||
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
|
||||
```
|
||||
|
||||
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
|
||||
|
||||
## Querying Graphite data
|
||||
|
||||
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
|
||||
@ -496,6 +498,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
|
||||
|
||||
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
|
||||
|
||||
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
|
||||
|
||||
|
||||
## How to send data from OpenTSDB-compatible agents
|
||||
|
||||
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
|
||||
@ -1136,7 +1141,9 @@ Example contents for `-relabelConfig` file:
|
||||
regex: true
|
||||
```
|
||||
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
|
||||
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
|
||||
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
|
||||
|
||||
|
||||
## Federation
|
||||
|
||||
|
||||
@ -256,12 +256,13 @@ Labels can be added to metrics by the following mechanisms:
|
||||
VictoriaMetrics components (including `vmagent`) support Prometheus-compatible relabeling.
|
||||
They provide the following additional actions on top of actions from the [Prometheus relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config):
|
||||
|
||||
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`.
|
||||
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`.
|
||||
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal.
|
||||
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal.
|
||||
* `keep_metrics`: keeps all the metrics with names matching the given `regex`.
|
||||
* `drop_metrics`: drops all the metrics with names matching the given `regex`.
|
||||
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`
|
||||
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`
|
||||
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal
|
||||
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal
|
||||
* `keep_metrics`: keeps all the metrics with names matching the given `regex`
|
||||
* `drop_metrics`: drops all the metrics with names matching the given `regex`
|
||||
* `graphite`: applies Graphite-style relabeling to metric name. See [these docs](#graphite-relabeling)
|
||||
|
||||
The `regex` value can be split into multiple lines for improved readability and maintainability. These lines are automatically joined with `|` char when parsed. For example, the following configs are equivalent:
|
||||
|
||||
@ -309,6 +310,38 @@ You can read more about relabeling in the following articles:
|
||||
* [Extracting labels from legacy metric names](https://www.robustperception.io/extracting-labels-from-legacy-metric-names)
|
||||
* [relabel_configs vs metric_relabel_configs](https://www.robustperception.io/relabel_configs-vs-metric_relabel_configs)
|
||||
|
||||
## Graphite relabeling
|
||||
|
||||
VictoriaMetrics components support `action: graphite` relabeling rules, which allow extracting various parts from Graphite-style metrics
|
||||
into the configued labels with the syntax similar to [Glob matching in statsd_exporter](https://github.com/prometheus/statsd_exporter#glob-matching).
|
||||
Note that the `name` field must be substituted with explicit `__name__` option under `labels` section.
|
||||
If `__name__` option is missing under `labels` section, then the original Graphite-style metric name is left unchanged.
|
||||
|
||||
For example, the following relabeling rule generates `requests_total{job="app42",instance="host124:8080"}` metric
|
||||
from "app42.host123.requests.total" Graphite-style metric:
|
||||
|
||||
```yaml
|
||||
- action: graphite
|
||||
match: "*.*.*.total"
|
||||
labels:
|
||||
__name__: "${3}_total"
|
||||
job: "$1"
|
||||
instance: "${2}:8080"
|
||||
```
|
||||
|
||||
Important notes about `action: graphite` relabeling rules:
|
||||
|
||||
- The relabeling rule is applied only to metrics, which match the given `match` expression. Other metrics remain unchanged.
|
||||
- The `*` matches the maximum possible number of chars until the next dot or until the next part of the `match` expression whichever comes first.
|
||||
It may match zero chars if the next char is `.`.
|
||||
For example, `match: "app*foo.bar"` matches `app42foo.bar` and `42` becomes available to use at `labels` section via `$1` capture group.
|
||||
- The `$0` capture group matches the original metric name.
|
||||
- The relabeling rules are executed in order defined in the original config.
|
||||
|
||||
The `action: graphite` relabeling rules are easier to write and maintain than `action: replace` for labels extraction from Graphite-style metric names.
|
||||
Additionally, the `action: graphite` relabeling rules usually work much faster than the equivalent `action: replace` rules.
|
||||
|
||||
|
||||
## Prometheus staleness markers
|
||||
|
||||
`vmagent` sends [Prometheus staleness markers](https://www.robustperception.io/staleness-and-promql) to `-remoteWrite.url` in the following cases:
|
||||
|
||||
@ -23,6 +23,22 @@ type RelabelConfig struct {
|
||||
Replacement *string `yaml:"replacement,omitempty"`
|
||||
Action string `yaml:"action,omitempty"`
|
||||
If *IfExpression `yaml:"if,omitempty"`
|
||||
|
||||
// Match is used together with Labels for `action: graphite`. For example:
|
||||
// - action: graphite
|
||||
// match: 'foo.*.*.bar'
|
||||
// labels:
|
||||
// job: '$1'
|
||||
// instance: '${2}:8080'
|
||||
Match string `yaml:"match,omitempty"`
|
||||
|
||||
// Labels is used together with Match for `action: graphite`. For example:
|
||||
// - action: graphite
|
||||
// match: 'foo.*.*.bar'
|
||||
// labels:
|
||||
// job: '$1'
|
||||
// instance: '${2}:8080'
|
||||
Labels map[string]string `yaml:"labels,omitempty"`
|
||||
}
|
||||
|
||||
// MultiLineRegex contains a regex, which can be split into multiple lines.
|
||||
@ -114,12 +130,12 @@ func (pcs *ParsedConfigs) String() string {
|
||||
if pcs == nil {
|
||||
return ""
|
||||
}
|
||||
var sb strings.Builder
|
||||
var a []string
|
||||
for _, prc := range pcs.prcs {
|
||||
fmt.Fprintf(&sb, "%s,", prc.String())
|
||||
s := "[" + prc.String() + "]"
|
||||
a = append(a, s)
|
||||
}
|
||||
fmt.Fprintf(&sb, "relabelDebug=%v", pcs.relabelDebug)
|
||||
return sb.String()
|
||||
return fmt.Sprintf("%s, relabelDebug=%v", strings.Join(a, ","), pcs.relabelDebug)
|
||||
}
|
||||
|
||||
// LoadRelabelConfigs loads relabel configs from the given path.
|
||||
@ -200,11 +216,38 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
|
||||
if rc.Replacement != nil {
|
||||
replacement = *rc.Replacement
|
||||
}
|
||||
var graphiteMatchTemplate *graphiteMatchTemplate
|
||||
if rc.Match != "" {
|
||||
graphiteMatchTemplate = newGraphiteMatchTemplate(rc.Match)
|
||||
}
|
||||
var graphiteLabelRules []graphiteLabelRule
|
||||
if rc.Labels != nil {
|
||||
graphiteLabelRules = newGraphiteLabelRules(rc.Labels)
|
||||
}
|
||||
action := rc.Action
|
||||
if action == "" {
|
||||
action = "replace"
|
||||
}
|
||||
switch action {
|
||||
case "graphite":
|
||||
if graphiteMatchTemplate == nil {
|
||||
return nil, fmt.Errorf("missing `match` for `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
if len(graphiteLabelRules) == 0 {
|
||||
return nil, fmt.Errorf("missing `labels` for `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
if len(rc.SourceLabels) > 0 {
|
||||
return nil, fmt.Errorf("`source_labels` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
if rc.TargetLabel != "" {
|
||||
return nil, fmt.Errorf("`target_label` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
if rc.Replacement != nil {
|
||||
return nil, fmt.Errorf("`replacement` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
if rc.Regex != nil {
|
||||
return nil, fmt.Errorf("`regex` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
|
||||
}
|
||||
case "replace":
|
||||
if targetLabel == "" {
|
||||
return nil, fmt.Errorf("missing `target_label` for `action=replace`")
|
||||
@ -274,6 +317,14 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown `action` %q", action)
|
||||
}
|
||||
if action != "graphite" {
|
||||
if graphiteMatchTemplate != nil {
|
||||
return nil, fmt.Errorf("`match` config cannot be applied to `action=%s`; it is applied only to `action=graphite`", action)
|
||||
}
|
||||
if len(graphiteLabelRules) > 0 {
|
||||
return nil, fmt.Errorf("`labels` config cannot be applied to `action=%s`; it is applied only to `action=graphite`", action)
|
||||
}
|
||||
}
|
||||
return &parsedRelabelConfig{
|
||||
SourceLabels: sourceLabels,
|
||||
Separator: separator,
|
||||
@ -284,6 +335,9 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
|
||||
Action: action,
|
||||
If: rc.If,
|
||||
|
||||
graphiteMatchTemplate: graphiteMatchTemplate,
|
||||
graphiteLabelRules: graphiteLabelRules,
|
||||
|
||||
regexOriginal: regexOriginalCompiled,
|
||||
hasCaptureGroupInTargetLabel: strings.Contains(targetLabel, "$"),
|
||||
hasCaptureGroupInReplacement: strings.Contains(replacement, "$"),
|
||||
|
||||
@ -45,6 +45,13 @@ func TestRelabelConfigMarshalUnmarshal(t *testing.T) {
|
||||
- null
|
||||
- nan
|
||||
`, "- regex:\n - \"-1.23\"\n - \"false\"\n - \"null\"\n - nan\n")
|
||||
f(`
|
||||
- action: graphite
|
||||
match: 'foo.*.*.aaa'
|
||||
labels:
|
||||
instance: '$1-abc'
|
||||
job: '${2}'
|
||||
`, "- action: graphite\n match: foo.*.*.aaa\n labels:\n instance: $1-abc\n job: ${2}\n")
|
||||
}
|
||||
|
||||
func TestLoadRelabelConfigsSuccess(t *testing.T) {
|
||||
@ -53,8 +60,9 @@ func TestLoadRelabelConfigsSuccess(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("cannot load relabel configs from %q: %s", path, err)
|
||||
}
|
||||
if n := pcs.Len(); n != 14 {
|
||||
t.Fatalf("unexpected number of relabel configs loaded from %q; got %d; want %d", path, n, 14)
|
||||
nExpected := 16
|
||||
if n := pcs.Len(); n != nExpected {
|
||||
t.Fatalf("unexpected number of relabel configs loaded from %q; got %d; want %d", path, n, nExpected)
|
||||
}
|
||||
}
|
||||
|
||||
@ -77,6 +85,51 @@ func TestLoadRelabelConfigsFailure(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestParsedConfigsString(t *testing.T) {
|
||||
f := func(rcs []RelabelConfig, sExpected string) {
|
||||
t.Helper()
|
||||
pcs, err := ParseRelabelConfigs(rcs, false)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %s", err)
|
||||
}
|
||||
s := pcs.String()
|
||||
if s != sExpected {
|
||||
t.Fatalf("unexpected string representation for ParsedConfigs;\ngot\n%s\nwant\n%s", s, sExpected)
|
||||
}
|
||||
}
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
TargetLabel: "foo",
|
||||
SourceLabels: []string{"aaa"},
|
||||
},
|
||||
}, "[SourceLabels=[aaa], Separator=;, TargetLabel=foo, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=replace, If=, "+
|
||||
"graphiteMatchTemplate=<nil>, graphiteLabelRules=[]], relabelDebug=false")
|
||||
var ie IfExpression
|
||||
if err := ie.Parse("{foo=~'bar'}"); err != nil {
|
||||
t.Fatalf("unexpected error when parsing if expression: %s", err)
|
||||
}
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
Labels: map[string]string{
|
||||
"job": "$1-zz",
|
||||
},
|
||||
If: &ie,
|
||||
},
|
||||
}, "[SourceLabels=[], Separator=;, TargetLabel=, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=graphite, If={foo=~'bar'}, "+
|
||||
"graphiteMatchTemplate=foo.*.bar, graphiteLabelRules=[replaceTemplate=$1-zz, targetLabel=job]], relabelDebug=false")
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "replace",
|
||||
SourceLabels: []string{"foo", "bar"},
|
||||
TargetLabel: "x",
|
||||
If: &ie,
|
||||
},
|
||||
}, "[SourceLabels=[foo bar], Separator=;, TargetLabel=x, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=replace, If={foo=~'bar'}, "+
|
||||
"graphiteMatchTemplate=<nil>, graphiteLabelRules=[]], relabelDebug=false")
|
||||
}
|
||||
|
||||
func TestParseRelabelConfigsSuccess(t *testing.T) {
|
||||
f := func(rcs []RelabelConfig, pcsExpected *ParsedConfigs) {
|
||||
t.Helper()
|
||||
@ -271,4 +324,110 @@ func TestParseRelabelConfigsFailure(t *testing.T) {
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("uppercase-missing-sourceLabels", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "uppercase",
|
||||
TargetLabel: "foobar",
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("lowercase-missing-targetLabel", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "lowercase",
|
||||
SourceLabels: []string{"foobar"},
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-missing-match", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Labels: map[string]string{
|
||||
"foo": "bar",
|
||||
},
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-missing-labels", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-superflouous-sourceLabels", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
Labels: map[string]string{
|
||||
"foo": "bar",
|
||||
},
|
||||
SourceLabels: []string{"foo"},
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-superflouous-targetLabel", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
Labels: map[string]string{
|
||||
"foo": "bar",
|
||||
},
|
||||
TargetLabel: "foo",
|
||||
},
|
||||
})
|
||||
})
|
||||
replacement := "foo"
|
||||
t.Run("graphite-superflouous-replacement", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
Labels: map[string]string{
|
||||
"foo": "bar",
|
||||
},
|
||||
Replacement: &replacement,
|
||||
},
|
||||
})
|
||||
})
|
||||
var re MultiLineRegex
|
||||
t.Run("graphite-superflouous-regex", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "graphite",
|
||||
Match: "foo.*.bar",
|
||||
Labels: map[string]string{
|
||||
"foo": "bar",
|
||||
},
|
||||
Regex: &re,
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("non-graphite-superflouos-match", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "uppercase",
|
||||
SourceLabels: []string{"foo"},
|
||||
TargetLabel: "foo",
|
||||
Match: "aaa",
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("non-graphite-superflouos-labels", func(t *testing.T) {
|
||||
f([]RelabelConfig{
|
||||
{
|
||||
Action: "uppercase",
|
||||
SourceLabels: []string{"foo"},
|
||||
TargetLabel: "foo",
|
||||
Labels: map[string]string{
|
||||
"foo": "Bar",
|
||||
},
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
212
lib/promrelabel/graphite.go
Normal file
212
lib/promrelabel/graphite.go
Normal file
@ -0,0 +1,212 @@
|
||||
package promrelabel
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
var graphiteMatchesPool = &sync.Pool{
|
||||
New: func() interface{} {
|
||||
return &graphiteMatches{}
|
||||
},
|
||||
}
|
||||
|
||||
type graphiteMatches struct {
|
||||
a []string
|
||||
}
|
||||
|
||||
type graphiteMatchTemplate struct {
|
||||
sOrig string
|
||||
parts []string
|
||||
}
|
||||
|
||||
func (gmt *graphiteMatchTemplate) String() string {
|
||||
return gmt.sOrig
|
||||
}
|
||||
|
||||
type graphiteLabelRule struct {
|
||||
grt *graphiteReplaceTemplate
|
||||
targetLabel string
|
||||
}
|
||||
|
||||
func (glr graphiteLabelRule) String() string {
|
||||
return fmt.Sprintf("replaceTemplate=%s, targetLabel=%s", glr.grt, glr.targetLabel)
|
||||
}
|
||||
|
||||
func newGraphiteLabelRules(m map[string]string) []graphiteLabelRule {
|
||||
a := make([]graphiteLabelRule, 0, len(m))
|
||||
for labelName, replaceTemplate := range m {
|
||||
a = append(a, graphiteLabelRule{
|
||||
grt: newGraphiteReplaceTemplate(replaceTemplate),
|
||||
targetLabel: labelName,
|
||||
})
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
func newGraphiteMatchTemplate(s string) *graphiteMatchTemplate {
|
||||
sOrig := s
|
||||
var parts []string
|
||||
for {
|
||||
n := strings.IndexByte(s, '*')
|
||||
if n < 0 {
|
||||
parts = appendGraphiteMatchTemplateParts(parts, s)
|
||||
break
|
||||
}
|
||||
parts = appendGraphiteMatchTemplateParts(parts, s[:n])
|
||||
parts = appendGraphiteMatchTemplateParts(parts, "*")
|
||||
s = s[n+1:]
|
||||
}
|
||||
return &graphiteMatchTemplate{
|
||||
sOrig: sOrig,
|
||||
parts: parts,
|
||||
}
|
||||
}
|
||||
|
||||
func appendGraphiteMatchTemplateParts(dst []string, s string) []string {
|
||||
if len(s) == 0 {
|
||||
// Skip empty part
|
||||
return dst
|
||||
}
|
||||
return append(dst, s)
|
||||
}
|
||||
|
||||
// Match matches s against gmt.
|
||||
//
|
||||
// On success it adds matched captures to dst and returns it with true.
|
||||
// Of failre it returns false.
|
||||
func (gmt *graphiteMatchTemplate) Match(dst []string, s string) ([]string, bool) {
|
||||
dst = append(dst, s)
|
||||
parts := gmt.parts
|
||||
if len(parts) > 0 {
|
||||
if p := parts[len(parts)-1]; p != "*" && !strings.HasSuffix(s, p) {
|
||||
// fast path - suffix mismatch
|
||||
return dst, false
|
||||
}
|
||||
}
|
||||
for i := 0; i < len(parts); i++ {
|
||||
p := parts[i]
|
||||
if p != "*" {
|
||||
if !strings.HasPrefix(s, p) {
|
||||
// Cannot match the current part
|
||||
return dst, false
|
||||
}
|
||||
s = s[len(p):]
|
||||
continue
|
||||
}
|
||||
// Search for the matching substring for '*' part.
|
||||
if i+1 >= len(parts) {
|
||||
// Matching the last part.
|
||||
if strings.IndexByte(s, '.') >= 0 {
|
||||
// The '*' cannot match string with dots.
|
||||
return dst, false
|
||||
}
|
||||
dst = append(dst, s)
|
||||
return dst, true
|
||||
}
|
||||
// Search for the the start of the next part.
|
||||
p = parts[i+1]
|
||||
i++
|
||||
n := strings.Index(s, p)
|
||||
if n < 0 {
|
||||
// Cannot match the next part
|
||||
return dst, false
|
||||
}
|
||||
tmp := s[:n]
|
||||
if strings.IndexByte(tmp, '.') >= 0 {
|
||||
// The '*' cannot match string with dots.
|
||||
return dst, false
|
||||
}
|
||||
dst = append(dst, tmp)
|
||||
s = s[n+len(p):]
|
||||
}
|
||||
return dst, len(s) == 0
|
||||
}
|
||||
|
||||
type graphiteReplaceTemplate struct {
|
||||
sOrig string
|
||||
parts []graphiteReplaceTemplatePart
|
||||
}
|
||||
|
||||
func (grt *graphiteReplaceTemplate) String() string {
|
||||
return grt.sOrig
|
||||
}
|
||||
|
||||
type graphiteReplaceTemplatePart struct {
|
||||
n int
|
||||
s string
|
||||
}
|
||||
|
||||
func newGraphiteReplaceTemplate(s string) *graphiteReplaceTemplate {
|
||||
sOrig := s
|
||||
var parts []graphiteReplaceTemplatePart
|
||||
for {
|
||||
n := strings.IndexByte(s, '$')
|
||||
if n < 0 {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, s, -1)
|
||||
break
|
||||
}
|
||||
if n > 0 {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, s[:n], -1)
|
||||
}
|
||||
s = s[n+1:]
|
||||
if len(s) > 0 && s[0] == '{' {
|
||||
// The index in the form ${123}
|
||||
n = strings.IndexByte(s, '}')
|
||||
if n < 0 {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, "$"+s, -1)
|
||||
break
|
||||
}
|
||||
idxStr := s[1:n]
|
||||
s = s[n+1:]
|
||||
idx, err := strconv.Atoi(idxStr)
|
||||
if err != nil {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, "${"+idxStr+"}", -1)
|
||||
} else {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, "${"+idxStr+"}", idx)
|
||||
}
|
||||
} else {
|
||||
// The index in the form $123
|
||||
n := 0
|
||||
for n < len(s) && s[n] >= '0' && s[n] <= '9' {
|
||||
n++
|
||||
}
|
||||
idxStr := s[:n]
|
||||
s = s[n:]
|
||||
idx, err := strconv.Atoi(idxStr)
|
||||
if err != nil {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, "$"+idxStr, -1)
|
||||
} else {
|
||||
parts = appendGraphiteReplaceTemplateParts(parts, "$"+idxStr, idx)
|
||||
}
|
||||
}
|
||||
}
|
||||
return &graphiteReplaceTemplate{
|
||||
sOrig: sOrig,
|
||||
parts: parts,
|
||||
}
|
||||
}
|
||||
|
||||
// Expand expands grt with the given matches into dst and returns it.
|
||||
func (grt *graphiteReplaceTemplate) Expand(dst []byte, matches []string) []byte {
|
||||
for _, part := range grt.parts {
|
||||
if n := part.n; n >= 0 && n < len(matches) {
|
||||
dst = append(dst, matches[n]...)
|
||||
} else {
|
||||
dst = append(dst, part.s...)
|
||||
}
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
func appendGraphiteReplaceTemplateParts(dst []graphiteReplaceTemplatePart, s string, n int) []graphiteReplaceTemplatePart {
|
||||
if len(s) > 0 {
|
||||
dst = append(dst, graphiteReplaceTemplatePart{
|
||||
s: s,
|
||||
n: n,
|
||||
})
|
||||
}
|
||||
return dst
|
||||
}
|
||||
93
lib/promrelabel/graphite_test.go
Normal file
93
lib/promrelabel/graphite_test.go
Normal file
@ -0,0 +1,93 @@
|
||||
package promrelabel
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestGraphiteTemplateMatchExpand(t *testing.T) {
|
||||
f := func(matchTpl, s, replaceTpl, resultExpected string) {
|
||||
t.Helper()
|
||||
gmt := newGraphiteMatchTemplate(matchTpl)
|
||||
matches, ok := gmt.Match(nil, s)
|
||||
if !ok {
|
||||
matches = nil
|
||||
}
|
||||
grt := newGraphiteReplaceTemplate(replaceTpl)
|
||||
result := grt.Expand(nil, matches)
|
||||
if string(result) != resultExpected {
|
||||
t.Fatalf("unexpected result; got %q; want %q", result, resultExpected)
|
||||
}
|
||||
}
|
||||
f("", "", "", "")
|
||||
f("test.*.*.counter", "test.foo.bar.counter", "${2}_total", "bar_total")
|
||||
f("test.*.*.counter", "test.foo.bar.counter", "$1_total", "foo_total")
|
||||
f("test.*.*.counter", "test.foo.bar.counter", "total_$0", "total_test.foo.bar.counter")
|
||||
f("test.dispatcher.*.*.*", "test.dispatcher.foo.bar.baz", "$3-$2-$1", "baz-bar-foo")
|
||||
f("*.signup.*.*", "foo.signup.bar.baz", "$1-${3}_$2_total", "foo-baz_bar_total")
|
||||
}
|
||||
|
||||
func TestGraphiteMatchTemplateMatch(t *testing.T) {
|
||||
f := func(tpl, s string, matchesExpected []string, okExpected bool) {
|
||||
t.Helper()
|
||||
gmt := newGraphiteMatchTemplate(tpl)
|
||||
tplGot := gmt.String()
|
||||
if tplGot != tpl {
|
||||
t.Fatalf("unexpected template; got %q; want %q", tplGot, tpl)
|
||||
}
|
||||
matches, ok := gmt.Match(nil, s)
|
||||
if ok != okExpected {
|
||||
t.Fatalf("unexpected ok result for tpl=%q, s=%q; got %v; want %v", tpl, s, ok, okExpected)
|
||||
}
|
||||
if okExpected {
|
||||
if !reflect.DeepEqual(matches, matchesExpected) {
|
||||
t.Fatalf("unexpected matches for tpl=%q, s=%q; got\n%q\nwant\n%q\ngraphiteMatchTemplate=%v", tpl, s, matches, matchesExpected, gmt)
|
||||
}
|
||||
}
|
||||
}
|
||||
f("", "", []string{""}, true)
|
||||
f("", "foobar", nil, false)
|
||||
f("foo", "foo", []string{"foo"}, true)
|
||||
f("foo", "", nil, false)
|
||||
f("foo.bar.baz", "foo.bar.baz", []string{"foo.bar.baz"}, true)
|
||||
f("*", "foobar", []string{"foobar", "foobar"}, true)
|
||||
f("**", "foobar", nil, false)
|
||||
f("*", "foo.bar", nil, false)
|
||||
f("*foo", "barfoo", []string{"barfoo", "bar"}, true)
|
||||
f("*foo", "foo", []string{"foo", ""}, true)
|
||||
f("*foo", "bar.foo", nil, false)
|
||||
f("foo*", "foobar", []string{"foobar", "bar"}, true)
|
||||
f("foo*", "foo", []string{"foo", ""}, true)
|
||||
f("foo*", "foo.bar", nil, false)
|
||||
f("foo.*", "foobar", nil, false)
|
||||
f("foo.*", "foo.bar", []string{"foo.bar", "bar"}, true)
|
||||
f("foo.*", "foo.bar.baz", nil, false)
|
||||
f("*.*.baz", "foo.bar.baz", []string{"foo.bar.baz", "foo", "bar"}, true)
|
||||
f("*.bar", "foo.bar.baz", nil, false)
|
||||
f("*.bar", "foo.baz", nil, false)
|
||||
}
|
||||
|
||||
func TestGraphiteReplaceTemplateExpand(t *testing.T) {
|
||||
f := func(tpl string, matches []string, resultExpected string) {
|
||||
t.Helper()
|
||||
grt := newGraphiteReplaceTemplate(tpl)
|
||||
tplGot := grt.String()
|
||||
if tplGot != tpl {
|
||||
t.Fatalf("unexpected template; got %q; want %q", tplGot, tpl)
|
||||
}
|
||||
result := grt.Expand(nil, matches)
|
||||
if string(result) != resultExpected {
|
||||
t.Fatalf("unexpected result for tpl=%q; got\n%q\nwant\n%q\ngraphiteReplaceTemplate=%v", tpl, result, resultExpected, grt)
|
||||
}
|
||||
}
|
||||
f("", nil, "")
|
||||
f("foo", nil, "foo")
|
||||
f("$", nil, "$")
|
||||
f("$1", nil, "$1")
|
||||
f("${123", nil, "${123")
|
||||
f("${123}", nil, "${123}")
|
||||
f("${foo}45$sdf$3", nil, "${foo}45$sdf$3")
|
||||
f("$1", []string{"foo", "bar"}, "bar")
|
||||
f("$0-$1", []string{"foo", "bar"}, "foo-bar")
|
||||
f("x-${0}-$1", []string{"foo", "bar"}, "x-foo-bar")
|
||||
}
|
||||
93
lib/promrelabel/graphite_timing_test.go
Normal file
93
lib/promrelabel/graphite_timing_test.go
Normal file
@ -0,0 +1,93 @@
|
||||
package promrelabel
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func BenchmarkGraphiteMatchTemplateMatch(b *testing.B) {
|
||||
b.Run("match-short", func(b *testing.B) {
|
||||
tpl := "*.bar.baz"
|
||||
s := "foo.bar.baz"
|
||||
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, true)
|
||||
})
|
||||
b.Run("mismtach-short", func(b *testing.B) {
|
||||
tpl := "*.bar.baz"
|
||||
s := "foo.aaa"
|
||||
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, false)
|
||||
})
|
||||
b.Run("match-long", func(b *testing.B) {
|
||||
tpl := "*.*.*.bar.*.baz"
|
||||
s := "foo.bar.baz.bar.aa.baz"
|
||||
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, true)
|
||||
})
|
||||
b.Run("mismatch-long", func(b *testing.B) {
|
||||
tpl := "*.*.*.bar.*.baz"
|
||||
s := "foo.bar.baz.bar.aa.bb"
|
||||
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, false)
|
||||
})
|
||||
}
|
||||
|
||||
func benchmarkGraphiteMatchTemplateMatch(b *testing.B, tpl, s string, okExpected bool) {
|
||||
gmt := newGraphiteMatchTemplate(tpl)
|
||||
b.ReportAllocs()
|
||||
b.SetBytes(1)
|
||||
b.RunParallel(func(pb *testing.PB) {
|
||||
var matches []string
|
||||
for pb.Next() {
|
||||
var ok bool
|
||||
matches, ok = gmt.Match(matches[:0], s)
|
||||
if ok != okExpected {
|
||||
panic(fmt.Errorf("unexpected ok=%v for tpl=%q, s=%q", ok, tpl, s))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func BenchmarkGraphiteReplaceTemplateExpand(b *testing.B) {
|
||||
b.Run("one-replacement", func(b *testing.B) {
|
||||
tpl := "$1"
|
||||
matches := []string{"", "foo"}
|
||||
resultExpected := "foo"
|
||||
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
|
||||
})
|
||||
b.Run("one-replacement-with-prefix", func(b *testing.B) {
|
||||
tpl := "x-$1"
|
||||
matches := []string{"", "foo"}
|
||||
resultExpected := "x-foo"
|
||||
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
|
||||
})
|
||||
b.Run("one-replacement-with-prefix-suffix", func(b *testing.B) {
|
||||
tpl := "x-$1-y"
|
||||
matches := []string{"", "foo"}
|
||||
resultExpected := "x-foo-y"
|
||||
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
|
||||
})
|
||||
b.Run("two-replacements", func(b *testing.B) {
|
||||
tpl := "$1$2"
|
||||
matches := []string{"", "foo", "bar"}
|
||||
resultExpected := "foobar"
|
||||
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
|
||||
})
|
||||
b.Run("two-replacements-with-delimiter", func(b *testing.B) {
|
||||
tpl := "$1-$2"
|
||||
matches := []string{"", "foo", "bar"}
|
||||
resultExpected := "foo-bar"
|
||||
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
|
||||
})
|
||||
}
|
||||
|
||||
func benchmarkGraphiteReplaceTemplateExpand(b *testing.B, tpl string, matches []string, resultExpected string) {
|
||||
grt := newGraphiteReplaceTemplate(tpl)
|
||||
b.ReportAllocs()
|
||||
b.SetBytes(1)
|
||||
b.RunParallel(func(pb *testing.PB) {
|
||||
var b []byte
|
||||
for pb.Next() {
|
||||
b = grt.Expand(b[:0], matches)
|
||||
if string(b) != resultExpected {
|
||||
panic(fmt.Errorf("unexpected result; got\n%q\nwant\n%q", b, resultExpected))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
@ -18,6 +18,14 @@ type IfExpression struct {
|
||||
lfs []*labelFilter
|
||||
}
|
||||
|
||||
// String returns string representation of ie.
|
||||
func (ie *IfExpression) String() string {
|
||||
if ie == nil {
|
||||
return ""
|
||||
}
|
||||
return ie.s
|
||||
}
|
||||
|
||||
// Parse parses `if` expression from s and stores it to ie.
|
||||
func (ie *IfExpression) Parse(s string) error {
|
||||
expr, err := metricsql.Parse(s)
|
||||
|
||||
@ -2,6 +2,7 @@ package promrelabel
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
@ -36,6 +37,36 @@ func TestIfExpressionParseSuccess(t *testing.T) {
|
||||
f(`foo{bar=~"baz", x!="y"}`)
|
||||
}
|
||||
|
||||
func TestIfExpressionMarshalUnmarshalJSON(t *testing.T) {
|
||||
f := func(s, jsonExpected string) {
|
||||
t.Helper()
|
||||
var ie IfExpression
|
||||
if err := ie.Parse(s); err != nil {
|
||||
t.Fatalf("cannot parse ifExpression %q: %s", s, err)
|
||||
}
|
||||
data, err := json.Marshal(&ie)
|
||||
if err != nil {
|
||||
t.Fatalf("cannot marshal ifExpression %q: %s", s, err)
|
||||
}
|
||||
if string(data) != jsonExpected {
|
||||
t.Fatalf("unexpected value after json marshaling;\ngot\n%s\nwant\n%s", data, jsonExpected)
|
||||
}
|
||||
var ie2 IfExpression
|
||||
if err := json.Unmarshal(data, &ie2); err != nil {
|
||||
t.Fatalf("cannot unmarshal ifExpression from json %q: %s", data, err)
|
||||
}
|
||||
data2, err := json.Marshal(&ie2)
|
||||
if err != nil {
|
||||
t.Fatalf("cannot marshal ifExpression2: %s", err)
|
||||
}
|
||||
if string(data2) != jsonExpected {
|
||||
t.Fatalf("unexpected data after unmarshal/marshal cycle;\ngot\n%s\nwant\n%s", data2, jsonExpected)
|
||||
}
|
||||
}
|
||||
f("foo", `"foo"`)
|
||||
f(`{foo="bar",baz=~"x.*"}`, `"{foo=\"bar\",baz=~\"x.*\"}"`)
|
||||
}
|
||||
|
||||
func TestIfExpressionUnmarshalFailure(t *testing.T) {
|
||||
f := func(s string) {
|
||||
t.Helper()
|
||||
|
||||
@ -25,6 +25,9 @@ type parsedRelabelConfig struct {
|
||||
Action string
|
||||
If *IfExpression
|
||||
|
||||
graphiteMatchTemplate *graphiteMatchTemplate
|
||||
graphiteLabelRules []graphiteLabelRule
|
||||
|
||||
regexOriginal *regexp.Regexp
|
||||
hasCaptureGroupInTargetLabel bool
|
||||
hasCaptureGroupInReplacement bool
|
||||
@ -32,8 +35,8 @@ type parsedRelabelConfig struct {
|
||||
|
||||
// String returns human-readable representation for prc.
|
||||
func (prc *parsedRelabelConfig) String() string {
|
||||
return fmt.Sprintf("SourceLabels=%s, Separator=%s, TargetLabel=%s, Regex=%s, Modulus=%d, Replacement=%s, Action=%s",
|
||||
prc.SourceLabels, prc.Separator, prc.TargetLabel, prc.Regex.String(), prc.Modulus, prc.Replacement, prc.Action)
|
||||
return fmt.Sprintf("SourceLabels=%s, Separator=%s, TargetLabel=%s, Regex=%s, Modulus=%d, Replacement=%s, Action=%s, If=%s, graphiteMatchTemplate=%s, graphiteLabelRules=%s",
|
||||
prc.SourceLabels, prc.Separator, prc.TargetLabel, prc.Regex, prc.Modulus, prc.Replacement, prc.Action, prc.If, prc.graphiteMatchTemplate, prc.graphiteLabelRules)
|
||||
}
|
||||
|
||||
// Apply applies pcs to labels starting from the labelsOffset.
|
||||
@ -147,6 +150,26 @@ func (prc *parsedRelabelConfig) apply(labels []prompbmarshal.Label, labelsOffset
|
||||
return labels
|
||||
}
|
||||
switch prc.Action {
|
||||
case "graphite":
|
||||
metricName := GetLabelValueByName(src, "__name__")
|
||||
gm := graphiteMatchesPool.Get().(*graphiteMatches)
|
||||
var ok bool
|
||||
gm.a, ok = prc.graphiteMatchTemplate.Match(gm.a[:0], metricName)
|
||||
if !ok {
|
||||
// Fast path - name mismatch
|
||||
graphiteMatchesPool.Put(gm)
|
||||
return labels
|
||||
}
|
||||
// Slow path - extract labels from graphite metric name
|
||||
bb := relabelBufPool.Get()
|
||||
for _, gl := range prc.graphiteLabelRules {
|
||||
bb.B = gl.grt.Expand(bb.B[:0], gm.a)
|
||||
valueStr := string(bb.B)
|
||||
labels = setLabelValue(labels, labelsOffset, gl.targetLabel, valueStr)
|
||||
}
|
||||
relabelBufPool.Put(bb)
|
||||
graphiteMatchesPool.Put(gm)
|
||||
return labels
|
||||
case "replace":
|
||||
// Store `replacement` at `target_label` if the `regex` matches `source_labels` joined with `separator`
|
||||
bb := relabelBufPool.Get()
|
||||
|
||||
@ -1580,7 +1580,6 @@ func TestApplyRelabelConfigs(t *testing.T) {
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
t.Run("upper-lower-case", func(t *testing.T) {
|
||||
f(`
|
||||
- action: uppercase
|
||||
@ -1618,7 +1617,6 @@ func TestApplyRelabelConfigs(t *testing.T) {
|
||||
Value: "bar;foo",
|
||||
},
|
||||
})
|
||||
})
|
||||
f(`
|
||||
- action: lowercase
|
||||
source_labels: ["foo"]
|
||||
@ -1637,6 +1635,49 @@ func TestApplyRelabelConfigs(t *testing.T) {
|
||||
Value: "quux",
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-match", func(t *testing.T) {
|
||||
f(`
|
||||
- action: graphite
|
||||
match: foo.*.baz
|
||||
labels:
|
||||
__name__: aaa
|
||||
job: ${1}-zz
|
||||
`, []prompbmarshal.Label{
|
||||
{
|
||||
Name: "__name__",
|
||||
Value: "foo.bar.baz",
|
||||
},
|
||||
}, true, []prompbmarshal.Label{
|
||||
{
|
||||
Name: "__name__",
|
||||
Value: "aaa",
|
||||
},
|
||||
{
|
||||
Name: "job",
|
||||
Value: "bar-zz",
|
||||
},
|
||||
})
|
||||
})
|
||||
t.Run("graphite-mismatch", func(t *testing.T) {
|
||||
f(`
|
||||
- action: graphite
|
||||
match: foo.*.baz
|
||||
labels:
|
||||
__name__: aaa
|
||||
job: ${1}-zz
|
||||
`, []prompbmarshal.Label{
|
||||
{
|
||||
Name: "__name__",
|
||||
Value: "foo.bar.bazz",
|
||||
},
|
||||
}, true, []prompbmarshal.Label{
|
||||
{
|
||||
Name: "__name__",
|
||||
Value: "foo.bar.bazz",
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
func TestFinalizeLabels(t *testing.T) {
|
||||
|
||||
@ -39,3 +39,11 @@
|
||||
- source_labels: [__tmp_uppercase]
|
||||
target_label: lower_aaa
|
||||
action: lowercase
|
||||
- if: '{foo=~"bar.*",baz="aa"}'
|
||||
target_label: aaa
|
||||
replacement: foobar
|
||||
- action: graphite
|
||||
match: 'foo.*.bar'
|
||||
labels:
|
||||
instance: 'foo-$1'
|
||||
job: '${1}-bar'
|
||||
|
||||
Loading…
Reference in New Issue
Block a user