mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-29 23:30:04 +01:00

Aliaksandr Valialkin 3a0c69651a

2022-10-14 11:33:25 +03:00

269 KiB

Raw Blame History

sort
16

CHANGELOG

The following tip changes can be tested by building VictoriaMetrics components from the latest commits according to the following docs:

tip

FEATURE: allow limiting memory usage on a per-query basis with -search.maxMemoryPerQuery command-line flag. See this feature request.
FEATURE: vmagent: drop all the labels with __ prefix from discovered targets in the same way as Prometheus does according to this article. Previously the following labels were available during metric-level relabeling: __address__, __scheme__, __metrics_path__, __scrape_interval__, __scrape_timeout__, __param_*. Now these labels are available only during target-level relabeling. This should reduce CPU usage and memory usage for vmagent setups, which scrape big number of targets.
FEATURE: vmagent: improve the performance for metric-level relabeling, which can be applied via metric_relabel_configs section at scrape_configs, via -remoteWrite.relabelConfig or via -remoteWrite.urlRelabelConfig command-line options.

FEATURE: vmagent: allow specifying full url in scrape target addresses (aka __address__ label). This makes valid the following -promscrape.config:

scrape_configs:
- job_name: abc
  metrics_path: /foo/bar
  scheme: https
  static_configs:
  - targets:
    # the following targets are scraped by the provided full urls
    - 'http://host1/metric/path1'
    - 'https://host2/metric/path2'
    - 'http://host3:1234/metric/path3?arg1=value1'
    # the following target is scraped by <scheme>://host4:1234<metrics_path>
    - host4:1234

See the corresponding issue.

FEATURE: vmagent: allow controlling staleness tracking on a per-scrape_config basis by specifying no_stale_markers: true or no_stale_markers: false option in the corresponding scrape_config.
FEATURE: vmui: limit the number of plotted series. This should prevent from browser crashes or hands when the query returns big number of time series. See this feature request.
BUGFIX: MetricsQL: properly merge buckets with identical le values, but with different string representation of these values when calculating histogram_quantile and histogram_share. For example, http_request_duration_seconds_bucket{le="5"} and http_requests_duration_seconds_bucket{le="5.0"}. Such buckets may be returned from distinct targets. Thanks to @647-coder for the pull request.

v1.82.1

Released at 14-10-2022

BUGFIX: vmui: automatically update graph, legend and url after the removal of query field. See this feature request and this comment.
BUGFIX: vmalert: remove duplicate alertname JSON entry from generated alerts. See this issue. Thanks to @Howie59 for the fix!
BUGFIX: vmalert: fix integration with Grafana via -vmalert.proxyURL, which has been broken in v1.82.0. See this issue.
BUGFIX: vmbackup: set default region to us-east-1 if AWS_REGION environment variable isn't set. The issue was introduced in vmbackup v1.82.0. See this pull request.
BUGFIX: vmbackupmanager: fix deletion of old backups at Azure blob storage.
BUGFIX: MetricsQL: properly apply regex filters when searching for time series. Previously unexpected time series could be returned from regex filter. See this issue. The issue was introduced in v1.82.0.
BUGFIX: vmagent: properly apply if section with regex filters. Previously unexpected metrics could be returned from if section. The issue was introduced in v1.82.0.

v1.82.0

Released at 07-10-2022

Update note 1: this release changes data format for /api/v1/export/native in incompatible way, so it cannot be imported into older version of VictoriaMetrics via /api/v1/import/native.

Update note 2: vmalert changes default value for command-line flag -datasource.queryStep from 0s to 5m. The change supposed to improve reliability of the rules evaluation when evaluation interval is lower than scraping interval.

Update note 3: vm_account_id and vm_project_id labels must be passed to tcp-based Graphite, InfluxDB and OpenTSDB endpoints at VictoriaMetrics cluster instead of undocumented VictoriaMetrics_AccountID and VictoriaMetrics_ProjectID labels when writing samples to the needed tenant. See these docs for details.

FEATURE: VictoriaMetrics cluster: support specifying tenant ids via vm_account_id and vm_project_id labels. See these docs and this feature request.
FEATURE: vmagent: improve relabeling performance by up to 3x for non-trivial regex values such as ([^:]+):.+, which can be used for extracting a host part from host:port label value.
FEATURE: MetricsQL: improve performance by up to 4x for queries containing non-trivial regex filters such as {path=~"/foo/.+|/bar"}.
FEATURE: improve performance scalability on systems with many CPU cores for /federate and /api/v1/export/... endpoints.
FEATURE: sanitize metric names for data ingested via DataDog protocol according to DataDog metric naming. The behaviour can be disabled by passing -datadog.sanitizeMetricName=false command-line flag. Thanks to @PerGon for the pull request.
FEATURE: add -usePromCompatibleNaming command-line flag to vmagent, to single-node VictoriaMetrics and to vminsert component of VictoriaMetrics cluster. This flag can be used for normalizing the ingested metric names and label names to Prometheus-compatible form. If this flag is set, then all the chars unsupported by Prometheus are replaced with _ chars in metric names and labels of the ingested samples. See this feature request.
FEATURE: accept whitespace in metric names and tags ingested via Graphite plaintext protocol according to the specs. See this issue.
FEATURE: check the correctess of raw sample timestamps stored on disk when reading them. This reduces the probability of possible silent corruption of the data stored on disk. This should help this and this issue.
FEATURE: atomically delete directories with snapshots, parts and partitions at storage level. Previously such directories can be left in partially deleted state when the deletion operation was interrupted by unclean shutdown. This may result in cannot open file ...: no such file or directory error on the next start. The probability of this error was quite high when NFS or EFS was used as persistent storage for VictoriaMetrics data. See this issue.
FEATURE: set the start arg to end - 5 minutes if isn't passed explicitly to /api/v1/labels and /api/v1/label/.../values. See this pull request.
FEATURE: allow to define the minimum TLS version to use when accepting https requests to VictoriaMetrics components if -tls command-line flag is set. The minimum TLS version can be set via -tlsMinVersion command-line flag. See this feature request.
FEATURE: vmctl: add vm-native-step-interval command line flag for vm-native mode. New option allows splitting the import process into chunks by time interval. This helps migrating data sets with high churn rate and provides better control over the process. See feature request.
FEATURE: vmui: add top queries tab, which shows various stats for recently executed queries. See these docs and this feature request.
FEATURE: vmui: move the "Execute Query" and "Add Query" buttons below the query fields, change icon for remove query. See this issue.
FEATURE: vmui: set the maximum number of queries to 4, remove multi Y-axes, left one for all queries and dotted lines to indicate queries in the graph. See this issue.
FEATURE: vmalert: add debug mode to the alerting rule settings for printing additional information into logs during evaluation. See debug param in alerting rule config.
FEATURE: vmalert: add experimental feature for displaying last 10 states of the rule (recording or alerting) evaluation. The state is available on the Rule page, which can be opened by clicking on Details link next to Rule's name on the /groups page.
FEATURE: vmalert: allow using extra labels in annotiations. See this feature request.
FEATURE: vmalert: allow configuring authorization params per list of targets in vmalert's notifier config for static_configs. See this issue.
FEATURE: vmalert: allow using {{$labels}} for templating in command-line flag -external.alert.source. The change supposed to provide additional flexibility for generating alert's source link based on labels values.
FEATURE: vmalert: add vm_account_id and vm_project_id labels to results of alerting and recording rules if -clusterMode is enabled. This improves multitenant support in vmalert.
FEATURE: vmagent: minimize the time needed for reading large responses from scrape targets in stream parsing mode. This should reduce scrape durations for such targets as kube-state-metrics running in a big Kubernetes cluster.
FEATURE: MetricsQL: add sort_by_label_numeric and sort_by_label_numeric_desc functions for numeric sort of input time series by the specified labels. See this feature request.
FEATURE: vmbackup and vmrestore: retry GCS operations for up to 3 minutes on temporary failures. See this issue.
FEATURE: vmbackup: add support for saving / restoring backups to / from Azure blob storage. See this feature request.
FEATURE: vmbackupmanager: expose vm_backup_in_flight metric, which can be used for determining which backup types - latest, hourly, daily, weekly or monthly - are currently executed.
FEATURE: vmgateway: add ability to extract JWT authorization token from non-standard HTTP header by passing it via -auth.httpHeader command-line flag. See this feature request.
FEATURE: vmagent: expose __meta_ec2_region label for ec2_sd_config in the same way as Prometheus 2.39 does.
FEATURE: vmagent: accept data ingestion requests via paths starting from /prometheus prefix in the same way as VictoriaMetrics does. For example, vmagent now accepts Prometheus remote_write data via both /api/v1/write and /prometheus/api/v1/write. This simplifies switching between single-node VictoriaMetrics and vmagent.
FEATURE: vmagent: add external_labels from global section at -promscrape.config after the relabeling is applied to scraped metrics. This aligns with Prometheus behaviour. Previously the external_labels were added to scrape targets, so they could be modified during relabeling. See this issue.
FEATURE: vmagent: allow specifying per--remoteWrite.url limits for on-disk size for pending data via -remoteWrite.maxDiskUsagePerURL command-line flag. Thanks to @rbizos for the pull request.
FEATURE: VictoriaMetrics cluster: log clear error when multiple identical -storageNode command-line flags are passed to vmselect or to vminsert. Previously these components were crashed with cryptic panic metric ... is already registered in this case. See this issue.
BUGFIX: do not export stale metrics via /federate api after the staleness markers. Previously such metrics were exported with NaN values. this could break some setups. See this issue.
BUGFIX: export ininity numbers as "Infinity" strings at /api/v1/export, so they can be parsed by standard JSON parsers. Previously infinity numbers were exported as Inf values, which couldn't be parsed by standard JSON parsers. See this issue.
BUGFIX: vmauth: properly handle request paths ending with / such as /vmui/. Previously vmui was dropping the traling /, which could prevent from using vmui via vmauth. See this issue.
BUGFIX: vmagent: properly encode query params for aws signed requests, use %20 instead of + as api requires. See this issue.
BUGFIX: vmagent: properly parse relabel config when regex ending with escaped $. See this issue.
BUGFIX: MetricsQL: properly calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d. Previously the sum_over_time(m[d]) could be improperly divided by smaller than d time range. See rate_over_sum() docs and this issue.
BUGFIX: MetricsQL: properly calculate increase(m[d]) over slow-changing counters with values smaller than 100. Previously increase could return unexpectedly big results in this case. See the related issue and this pull request.
BUGFIX: MetricsQL: ignore empty series when applying limit_offset. It should improve queries with additional filters by value in expressions like limit_offset(1,1, foo > 1).
BUGFIX: MetricsQL: properly calculate quantiles_over_time when the lookbehind window contains only a single sample. Previously an empty result was incorrectly returned in this case.
BUGFIX: vmui: fix RangeError: Maximum call stack size exceeded error when the query returns too many data points at Table view. See this pull request.
BUGFIX: vmui: fix workaround for adding more queries via URL. See this issue.
BUGFIX: vmalert: re-evaluate annotations per each alert evaluation. Previously, annotations were evaluated only on alert's value change. This could result in stale annotations in some cases described in this pull request.
BUGFIX: prevent from excessive CPU usage when the storage enters read-only mode. The previous fix in v1.81.0 wasn't complete.
BUGFIX: vmalert: change default value for command-line flag -datasource.queryStep from 0s to 5m. Param step is added by vmalert to every rule evaluation request sent to datasource. Before this change, step was equal to group's evaluation interval by default. Param step for instant queries defines how far VM can look back for the last written data point. The change supposed to improve reliability of the rules evaluation when evaluation interval is lower than scraping interval.
BUGFIX: properly calculate vm_rows_scanned_per_query histogram exported at /metrics page of vmselect and single-node VictoriaMetrics. Previously it could return misleadingly high numbers for rollup functions, which scan only a few samples on the provided lookbehind window in square brackets. For example, increase(m[1d]) always scans only 2 rows (aka raw samples) per each returned time series.

v1.81.2

Released at 08-09-2022

BUGFIX: VictoriaMetrics cluster: properly calculate query results at vmselect. See this issue. The issue has been introduced in v1.81.0.

v1.81.1

Released at 02-09-2022

It isn't recommended to use VictoriaMetrics cluster v1.81.1 because of the bug, which may result in incorrect query results. Upgrade to v1.81.2 instead.

FEATURE: MetricsQL: evaluate q1, ..., qN in parallel when calculating union(q1, .., qN). Previously union args were evaluated sequentially. This could result in lower than expected performance.
BUGFIX: VictoriaMetrics cluster: fix potential panic at vmselect under high load, which has been introduced in v1.81.0. See this issue.

v1.81.0

It isn't recommended to use VictoriaMetrics cluster v1.81.0 because of the bug, which may result in vmselect crashes under high load. Upgrade to v1.81.2 instead.

Released at 31-08-2022

Update note 1: vmalert by default hides values of -remoteWrite.url, -remoteRead.url and -datasource.url in logs and at http://vmalert:8880/flags for security reasons. See the corresponding SECURITY change in the Chagelog below for additional info.

Update note 2: vmalert by default points alert source url to /vmalert/alert?... aka web UI instead of /vmalert/api/v1/alert?... aka JSON handler. The old behavior can be achieved by setting {% raw %}-external.alert.source=vmalert/api/v1/alert?group_id={{.GroupID}}&alert_id={{.AlertID}}{% endraw %} command-line flag.

SECURITY: vmalert: do not expose -remoteWrite.url, -remoteRead.url and -datasource.url command-line flag values in logs and at http://vmalert:8880/flags page by default, since they may contain sensitive data such as auth keys. This aligns vmalert behaviour with vmagent, which doesn't expose -remoteWrite.url command-line flag value in logs and at http://vmagent:8429/flags page by default. Specify -remoteWrite.showURL, -remoteRead.showURL and -datasource.showURL command-line flags for showing values for the corresponding -*.url flags in logs. Thanks to @mble for the pull request.
SECURITY: upgrade base docker image (alpine) from 3.16.1 to 3.16.2. See alpine 3.16.2 release notes.
FEATURE: return shorter error messages to Grafana and to other clients requesting /api/v1/query and /api/v1/query_range endpoints. This should simplify reading these errors by humans. The long error message with full context is still written to logs.
FEATURE: add the ability to fine-tune the number of points, which can be generated per each matching time series during subquery evaluation. This can be done with the -search.maxPointsSubqueryPerTimeseries command-line flag. See this feature request.
FEATURE: vmagent: improve the performance for relabeling rules with commonly used regular expressions in regex and if fields such as some_string, prefix.*, prefix.+, foo|bar|baz, .*foo.* and .+foo.+.
FEATURE: vmagent: reduce CPU usage when discovering big number of Kubernetes targets with big number of labels and annotations.
FEATURE: vmagent: add ability to accept multitenant data via OpenTSDB /api/put protocol at /insert/<tenantID>/opentsdb/api/put http endpoint if multitenant support is enabled at vmagent. Thanks to @chengjianyun for the pull request.
FEATURE: monitoring: expose vm_hourly_series_limit_max_series, vm_hourly_series_limit_current_series, vm_daily_series_limit_max_series and vm_daily_series_limit_current_series metrics when -search.maxHourlySeries or -search.maxDailySeries limits are set. This allows alerting when the number of unique series reaches the configured limits. See these docs for details.
FEATURE: VictoriaMetrics cluster: reduce the amounts of logging at vmstorage when vmselect connects/disconnects to vmstorage.
FEATURE: VictoriaMetrics cluster: improve performance for heavy queries on systems with many CPU cores.
FEATURE: vmagent: add ability to use {% raw %}{{label_name}}{% endraw %} placeholders in the replacement option of relabeling rules. This simplifies constructing label values from multiple existing label values. See these docs for details.
FEATURE: vmagent: generate additional per-target metrics - scrape_series_limit, scrape_series_current and scrape_series_limit_samples_dropped if series limit is set according to these docs. This simplifies alerting on targets with the exceeded series limit. See these docs for details on these metrics.
FEATURE: vmagent: add support for MX record types in dns_sd_configs in the same way as Prometheus 2.38 does.
FEATURE: vmagent: add __meta_kubernetes_service_port_number meta-label for role: service in kubernetes_sd_configs in the same way as Prometheus 2.38 does.
FEATURE: vmagent: add __meta_kubernetes_pod_container_image meta-label for role: pod in kubernetes_sd_configs in the same way as Prometheus 2.38 does.
FEATURE: vmagent: retry HTTP requests after some wait time during service discovery and during target scrapes if the server returns 429 HTTP status code (aka Too many requests). See this issue.
FEATURE: vmui: add a legend in the top right corner for shortcut keys. See this feature request.
FEATURE: vmalert: add toTime() template function in the same way as Prometheus 2.38 does. See these docs.
FEATURE: vmalert: add $alertID and $groupID template variables. These variables may be used for templating annotations or -external.alert.source command-line flag. See the full list of supported variables here.
FEATURE: vmalert: add $activeAt template variable. See this feature request. See the full list of supported variables here. Thanks to @laixintao for the pull request.
FEATURE: vmalert: point alert source to vmalert's UI at /vmalert/alert?... instead of JSON handler at /vmalert/api/v1/alert?.... This improves user experience. The old behavior can be achieved by setting {% raw %}-external.alert.source=vmalert/api/v1/alert?group_id={{.GroupID}}&alert_id={{.AlertID}}{% endraw %} command-line flag.
BUGFIX: prevent from excess CPU usage when the storage enters read-only mode.
BUGFIX: improve performance for requests to /api/v1/labels and /api/v1/label/.../values when the filter in the match[] query arg matches small number of time series. The performance for this case has been reduced in v1.78.0. See this and this issues.
BUGFIX: increase the default limit on the number of concurrent merges for small parts from 8 to 16. This should help resolving potential issues with heavy data ingestion. See this comment from @lukepalmer .
BUGFIX: MetricsQL: fix panic when incorrect arg is passed as phi into histogram_quantiles function. See this issue.

v1.80.0

Released at 08-08-2022

FEATURE: vmalert: allow configuring additional HTTP request headers for -datasource.url, -remoteWrite.url and -remoteRead.url via -datasource.headers, -remoteWrite.headers and -remoteRead.headers command-line flags. Additional HTTP request headers also can be set on group level via headers param - see these docs and this issue.
FEATURE: MetricsQL: execute left and right sides of certain operations in parallel. For example, q1 or q2, aggr_func(q1) <op> q2, q1 <op> aggr_func(q1). This may improve query performance if VictoriaMetrics has enough free resources for parallel processing of both sides of the operation. See this feature request.
FEATURE: vmauth: allow multiple sections with duplicate username but with different password values at -auth.config file.
FEATURE: add ability to push internal metrics (e.g. metrics exposed at /metrics page) to the configured remote storage from all the VictoriaMetrics components. See these docs.
FEATURE: improve performance for heavy queries over big number of time series on systems with big number of CPU cores. See this issue. Thanks to @zqyzyq for the idea.
FEATURE: improve performance for registering new time series in indexdb by up to 50%. Thanks to @ahfuzhang for the issue.
FEATURE: vmagent: add ability to specify tenantID in target labels. In this case metrics from the given target are routed to the given __tenant_id__. See these docs and this feature request.
FEATURE: vmagent: add service discovery for Yandex Cloud. See these docs and this feature request.
FEATURE: vmui. Zoom in the graph by selecting the needed time range in the same way Grafana does. Hold ctrl (or cmd on MacOS) in order to move the graph to the left/right. Hold ctrl (or cmd on MacOS) and scroll up/down in order to zoom in/out the area under the cursor. See this feature request.
BUGFIX: VictoriaMetrics cluster: fix potential panic in multi-level cluster setup when top-level vmselect is configured with -replicationFactor bigger than 1. See this issue.
BUGFIX: vmagent: properly handle custom endpoint value in ec2_sd_configs. It was ignored since v1.77.0 because of a bug in the implementation of this feature request. See this issue.
BUGFIX: vmagent: add missing __meta_kubernetes_ingress_class_name meta-label for role: ingress service discovery in Kubernetes. See this commit from Prometheus.
BUGFIX: vmagent: allow stale responses from Consul service discovery (aka consul_sd_configs) by default in the same way as Prometheus does. This should reduce load on Consul when discovering big number of targets. Stale responses can be disabled by specifying allow_stale: false option in consul_sd_config. See this issue.
BUGFIX: vmagent: dockerswarm_sd_configs: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels as Prometheus does. See this issue.
BUGFIX: vmagent: set up metric to 0 for partial scrapes in stream parsing mode. Previously the up metric was set to 1 when at least a single metric has been scraped before the error. This aligns the behaviour of vmselect with Prometheus.
BUGFIX: vmagent: restart all the scrape jobs during config reload after global section is changed inside -promscrape.config. See this issue.
BUGFIX: vmagent: properly assume role with AWS ECS credentials. See this issue. Thanks to @transacid for the fix.
BUGFIX: vmagent: do not split regex in relabeling rules into multiple lines if it contains groups. This fixes this issue.
BUGFIX: MetricsQL: return series from q1 if q2 doesn't return matching time series in the query q1 ifnot q2. Previously series from q1 weren't returned in this case.
BUGFIX: vmui: properly show date picker at Table tab. See this issue.
BUGFIX: properly generate http redirects if -http.pathPrefix command-line flag is set. See this issue.