lib/promscrape: docs update after e4182dd896

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 
 * FEATURE: reduce memory usage when query touches big number of time series.
+* FEATURE: vmagent: reduce memory usage when `kubernetes_sd_config` discovers big number of scrape targets (e.g. hundreds of thouthands) and the majority of these targets (99%)
+  are dropped during relabeling. Previously labels for all the dropped targets were displayed at `/api/v1/targets` page. Now only up to `-promscrape.maxDroppedTargets` such
+  targets are displayed. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878 for details.
 
 
 # [v1.45.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.45.0)

docs/vmagent.md

@@ -231,11 +231,8 @@ This information may be useful for debugging target relabeling.
   by passing `-promscrape.suppressScrapeErrors` command-line flag to `vmagent`. The most recent scrape error per each target can be observed at `http://vmagent-host:8429/targets`
   and `http://vmagent-host:8429/api/v1/targets`.
 
-* For debugging scrapes targets relabeling process, you can observe status of targets at `/api/v1/targets` page.
-  You may need to increase value for the flag `--promscrape.maxDroppedTargets`.  
-  Its needed, when `vmagent` scrapes thousands of targets, and some targets dropped during relabeling.
-  It requires additional memory for storing those targets (up to 10kb per target).
-  With service_discovery services, like kubernetes, it may be around 10 000 dropped scrape targets per 100 active scrape targets.
+* The `/api/v1/targets` page could be useful for debugging relabeling process for scrape targets.
+  This page contains original labels for targets dropped during relabeling (see "droppedTargets" section in the page output). By default up to `-promscrape.maxDroppedTargets` targets are shown here. If your setup drops more targets during relabeling, then increase `-promscrape.maxDroppedTargets` command-line flag value in order to see all the dropped targets. Note that tracking each dropped target requires up to 10Kb of RAM, so big values for `-promscrape.maxDroppedTargets` may result in increased memory usage if big number of scrape targets are dropped during relabeling.
 
 * If `vmagent` scrapes targets with millions of metrics per each target (for instance, when scraping [federation endpoints](https://prometheus.io/docs/prometheus/latest/federation/)),
   then it is recommended enabling `stream parsing mode` in order to reduce memory usage during scraping. This mode may be enabled either globally for all the scrape targets

lib/promscrape/config.go

@@ -32,9 +32,6 @@ var (
 	dryRun = flag.Bool("promscrape.config.dryRun", false, "Checks -promscrape.config file for errors and unsupported fields and then exits. "+
 		"Returns non-zero exit code on parsing errors and emits these errors to stderr. "+
 		"Pass -loggerLevel=ERROR if you don't need to see info messages in the output")
-	maxDroppedTargets = flag.Int("promscrape.maxDroppedTargets", 1000, "Defines how many targets, dropped during service discovery,"+
-		" can be stored for  /api/v1/targets page. "+
-		"This value may be increased for debugging cause of dropping targets during service discovery relabeling")
 )
 
 // Config represents essential parts from Prometheus config defined at https://prometheus.io/docs/prometheus/latest/configuration/configuration/

lib/promscrape/targetstatus.go

@@ -1,6 +1,7 @@
 package promscrape
 
 import (
+	"flag"
 	"fmt"
 	"io"
 	"sort"
@@ -12,6 +13,10 @@ import (
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/promrelabel"
 )
 
+var maxDroppedTargets = flag.Int("promscrape.maxDroppedTargets", 1000, "The maximum number of `droppedTargets` shown at /api/v1/targets page. "+
+	"Increase this value if your setup drops more scrape targets during relabeling and you need investigating labels for all the dropped targets. "+
+	"Note that the increased number of tracked dropped targets may result in increased memory usage")
+
 var tsmGlobal = newTargetStatusMap()
 
 // WriteHumanReadableTargetsStatus writes human-readable status for all the scrape targets to w.