VictoriaMetrics/lib
Zakhar Bessarab b3b29ba6ac
lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk (#5725)
* lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk

Added a custom `http.RoundTripper` implementation which checks for root CA content changes and updates `tls.Config` used by `http.RoundTripper` after detecting CA change.

Client certificate changes are not tracked by this implementation since `tls.Config` already supports passing certificate dynamically by overriding `tls.Config.GetClientCertificate`.

This change implements dynamic reload of root CA only for streaming client used for scraping. Blocking client (`fasthttp.HostClient`) does not support using custom transport so can't use this implementation.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5526

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: update NewRoundTripper API

Update API to allow user to update only parameters required for transport.

Add warning log when reloading Root CA failed.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: fix mutex acquire logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: replace RWMutex with regular mutex to simplify the code

- remove additional mutex used for getRootCABytes - require callee to use mutex
- replace RWMutex with regular mutex

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: refactor

- hold the mutex lock to avoid round tripper being re-created twice
- move recreation logic into separate func to simplify the code

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-04-03 10:01:43 +02:00
..
appmetrics all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page 2023-12-19 03:20:40 +02:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:17:55 +02:00
awsapi lib/awsapi: properly assume role with webIdentity token (#5495) 2023-12-20 19:05:39 +02:00
backup lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
blockcache lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
bloomfilter lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-06-19 22:34:20 -07:00
buildinfo
bytesutil lib/bytesutil: use unsafe.String instead of unsafe conversion of slice header to string header 2024-02-29 17:27:51 +02:00
cgroup lib/cgroup: remove SetGOGC() function 2024-02-05 12:11:08 +02:00
decimal lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
encoding lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 13:26:53 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 15:43:05 -07:00
fastnum lib/fastnum: use unsafe.Slice() instead of deprecated reflect.SliceHeader 2024-02-29 17:17:13 +02:00
fasttime lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
filestream lib/filestream: do not measure read / write duration from / to in-memory buffers 2024-01-23 14:52:22 +02:00
flagutil lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:06 -08:00
fs Revert "app/vmselect: make vmselect resilient to absence of cache folder (#5987)" 2024-03-30 07:29:24 +02:00
htmlcomponents lib/htmlcomponents: use relative links for the top page and for favicon.ico 2023-11-13 20:29:05 +01:00
httpserver all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
httputils lib/httputils: rename CAFile -> caFile in order to be consistent with local var naming in Go 2024-03-17 23:19:52 +02:00
influxutils
ingestserver lib/ingestserver: properly log the number of closed connections 2023-11-14 21:53:24 +01:00
leveledbytebufferpool
logger lib/logger: increase default -loggerMaxArgLen command-line flag value from 500 to 1000 2023-11-14 19:52:27 +01:00
logjson app/vlinsert/jsonline: code prettifying 2023-06-21 19:39:22 -07:00
logstorage all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
lrucache lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
memory all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:12:51 +01:00
mergeset lib/mergeset: use unsafe.Slice and unsafe.String instead of deprecated reflect.SliceHeader with unsafe conversion from slice header to string header 2024-02-29 17:29:33 +02:00
metricsql
netutil all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
persistentqueue all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
procutil all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:12:51 +01:00
promauth lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk (#5725) 2024-04-03 10:01:43 +02:00
prompb lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto 2024-01-14 23:04:45 +02:00
prompbmarshal lib/prompbmarshal: use clear() instead of a loop for clearing tss inside ResetTimeSeries() 2024-03-03 23:40:34 +02:00
promrelabel lib/protoparser/opentelemetry: follow-up after 47892b4a4c 2024-04-03 02:25:29 +03:00
promscrape lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk (#5725) 2024-04-03 10:01:43 +02:00
promutils app/{vmagent,vminsert}: add -streamAggr.dropInputSamples command-line flag for dropping the specified labels from input samples before deduplication and streaming aggregation 2024-03-05 02:15:01 +02:00
protoparser lib/protoparser/opentelemetry: follow-up after 47892b4a4c 2024-04-03 02:25:29 +03:00
proxy lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode 2024-01-30 18:39:10 +02:00
pushmetrics lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-15 13:50:36 +02:00
querytracer lib/querytracer: add missing blank comment line after 3121d76bee 2023-11-15 16:10:43 +01:00
ratelimiter app/vmagent: properly shutdown when -maxIngestionRate limit is reached 2024-03-30 06:43:48 +02:00
regexutil all: upgrade Go builder from Go1.21.7 to Go1.22.0 2024-02-12 21:59:51 +02:00
snapshot lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
storage all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
streamaggr all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
stringsutil lib/stringsutil: add tests for LimitStringLen() function 2023-11-13 10:32:33 +01:00
syncwg
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 20:56:11 -07:00
timerpool
timeutil all: add up to 10% random jitter to the interval between periodic tasks performed by various components 2024-01-22 18:40:32 +02:00
uint64set all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-02 23:16:29 +03:00
workingsetcache lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
writeconcurrencylimiter app/vmagent/remotewrite: clarify the reason behind the default value for -remoteWrite.queues in the same way as the reason for -maxConcurrentInserts is defined at 73f5fb0f0c 2024-03-06 13:43:08 +02:00