Commit Graph

120 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
233e5f0a9e
lib/httpserver: skip basic auth check for additional request paths, which should call httpserver.CheckAuthFlag()
This is a follow-up for 61dce6f2a1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329
2024-07-16 01:00:45 +02:00
Aliaksandr Valialkin
87f1c8bd6c
lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
Aliaksandr Valialkin
f4b1cbfef0
all: replace old https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html url with the new one - https://docs.victoriametrics.com/cluster-victoriametrics/ 2024-04-18 02:54:20 +02:00
Alexander Marshalov
806c07ddd5
vmsingle/vmselect returns http status 429 (TooManyRequests) instead of 503 (ServiceUnavailable) when max concurrent requests limit is reached. (#5682) 2024-01-24 17:55:06 +01:00
Yury Molodov
eb6def0695
vmui: add flag for default timezone setting (#5611)
* vmui: add flag for default timezone setting #5375

* vmui: validate timezone before client return

* Update app/vmselect/vmui.go

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-23 04:11:19 +02:00
Aliaksandr Valialkin
1f105dde98
all: allow dynamically reading *AuthKey flag values from files and urls
Examples:

1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath
2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath
3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url

The flag value is automatically updated when the file contents changes.
2024-01-21 22:03:38 +02:00
Alexander Marshalov
02a0a7f428
added field version to the response for /api/v1/status/buildinfo API for using more efficient API in Grafana for receiving label values, added additional info about setup Grafana datasource (#5370) (#5437) 2023-12-07 16:37:36 +02:00
Aliaksandr Valialkin
f62e03b3d2
app/vmselect: do not limit concurrency for static and fast queries
Previously concurrency for static and fast queries was limited with the -search.maxConcurrentRequests
command-line flag. This could complicate identifying heavy queries via `vmui` at `Top queries` and `Active queries` pages,
since `vmui` and these pages couldn't be opened on overloaded vmselect.

Thanks to @f41gh7 for the idea.
2023-12-01 17:25:01 +02:00
Aliaksandr Valialkin
aae06e003e
app/vmselect: simplify code a bit after 63e0f16062
Use only a single call to prometheus.WriteErrorResponse() inside sendPrometheusError
2023-11-16 18:14:33 +01:00
Aliaksandr Valialkin
31f7ef0811
app/{vmselect,vlselect}: enable caching of static contents from /vmui/static/ folder at client side
This should improve repated VMUI page load times on slow networks

See https://developer.chrome.com/docs/lighthouse/performance/uses-long-cache-ttl/
2023-10-12 09:33:40 +02:00
Aliaksandr Valialkin
edee262ecc
Makefile: update golangci-lint from v1.51.2 to v1.54.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.54.2
2023-09-01 10:16:42 +02:00
Tamara Vashchuk
7349f18c55
vmui: Add button to prettify query (#4694)
* Add button to prettify query

Just capitalizes query text for now

* Add /prettify-query API handler

* Replace UI pretiffier using prettifier API

* Add showing server errors

Had to pass setQueryErrors from useFetchQuery.ts

* Use serverUrl from global AppState

* Change icon to AutoAwsome icon + added style change color when button is active

* Add sync/await to prettifyQuery function

* Doc public function for lint

* Minor async fix

* Removed extra blank lines

* Extract usePrettifyQuery hook

* Made more generic style for :active button

* Refactor usePrettifyQuery

However, prettify errors don't clean up query errors, but should

* Add prettyQuery functionality to CHANGELOG.md

* Reuse queryErrors

* Unhide errors on start

---------

Co-authored-by: Tamara <toma.vashchuk@gmail.com>
2023-08-18 20:12:48 +03:00
Aliaksandr Valialkin
c5f94fa5fc
app/vmselect: rename promql.WriteActiveQueries() to promql.ActiveQueriesHandler()
This makes it more consistent with the rest of handlers inside app/vmselect/main.go

This is a follow-up for 6a96fd8ed5

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4598
2023-07-20 11:32:33 -07:00
Aliaksandr Valialkin
8a91eb25c4
app/vmselect: follow-up after 6a96fd8ed5
- Add `Active queries` chapter to VMUI docs
- Set `Content-Type: json` header inside promql.WriteActiveQueries() handler,
  in order to be consistent with other request handlers called at app/vmselect/main.go
- Pass the request to promql.WriteActiveQueries() handler, so it can change its output
  depending on the provided request params. This also improves consistency of
  promql.WriteActiveQueries() args with other request hanlers at app/vmselect/main.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4653
2023-07-19 16:26:03 -07:00
Yury Molodov
6a96fd8ed5
vmui: add Active Queries page (#4653)
* feat: add page to display a list of active queries (#4598)

* app/vmagent: code formatting

* fix: remove console

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-07-19 15:47:21 -07:00
Aliaksandr Valialkin
8be52ef217
app/vlselect: handle vmui at /select/vmui path instead of /vmui
This simplifies routing at auth proxies such as vmauth to vlselect component,
which serves VMUI - just route all the requests, which start with /select/, to vlselect.
2023-06-21 19:52:50 -07:00
Aliaksandr Valialkin
78eaa056c0
app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils
While at it, move app/vmselect/bufferedwriter to lib/bufferedwriter, since it is going to be used in VictoriaLogs
2023-06-19 22:34:20 -07:00
Aliaksandr Valialkin
2b53ff774b
app/vmselect: log locations of sendPrometheusError() calls
Previously the location inside the sendPrometheusError() was logged.
This could make hard investigating error locations via `vm_log_messages_total` metric.
2023-05-18 20:39:53 -07:00
Aliaksandr Valialkin
ec3943d14a
app/vmselect: small cleanup after 4f3f9950d0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807
2023-05-08 14:57:11 -07:00
Aliaksandr Valialkin
1db9b78b88
app/vmselect: small cleanup after 68e31a6000
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3811
2023-05-08 14:34:37 -07:00
Roman Khavronenko
baf456978d
vmselect: exit early from queue on context cancel (#4223)
* vmselect: exit early from queue on context cancel

When `-search.maxConcurrentRequests` is reached, vmselect puts
request in the queue. It is expected, that requests in the queue
will be processed as soon as it would be enough capacity to do so.

However, it could happen that while request was waiting its turn,
the client could have already cancel it (close the connection,
or just close the tab with UI). In this case, we should de-queue
such requests to avoid spending extra resources on them.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: address review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-03 10:42:17 +02:00
Yury Molodov
4f3f9950d0
vmui: add metric relabel debug (#3889)
* feat: add metric relabel debug (#3807)

* fix: add link to relabeling cookbook

* lib/promrelabel: merge, fix conflicts

* lib/promrelabel: fix diff

* docs/vmui: add metric relabel playground

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-04-26 11:53:29 +03:00
Yury Molodov
68e31a6000
vmui: Integrate WITH template playground (#3831)
* feat: add WithTemplate page

* app/vmselect/prometheus: enable json mode for expand with expr API

* app/vmselect/prometheus: enable CORS and add content type

* feat: add api for expand with templates

* fix: remove console from useExpandWithExprs

* app/vmselect/prometheus: fix escaping

* vmui:  integrate WITH template

* app/vmctl: check content type instead of form param

* fix: add content-type for fetch with-exprs

* fix: add a header to the server's response that allows the "Content-Type" header

* app/vmctl: added comment and cleanup

* app/vmctl: use format query param

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-04-25 11:40:01 +03:00
Aliaksandr Valialkin
ffdf430be0
app/vmselect/graphite: open source Graphite Render API 2023-03-31 23:25:04 -07:00
Aliaksandr Valialkin
b285207aa7
app/vmselect: add -search.logQueryMemoryUsage command-line flag for logging queries, which take big amounts of memory
Thanks to @michal-kralik for initial attempts for this feature:

- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3651
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3715

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3553
2023-02-23 18:47:08 -08:00
Aliaksandr Valialkin
9f5b5708ff
app/vmselect: handle the /custom-dashboards request from /graph/ page in the same way as from the /vmui/ page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322
2023-01-11 23:41:04 -08:00
Aliaksandr Valialkin
a194982117
app/vmselect: follow-up after 820312a2b1
- Move the feature description at the correct place at docs/CHANGELOG.md
- Run `make vmui-update`
- Various cosmetic fixes

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322
2023-01-11 23:28:00 -08:00
Dmytro Kozlov
820312a2b1
app/vmui: define custom path for dashboards json file (#3545)
* app/vmui: define custom path for dashboards json file

* app/vmui: remove unneeded code

* app/vmui: move handler to own file, fix show dashboards,

* app/vmui: move flag to handler, add flag description

* app/vmauth: fix part of the comments

* feat: add store for dashboards

* fix: prevent fetch dashboards for app mode

* app/vmauth: use simple cache for predefined dashboards

* app/vmauth: update dashboards doc

* app/vmauth: fix ci

* app/vmui: decrease timeout

* app/vmselect: removed cache, fix comments

* app/vmselect: remove unused const

* app/vmselect: fix error log, use slice byte instead of struct

Co-authored-by: Yury Moladau <yurymolodov@gmail.com>
2023-01-11 23:06:07 -08:00
Aliaksandr Valialkin
dff47c73b7
app/vmselect: improve logging when the incoming query cannot be executed because of timeout in the wait queue 2023-01-11 01:06:05 -08:00
Aliaksandr Valialkin
95ce1ba6ce
lib/httpserver: directly pass flag value to CheckAuthFlag()
There is no sense in passing a pointer to flag value there.

This is a follow-up for 4225a0bd75
2023-01-10 15:52:23 -08:00
Zakhar Bessarab
4225a0bd75
Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags (#3582)
* {lib/server, app/}: use `httpAuth.*` flag as fallback for `*AuthKey` if it is not set

* lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.*` flags

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-10 15:46:13 -08:00
Aliaksandr Valialkin
af263fe881
all: small improvements in error messages and command-line flag descriptions related to concurrency limiters 2023-01-07 00:11:44 -08:00
Aliaksandr Valialkin
c63755c316
lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit
Previously the -maxConcurrentInserts was limiting the number of established client connections,
which write data to VictoriaMetrics. Some of these connections could be idle.
Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting
the number of such connections. So now the -maxConcurrentInserts command-line option
limits the number of concurrently executed insert requests, not including idle connections.

It is recommended removing -maxConcurrentInserts command-line option, since the default value
for this option should work good for most cases.
2023-01-06 22:20:19 -08:00
Aliaksandr Valialkin
e7637885a6
app/vmselect: improve error message when the request cannot be started because too many concurrent requests are already executed 2023-01-06 22:10:42 -08:00
Aliaksandr Valialkin
38341802c2
app/vmselect: add /expand-with-exprs page 2022-12-14 16:18:55 -08:00
Aliaksandr Valialkin
a30ae502ef
lib/promscrape: allow editing relabeling configs and labels at /target-relabel-debug page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2022-12-10 12:44:45 -08:00
Aliaksandr Valialkin
3e7276639e
app/{vminsert,vmselect}: move the handler for /metric-relabel-debug from vminsert to vmselect to be consistent with the cluster version 2022-12-10 02:45:40 -08:00
Aliaksandr Valialkin
758e8a15fd
app/vmselect: typo fixes in code comments 2022-12-06 20:58:16 -08:00
Aliaksandr Valialkin
04a05f161c
app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set
This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-10 21:45:13 +03:00
Aliaksandr Valialkin
8e1ccecd97
docs: mention -search.maxMemoryPerQuery in the description to -search.maxConcurrentQueries command-line flag
This is a follow-up for 5138eaeea0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-09 13:56:59 +03:00
Aliaksandr Valialkin
5138eaeea0
app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-08 01:08:05 +03:00
Aliaksandr Valialkin
725dfb0ed6
lib/httpserver: use 302 redirects instead of 301 redirects
Incorrect 301 redirects can be cached by user agents such as web browsers.
This can complicate recovery procedure after the incorrect redirect is fixed,
e.g. web browser cache must be reset.

The related issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752
2022-10-01 16:53:35 +03:00
Aliaksandr Valialkin
4ac79d29ad
app/vmselect: follow-up after 63e0f16062
* Explicitly store a pointer to UserReadableError in the error interface.
  Previously Go automatically converted the value to a pointer before storing in the error interface.

* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
  which calls errors.Is() and errors.As().

* Document the change in docs/CHANGELOG.md
2022-08-15 13:50:16 +03:00
Roman Khavronenko
63e0f16062
vmselect: introduce UserReadableError type of error (#2894)
When read query fails, VM returns rich error message with
all the details. While these details might be useful
for debugging specific cases, they're usually too verbose
for users.
Introducing a new error type `UserReadableError` is supposed
to allow to return to user only the most important parts
of the error trace. This supposed to improve error readability
in web interfaces such as VMUI or Grafana.

The full error trace is still logged with the full context
and can be found in vmselect logs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:38:47 +03:00
Aliaksandr Valialkin
6b0550c023
app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918
2022-08-02 12:59:07 +03:00
Aliaksandr Valialkin
ce68e76d62
app/vmselect: follow-up after 8667307d73 2022-07-11 20:14:34 +03:00
Roman Khavronenko
8667307d73
vmselect: cover special cases for vmalert's routing in single-node version (#2845)
* vmselect: cover special cases for vmalert's routing in single-node version

* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-11 19:52:22 +03:00
Aliaksandr Valialkin
17de8a41c2
all: follow-up after ed89106274 2022-07-06 12:44:46 +03:00
Roman Khavronenko
ed89106274
vmselect: allow proxying requests to vmalert from single-node (#2834)
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-06 10:47:26 +02:00
Aliaksandr Valialkin
7bf75c7e61
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total 2022-06-22 13:15:31 +03:00