added info about search.latencyOffset to key concepts (#4567) (#4584)

Signed-off-by: Alexander Marshalov <_@marshalov.org>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

docs/keyConcepts.md

@@ -742,6 +742,24 @@ useful in the following scenarios:
 
 If you need to export raw samples from VictoriaMetrics, then take a look at [export APIs](https://docs.victoriametrics.com/#how-to-export-time-series).
 
+### Query latency
+
+By default, Victoria Metrics does not immediately return the recently written samples. Instead, it retrieves the last results written prior to the time specified by the `search.latencyOffset` flag, which has a default offset of 30 seconds.
+This is true for both `query` and `query_range` and may give the impression that data is written to the VM with a 30-second delay.
+
+But this flag avoids non-consistent results due to the fact that only part of the values are scraped in the last scrape interval.
+
+Here is an illustration of a potential problem when `search.latencyOffset` is set to zero:
+
+<img src="keyConcepts_withoutLatencyOffset.png">
+
+When this flag is set, the VM will return the last metric value collected before the `search.latencyOffset`
+duration throughout the `search.latencyOffset` duration:
+
+<img src="keyConcepts_withLatencyOffset.png">
+
+It can be overridden on per-query basis via `latency_offset` arg.
+
 ### MetricsQL
 
 VictoriaMetrics provide a special query language for executing read queries - [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).