From f403382ee95aa30cde0e675168130d4adcbb462f Mon Sep 17 00:00:00 2001 From: Vika Date: Wed, 27 May 2020 14:50:28 +0000 Subject: [PATCH] update wiki pages --- CaseStudies.md | 2 +- Cluster-VictoriaMetrics.md | 19 ++++++++++++++----- FAQ.md | 11 +++++++++-- Single-server-VictoriaMetrics.md | 5 ++++- 4 files changed, 28 insertions(+), 9 deletions(-) diff --git a/CaseStudies.md b/CaseStudies.md index e9a8dcb..ac9532e 100644 --- a/CaseStudies.md +++ b/CaseStudies.md @@ -231,7 +231,7 @@ Such a scheme has the following benefits comparing to Prometheus: Cons are the following: -- VictoriaMetrics doesn't support replication - we run extra instance of VictoriaMetrics and Promxy in front of VictoriaMetrics pair for high availability. +- VictoriaMetrics didn't support replication (it [supports replication now](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md#replication-and-data-safety)) - we run extra instance of VictoriaMetrics and Promxy in front of VictoriaMetrics pair for high availability. - VictoriaMetrics stores 1 extra month for defined retention (if retention is set to N months, then VM stores N+1 months of data), but this is still better than other solutions. Some numbers from our single-node VictoriaMetrics setup: diff --git a/Cluster-VictoriaMetrics.md b/Cluster-VictoriaMetrics.md index 8135933..3ee4f65 100644 --- a/Cluster-VictoriaMetrics.md +++ b/Cluster-VictoriaMetrics.md @@ -16,8 +16,9 @@ Join [our Slack](http://slack.victoriametrics.com/) or [contact us](mailto:info@ ## Prominent features - Supports all the features of [single-node version](https://github.com/VictoriaMetrics/VictoriaMetrics). -- Performance and capacity scales horizontally. -- Supports multiple independent namespaces for time series data (aka multi-tenancy). +- Performance and capacity scales horizontally. See [these docs for details](#cluster-resizing-and-scalability). +- Supports multiple independent namespaces for time series data (aka multi-tenancy). See [these docs for details](#multitenancy). +- Supports replication. See [these docs for details](#replication-and-data-safety). ## Architecture overview @@ -203,7 +204,7 @@ or [an alternative dashboard for VictoriaMetrics cluster](https://grafana.com/gr across `vmstorage` nodes. -### Cluster resizing and scalability. +### Cluster resizing and scalability Cluster performance and capacity scales with adding new nodes. @@ -250,6 +251,8 @@ Each instance type - `vminsert`, `vmselect` and `vmstorage` - can run on the mos * The recommended total number of vCPU cores for all the `vminsert` instances can be calculated from the ingestion rate: `vCPUs = ingestion_rate / 150K`. * The recommended number of vCPU cores per each `vminsert` instance should equal to the number of `vmstorage` instances in the cluster. * The amount of RAM per each `vminsert` instance should be 1GB or more. RAM is used as a buffer for spikes in ingestion rate. + The maximum amount of used RAM per `vminsert` node can be tuned with `-memory.allowedPercent` command-line flag. For instance, `-memory.allowedPercent=20` + limits the maximum amount of used RAM to 20% of the available RAM on the host system. * Sometimes `-rpc.disableCompression` command-line flag on `vminsert` instances could increase ingestion capacity at the cost of higher network bandwidth usage between `vminsert` and `vmstorage`. @@ -281,7 +284,7 @@ Upgrade follows `Cluster resizing procedure` under the hood. ### Replication and data safety -VictoriaMetrics offloads replication to the underlying storage pointed by `-storageDataPath`. +By default VictoriaMetrics offloads replication to the underlying storage pointed by `-storageDataPath`. It is recommended storing data on [Google Compute Engine persistent disks](https://cloud.google.com/compute/docs/disks/#pdspecs), since they are protected from data loss and data corruption. They also provide consistently high performance and [may be resized](https://cloud.google.com/compute/docs/disks/add-persistent-disk) without downtime. @@ -289,7 +292,13 @@ HDD-based persistent disks should be enough for the majority of use cases. It is recommended using durable replicated persistent volumes in Kubernetes. -Note that [replication doesn't save from disaster](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883). +If `-replicationFactor=N` command-line flag is passed to `vminsert`, then `vminsert` puts `N` copies of the ingested data to distinct `vmstorage` nodes. +This guarantees that all the data remains available for querying if up to `N-1` `vmstorage` nodes are unavailable. Note that `-dedup.minScrapeInterval=1ms` command-line +flag must be passed to `vmselect` if `-replicationFactor` exceeds 1 in order to de-duplicate replicated data during queries. +It is OK if `-dedup.minScrapeInterval` exceeds 1ms. + +Note that [replication doesn't save from disaster](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883), +so it is recommended performing regular backups. See [these docs](#backups) for details. ### Backups diff --git a/FAQ.md b/FAQ.md index c2f646d..4fa7928 100644 --- a/FAQ.md +++ b/FAQ.md @@ -136,6 +136,8 @@ Yes: * [TSBS benchmark on high-cardinality time series: VictoriaMetrics vs InfluxDB vs TimescaleDB](https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b) * [Standard TSBS benchmark: VictoriaMetrics vs InfluxDB vs TimescaleDB](https://medium.com/@valyala/when-size-matters-benchmarking-victoriametrics-vs-timescale-and-influxdb-6035811952d4) +See also [other articles about VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Articles). + ### What is the pricing for VictoriaMetrics? @@ -145,11 +147,11 @@ The following versions are open source and free: We provide commercial support for both versions. [Contact us](mailto:info@victoriametrics.com) for the pricing. -The following versions are commercial: +The following commercial versions of VictoriaMetrics are planned: * Managed cluster in the Cloud. * SaaS version. -[Contact us](mailto:info@victoriametrics.com) for the pricing. +[Contact us](mailto:info@victoriametrics.com) for more information and for the pricing. ### Why VictoriaMetrics doesn't support [Prometheus remote read API](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cremote_read%3E)? @@ -168,6 +170,11 @@ or via [Prometheus datasource in Grafana](http://docs.grafana.org/features/datas Yes. See [these docs](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/README.md#deduplication) for details. +### Does VictoriaMetrics support replication? + +Yes. See [these docs](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md#replication-and-data-safety) for details. + + ### Where is the source code of VictoriaMetrics? Source code for the following versions is available in the following places: diff --git a/Single-server-VictoriaMetrics.md b/Single-server-VictoriaMetrics.md index bc4cb1a..17181f8 100644 --- a/Single-server-VictoriaMetrics.md +++ b/Single-server-VictoriaMetrics.md @@ -1000,11 +1000,14 @@ for data with timestamps close to the current time. ### Replication -VictoriaMetrics relies on replicated durable persistent storage such as [Google Cloud disks](https://cloud.google.com/compute/docs/disks#pdspecs) +Single-node VictoriaMetrics relies on replicated durable persistent storage such as [Google Cloud disks](https://cloud.google.com/compute/docs/disks#pdspecs) or [Amazon EBS](https://aws.amazon.com/ebs/). It is also recommended making periodic backups, since [replication doesn't save from disaster](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883). See [backup docs](#backups) for details. +Cluster version of VictoriaMetrics supports replication. See [these docs](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md#replication-and-data-safety) +for details. + See also [high availability docs](#high-availability) and [docs about cluster version of VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/cluster/README.md).