VictoriaMetrics/docs/FAQ.md

FAQ

What is the main purpose of VictoriaMetrics?

To provide the best long-term remote storage solution for Prometheus.

Which features does VictoriaMetrics have?

• Supports Prometheus querying API, so it can be used as Prometheus drop-in replacement in Grafana. Additionally, VictoriaMetrics extends PromQL with opt-in useful features.
• High performance and good scalability for both inserts and selects. Outperforms InfluxDB and TimescaleDB by up to 20x.
• Uses 10x less RAM than InfluxDB when working with millions of unique time series (aka high cardinality).
• High data compression, so up to 70x more data points may be crammed into a limited storage comparing to TimescaleDB.
• Optimized for storage with high-latency IO and low iops (HDD and network storage in AWS, Google Cloud, Microsoft Azure, etc). See graphs from these benchmarks.
• A single-node VictoriaMetrics may substitute moderately sized clusters built with competing solutions such as Thanos, M3DB, Cortex, InfluxDB or TimescaleDB. See vertical scalability benchmarks and comparing Thanos to VictoriaMetrics.
• Easy operation:
  • VictoriaMetrics consists of a single executable without external dependencies.
  • All the configuration is done via explicit command-line flags with reasonable defaults.
  • All the data is stored in a single directory pointed by -storageDataPath flag.
  • Easy backups from instant snapshots.
• Storage is protected from corruption on unclean shutdown (i.e. hardware reset or kill -9) thanks to the storage architecture.
• Supports metrics' ingestion and backfilling via the following protocols:
  • Prometheus remote write API
  • InfluxDB line protocol
  • Graphite plaintext protocol with tags if -graphiteListenAddr is set.
  • OpenTSDB put message if -opentsdbListenAddr is set.
• Ideally works with big amounts of time series data from IoT sensors, connected car sensors and industrial sensors.
• Has open source cluster version.

Which clients do you target?

The following Prometheus users may be interested in VictoriaMetrics:

• Users who don't want to bother with Prometheus' local storage operational burden - backups, replication, capacity planning, scalability, etc.
• Users with multiple Prometheus instances who want performing arbitrary queries over all the metrics collected by their Prometheus instances (aka global querying view).
• Users who want reducing costs for storing huge amounts of time series data.

How to start using VictoriaMetrics?

Start with single-node version. It is easy to configure and operate. It should fit the majority of use cases.

Is it safe to enable remote write storage in Prometheus?

Yes. Prometheus continues writing data to local storage after enabling remote storage write, so all the existing local storage data and new data is available for querying via Prometheus as usual.

How does VictoriaMetrics compare to other clustered TSDBs on top of Prometheus such as M3 from Uber, Thanos, Cortex, etc.?

VictoriaMetrics is simpler, faster, more cost-effective and it provides MetricsQL with useful extensions for PromQL. The simplicity is twofold:

• It is simpler to configure and operate. There is no need in configuring third-party sidecars or fighting with gossip protocol.
• VictoriaMetrics has simpler architecture, which means less bugs and more useful features in the long run comparing to competing TSDBs.

See comparing Thanos to VictoriaMetrics cluster and Remote Write Storage Wars talk from PromCon 2019.

VictoriaMetrics also uses less RAM than Thanos components.

What is the difference between VictoriaMetrics and Cortex?

VictoriaMetrics is similar to Cortex in the following aspects:

• Both systems accept data from Prometheus via standard remote_write API, i.e. there is no need in running sidecars unlike in Thanos case.
• Both systems support multi-tenancy out of the box. See the corresponding docs for VictoriaMetrics.

The main differences between Corex and VictoriaMetrics:

• Cortex re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
• Cortex provides Ruler and Alertmanager components, which are currently missing in VictoriaMetrics. However, these components can be substituted by Promxy.
• Cortex heavily relies on third-party services such as Consul, Memcache, DynamoDB, BigTable, Cassandra, etc. This may increase operational complexity and reduce system reliability comparing to VictoriaMetrics' case, which doesn't use any external services. Compare Cortex Architecture to VictoriaMetrics architecture.
• VictoriaMetrics provides production-ready single-node solution, which is much easier to setup and operate than Cortex cluster.
• Cortex may lose up to 12 hours of recent data on Ingestor failure - see the corresponding docs. VictoriaMetrics may lose only a few seconds of recent data, which isn't synced to persistent storage yet. See this article for details.
• Cortex is usually slower and requires more CPU and RAM than VictoriaMetrics. See this talk from Adidas at PromCon 2019.

What is the difference between VictoriaMetrics and Thanos?

• Thanos re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
• Thanos provides Ruler component, while VictoriaMetrics relies on Promxy for alerting and recording rules.
• VictoriaMetrics accepts data via standard remote_write API for Prometheus, while Thanos uses non-standard Sidecar, which must run alongside each Prometheus instance.
• Thanos Sidecar requires disabling data compaction in Prometheus, which may hurt Prometheus performance and increase RAM usage.
• Thanos stores data on object storage (Amazon S3 or Google GCS), while VictoriaMetrics stores data on block storage (GCP persistent disks, Amazon EBS or bare metal HDD).
• Thanos may lose up to 2 hours of recent data, which wasn't uploaded yet to object storage. VictoriaMetrics may lose only a few seconds of recent data, which isn't synced to persistent storage yet. See this article for details.
• Thanos may be harder to setup and operate comparing to VictoriaMetrics, since it has more moving parts, which can be connected with less reliable networks. See this article for details.
• Thanos is usually slower and requires more CPU and RAM than VictoriaMetrics. See this talk from Adidas at PromCon 2019.

How does VictoriaMetrics compare to InfluxDB?

VictoriaMetrics requires 10x less RAM and it works faster. It is easier to configure and operate. It provides better query language than InfluxQL or Flux.

How does VictoriaMetrics compare to TimescaleDB?

TimescaleDB insists on using SQL as a query language. While SQL is more powerful than PromQL, this power is rarely required during typical TSDB usage. Real-world queries usually look clearer and simpler when written in PromQL than in SQL. Additionally, VictoriaMetrics requires up to 70x less storage space comparing to TimescaleDB for storing the same amount of time series data.

Does VictoriaMetrics use Prometheus technologies like other clustered TSDBs built on top of Prometheus such as M3 from Uber, Thanos, Cortex?

No. VictoriaMetrics core is written in Go from scratch by fasthttp author. The architecture is optimized for storing and querying large amounts of time series data with high cardinality. VictoriaMetrics storage uses certain ideas from ClickHouse. Special thanks to Alexey Milovidov.

Are there performance comparisons with other solutions?

Yes:

• Measuring vertical scalability for time series databases: VictoriaMetrics vs InfluxDB vs TimescaleDB.
• Measuring insert performance on high-cardinality time series: VictoriaMetrics vs InfluxDB
• TSBS benchmark on high-cardinality time series: VictoriaMetrics vs InfluxDB vs TimescaleDB
• Standard TSBS benchmark: VictoriaMetrics vs InfluxDB vs TimescaleDB

What is the pricing for VictoriaMetrics?

The following versions are open source and free:

• Single-node version.
• Cluster version.

We provide commercial support for both versions. Contact us for the pricing.

The following versions are commercial:

• Managed cluster in the Cloud.
• SaaS version.

Contact us for the pricing.

Why VictoriaMetrics doesn't support Prometheus remote read API?

Remote read API requires transferring all the raw data for all the requested metrics over the given time range. For instance, if a query covers 1000 metrics with 10K values each, then the remote read API had to return 1000*10K=10M metric values to Prometheus. This is slow and expensive. Prometheus remote read API isn't intended for querying foreign data aka global query view. See this issue for details.

So just query VictoriaMetrics directly via Prometheus Querying API or via Prometheus datasoruce in Grafana.

Does VictoriaMetrics deduplicate data from Prometheus instances scraping the same targets (aka HA pairs)?

Data from all the Prometheus instances is saved in VictoriaMetrics without deduplication.

The deduplication for Prometheus HA pair may be easily implemented on top of VictoriaMetrics with the following steps:

1. Run multiple VictoriaMetrics instances in multiple availability zones (datacenters).
2. Configure each Prometheus from each HA pair to write data to VictoriaMetrics in distinct availability zone.
3. Put Promxy in front of all the VictoriaMetrics instances.
4. Send queries to Promxy - it will deduplicate data from VictoriaMetrics instances behind it.

Where is the source code of VictoriaMetrics?

Source code for the following versions is available in the following places:

• Single-node version.
• Cluster version.

Does VictoriaMetrics fit for data from IoT sensors and industrial sensors?

VictoriaMetrics is able to handle data from hundreds of millions of IoT sensors and industrial sensors. It supports high cardinality data, perfectly scales up on a single node and scales horizontally to multiple nodes.

Where can I ask questions about VictoriaMetrics?

See VictoriaMetrics-users group.

Where can I file bugs and feature requests regarding VictoriaMetrics?

File bugs and feature requests here.

Are you looking for investors?

Yes. Mail us if you are interested in.