Metric Event

Description

A Vector metric event represents a numerical operation performed on a time series. Unlike other tools, metrics in Vector are first class citizens, they are not represented as structured logs. This makes them interoperable with various metrics services without the need for any transformation.

Vector's metric data model favors accuracy and correctness over ideological purity. Therefore, Vector's metric types are a conglomeration of various metric types found in the wild, such as Prometheus and Statsd. This ensures metric data is correctly interoperable between systems.

Schema

counter

{
"metric" : {
"counter" : {
"value" : 1
},
"kind" : "absolute",
"name" : "memory_available_bytes",
"namespace" : "host",
"tags" : {
"host" : "my.host.com",
"instance_id" : "abcd1234"
},
"timestamp" : "2020-10-10T17:07:36+00:00"
}
}
  • commonoptionaltable

    counter

    A single value that can only be incremented or reset to zero value, it cannot be decremented.

    • commonrequiredfloat
      value

      The value to increment the counter by. Can only be positive.

  • enumcommonrequiredstring

    kind

    The metric value kind.

    • Syntax: literal
    • Enum, must be one of: "absolute" "incremental"
  • commonrequiredstring

    name

    The metric name.

    • Syntax: literal
  • commonrequiredstring

    namespace

    The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

    • Syntax: literal
  • commonrequiredtable

    tags

    The metric tags. Key/value pairs, nesting is not allowed.

    • commonoptional*
      *

      Key/value pairs, nesting is not allowed.

    • commonrequiredtimestamp

      timestamp

      The metric timestamp; when the metric was created.

    distribution

    {
    "metric" : {
    "distribution" : {
    "samples" : {
    "rate" : 12,
    "value" : null
    },
    "statistic" : "histogram"
    },
    "kind" : "absolute",
    "name" : "memory_available_bytes",
    "namespace" : "host",
    "tags" : {
    "host" : "my.host.com",
    "instance_id" : "abcd1234"
    },
    "timestamp" : "2020-10-10T17:07:36+00:00"
    }
    }
    • commonoptionaltable

      distribution

      A distribution represents a distribution of sampled values. It is used with services that support global histograms and summaries.

      • commonrequired[table]
        samples

        The set of sampled values.

        • commonrequireduint
          rate

          The rate at which this value was sampled.

        • commonrequireduint
          value

          The value being sampled.

          • enumcommonrequiredstring
            statistic

            The statistic to be calculated from the values.

            • Syntax: literal
            • Enum, must be one of: "histogram" "summary"
        • enumcommonrequiredstring

          kind

          The metric value kind.

          • Syntax: literal
          • Enum, must be one of: "absolute" "incremental"
        • commonrequiredstring

          name

          The metric name.

          • Syntax: literal
        • commonrequiredstring

          namespace

          The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

          • Syntax: literal
        • commonrequiredtable

          tags

          The metric tags. Key/value pairs, nesting is not allowed.

          • commonoptional*
            *

            Key/value pairs, nesting is not allowed.

          • commonrequiredtimestamp

            timestamp

            The metric timestamp; when the metric was created.

          gauge

          {
          "metric" : {
          "gauge" : {
          "value" : 1
          },
          "kind" : "absolute",
          "name" : "memory_available_bytes",
          "namespace" : "host",
          "tags" : {
          "host" : "my.host.com",
          "instance_id" : "abcd1234"
          },
          "timestamp" : "2020-10-10T17:07:36+00:00"
          }
          }
          • commonoptionaltable

            gauge

            A gauge represents a point-in-time value that can increase and decrease. Vector's internal gauge type represents changes to that value. Gauges should be used to track fluctuations in values, like current memory or CPU usage.

            • commonrequiredfloat
              value

              A specific point-in-time value for the gauge.

          • enumcommonrequiredstring

            kind

            The metric value kind.

            • Syntax: literal
            • Enum, must be one of: "absolute" "incremental"
          • commonrequiredstring

            name

            The metric name.

            • Syntax: literal
          • commonrequiredstring

            namespace

            The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

            • Syntax: literal
          • commonrequiredtable

            tags

            The metric tags. Key/value pairs, nesting is not allowed.

            • commonoptional*
              *

              Key/value pairs, nesting is not allowed.

            • commonrequiredtimestamp

              timestamp

              The metric timestamp; when the metric was created.

            histogram

            {
            "metric" : {
            "histogram" : {
            "buckets" : {
            "count" : 1,
            "upper_limit" : null
            },
            "count" : 1,
            "sum" : 1
            },
            "kind" : "absolute",
            "name" : "memory_available_bytes",
            "namespace" : "host",
            "tags" : {
            "host" : "my.host.com",
            "instance_id" : "abcd1234"
            },
            "timestamp" : "2020-10-10T17:07:36+00:00"
            }
            }
            • commonoptionaltable

              histogram

              Also called a "timer". A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.

              • commonrequired[table]
                buckets

                The set of buckets containing the histogram values.

                • commonrequireduint
                  count

                  The number of values contained within this bucket.

                • commonrequireduint
                  upper_limit

                  The upper limit of the samples within the bucket.

                  • commonrequireduint
                    count

                    The total number of values contained within the histogram.

                  • commonrequiredfloat
                    sum

                    The sum of all values contained within the histogram.

                • enumcommonrequiredstring

                  kind

                  The metric value kind.

                  • Syntax: literal
                  • Enum, must be one of: "absolute" "incremental"
                • commonrequiredstring

                  name

                  The metric name.

                  • Syntax: literal
                • commonrequiredstring

                  namespace

                  The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

                  • Syntax: literal
                • commonrequiredtable

                  tags

                  The metric tags. Key/value pairs, nesting is not allowed.

                  • commonoptional*
                    *

                    Key/value pairs, nesting is not allowed.

                  • commonrequiredtimestamp

                    timestamp

                    The metric timestamp; when the metric was created.

                  set

                  {
                  "metric" : {
                  "kind" : "absolute",
                  "name" : "memory_available_bytes",
                  "namespace" : "host",
                  "set" : {
                  "values" : [
                  "value1",
                  "value2"
                  ]
                  },
                  "tags" : {
                  "host" : "my.host.com",
                  "instance_id" : "abcd1234"
                  },
                  "timestamp" : "2020-10-10T17:07:36+00:00"
                  }
                  }
                  • enumcommonrequiredstring

                    kind

                    The metric value kind.

                    • Syntax: literal
                    • Enum, must be one of: "absolute" "incremental"
                  • commonrequiredstring

                    name

                    The metric name.

                    • Syntax: literal
                  • commonrequiredstring

                    namespace

                    The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

                    • Syntax: literal
                  • commonoptionaltable

                    set

                    A set represents an array of unique values.

                    • commonrequired[string]
                      values

                      The list of unique values.

                  • commonrequiredtable

                    tags

                    The metric tags. Key/value pairs, nesting is not allowed.

                    • commonoptional*
                      *

                      Key/value pairs, nesting is not allowed.

                    • commonrequiredtimestamp

                      timestamp

                      The metric timestamp; when the metric was created.

                    summary

                    {
                    "metric" : {
                    "kind" : "absolute",
                    "name" : "memory_available_bytes",
                    "namespace" : "host",
                    "summary" : {
                    "count" : 54,
                    "quantiles" : {
                    "upper_limit" : null,
                    "value" : null
                    },
                    "sum" : 1
                    },
                    "tags" : {
                    "host" : "my.host.com",
                    "instance_id" : "abcd1234"
                    },
                    "timestamp" : "2020-10-10T17:07:36+00:00"
                    }
                    }
                    • enumcommonrequiredstring

                      kind

                      The metric value kind.

                      • Syntax: literal
                      • Enum, must be one of: "absolute" "incremental"
                    • commonrequiredstring

                      name

                      The metric name.

                      • Syntax: literal
                    • commonrequiredstring

                      namespace

                      The metric namespace. Depending on the service, this will prepend the name or use native namespacing facilities.

                      • Syntax: literal
                    • commonoptionaltable

                      summary

                      Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.

                      • commonrequireduint
                        count

                        The total number of values contained within the summary.

                      • commonrequired[table]
                        quantiles

                        The set of observations.

                        • commonrequireduint
                          upper_limit

                          The upper limit for this quantile range, where 0 ≤ upper_limit ≤ 1.

                          • commonrequireduint
                            value

                            The value of this quantile range.

                            • commonrequiredfloat
                              sum

                              The sum of all values contained within the histogram.

                          • commonrequiredtable

                            tags

                            The metric tags. Key/value pairs, nesting is not allowed.

                            • commonoptional*
                              *

                              Key/value pairs, nesting is not allowed.

                            • commonrequiredtimestamp

                              timestamp

                              The metric timestamp; when the metric was created.

                            Components

                            View all metric-compatible components

                            How it works

                            Global histograms and distributions

                            Some services, namely Datadog, support distributions. Distributions represent a distribution of sampled values and global statisctics are computed downstream by the service. This strategy enables global histograms avoiding the need to guess and determine buckets beforehand.

                            Dogstatsd

                            Vector supports the Dogstatsd variant of Statsd, enabling the use of distributions through Statsd.

                            Interoperability

                            Metrics are first-class citizens in Vector, they are not represented as structured logs. Therefore, metrics in Vector are fully interoperable betwee disparate metrics services. For example, Vector can ingest metrics from Statsd and output them to Prometheus without any special configuration.

                            Namespaces

                            Metric namespaces in Vector are treated separately from the metric name. This allows you to easily change namespaces at the source and sink level and it ensures Vector can map namespaces properly to different metrics services. Some metrics services require the namespace to be prepended to the metric name, while others requier namespaces to be separate.

                            Tags, labels, & high-cardinality

                            Metrics tags, also known as labels, allows you to augment metrics with context. Tags are key/value pairs that cannot be nested. A caveat with tags is using high-cardinality labels. You should avoid using high-cardinality labels whenever possible. Vector provides a tag_cardinality_limi transform to protect against the accidental use of high-cardinality values.