Global options reference

Global configuration parameters

acknowledgements

common optional object
Controls how acknowledgements are handled by all sources. These settings may be overridden in individual sources.

acknowledgements.enabled

common optional bool
Controls if sources will wait for destination sinks to deliver the events, or persist them to a disk buffer, before acknowledging receipt. If set to true, all capable sources will have acknowledgements enabled.
default: false

data_dir

optional string literal
The directory used for persisting Vector state, such as on-disk buffers, file checkpoints, and more. Please make sure the Vector project has write permissions to this directory.
Examples
"/var/lib/vector"
"/var/local/lib/vector/"
"/home/user/vector/"
default: /var/lib/vector/

enrichment_tables

optional object

Configuration options for an enrichment table to be used in a remap transform. Currently supported are:

For the lookup in the enrichment tables to be as performant as possible, the data is indexed according to the fields that are used in the search. Note that indices can only be created for fields for which an exact match is used in the condition. For range searches, an index isn’t used and the enrichment table drops back to a sequential scan of the data. A sequential scan shouldn’t impact performance significantly provided that there are only a few possible rows returned by the exact matches in the condition. We don’t recommend using a condition that uses only date range searches.

enrichment_tables.file

required object
Configuration options for the file that provides the enrichment table.
Configuration options for the encoding of the enrichment table’s file.
The delimiter used to separate fields in each row of the CSV file.
Examples
":"
default: ,

Set include_headers to true if the first row of the CSV file contains the headers for each column. This is the default behavior.

If you set it to false, there are no headers and the columns are referred to by their numerical index.

default: true
enrichment_tables.file.encoding.type
common optional string literal
The encoding of the file. Currently, only CSV is supported.
default: csv
enrichment_tables.file.path
required string literal
The path of the enrichment table file. Currently, only CSV files are supported.
Examples
"/data/info.csv"
"./info.csv"
enrichment_tables.file.schema
common optional object

Key/value pairs representing mapped log field names and types. This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.

Timestamp coercions need to be prefaced with timestamp|, for example "timestamp|%F". Timestamp specifiers can use either of the following:

  1. One of the built-in-formats listed in the Timestamp Formats table below.
  2. The time format specifiers from Rust’s chrono library.

Types

  • bool
  • string
  • float
  • integer
  • date
  • timestamp (see the table below for formats)

Timestamp Formats

FormatDescriptionExample
%F %TYYYY-MM-DD HH:MM:SS2020-12-01 02:37:54
%v %TDD-Mmm-YYYY HH:MM:SS01-Dec-2020 02:37:54
%FT%TISO 8601[RFC 3339](https://tools.ietf.org/html/rfc3339) format without time zone2020-12-01T02:37:54
%a, %d %b %Y %TRFC 822/2822 without time zoneTue, 01 Dec 2020 02:37:54
%a %d %b %T %Ydate command output without time zoneTue 01 Dec 02:37:54 2020
%a %b %e %T %Yctime formatTue Dec 1 02:37:54 2020
%sUNIX timestamp1606790274
%FT%TZISO 8601/RFC 3339 UTC2020-12-01T09:37:54Z
%+ISO 8601/RFC 3339 UTC with time zone2020-12-01T02:37:54-07:00
%a %d %b %T %Z %Ydate command output with time zoneTue 01 Dec 02:37:54 PST 2020
%a %d %b %T %z %Ydate command output with numeric time zoneTue 01 Dec 02:37:54 -0700 2020
%a %d %b %T %#z %Ydate command output with numeric time zone (minutes can be missing or present)Tue 01 Dec 02:37:54 -07 2020

Note: the examples in this table are for 54 seconds after 2:37 am on December 1st, 2020 in Pacific Standard Time.

Examples
{
  "duration": "float",
  "status": "int",
  "success": "bool",
  "timestamp_custom": "timestamp|%a %b %e %T %Y",
  "timestamp_iso8601": "timestamp|%F",
  "timestamp_unix": "timestamp|%F %T"
}

Configuration options for MaxMind databases.

The following MaxMind databases are currently supported:

  • GeoLite2-ASN.mmdb (free) — Determine the autonomous system number and organization associated with an IP address.
  • GeoLite2-City.mmdb (free) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
  • GeoIP2-City.mmdb (paid) — Determine the country, subdivisions, city, and postal code associated with IPv4 and IPv6 addresses worldwide.
  • GeoIP2-ISP.mmdb (paid) — Determine the Internet Service Provider (ISP), organization name, and autonomous system organization and number associated with an IP address.

The database file should be in the MaxMind DB file format.

This enrichment table only supports lookup with IP address.

enrichment_tables.geoip.locale
optional string literal
The locale to use to lookup the country name and region name for the city database. See Locations Files
Examples
"de"
"en"
"es"
"fr"
"ja"
"pt-BR"
"ru"
"zh-CN"
default: en
enrichment_tables.geoip.path
required string literal
Path to the MaxMind GeoIP2 or GeoLite2 binary city database file (GeoLite2-City.mmdb). Other databases, such as the country database, are not supported.
Examples
"/path/to/GeoLite2-City.mmdb"
"/path/to/GeoLite2-ISP.mmdb"

expire_metrics

optional object
If set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given time. This value must be positive.

Warning

Deprecated, please use expire_metrics_secs instead.

expire_metrics.nsecs

common optional uint
The fractional number of seconds after which to expire metrics.
Examples
0

expire_metrics.secs

common optional uint
The whole number of seconds after which to expire metrics.
Examples
60

expire_metrics_secs

optional float

If set, Vector will configure the internal metrics system to automatically remove all metrics that have not been updated in the given number of seconds. This value must be positive.

Note that internal counters that are expired but are later updated will have their values reset to zero. Be careful to set this value high enough to avoid expiring critical but infrequently updated internal counters.

Examples
60

healthchecks

optional object
Configures health checks for all sinks.

healthchecks.enabled

common optional bool
Disables all health checks if false, otherwise sink specific option overrides it.
default: true
Exit on startup if any sinks' health check fails. Overridden by --require-healthy command line flag.
default: false

log_schema

optional object
Configures default log schema for all events. This is used by Vector source components to assign the fields on incoming events.

log_schema.host_key

common optional string literal
Sets the event key to use for the event host field.
Examples
"host"
"@host"
default: host

log_schema.message_key

common optional string literal
Sets the event key to use for the event message field.
Examples
"message"
"@message"
default: message

log_schema.metadata_key

common optional string literal
Sets the event key to use for event metadata field (e.g. error or abort annotations in the remap transform).
Examples
"@metadata"
"meta"
default: metadata

log_schema.source_type_key

common optional string literal
Sets the event key to use for the event source type field that is set by some sources.
Examples
"source_type"
"@source_type"
default: source_type

log_schema.timestamp_key

common optional string literal
Sets the event key to use for the event timestamp field.
Examples
"timestamp"
"@timestamp"
default: timestamp

proxy

optional object
Configures an HTTP(S) proxy for Vector to use.

proxy.enabled

optional bool
If false the proxy will be disabled.
default: true

proxy.http

optional string literal
The URL to proxy HTTP requests through.
Examples
"http://foo.bar:3128"

proxy.https

optional string literal
The URL to proxy HTTPS requests through.
Examples
"http://foo.bar:3128"

proxy.no_proxy

optional [string]

A list of hosts to avoid proxying. Allowed patterns here include:

PatternExample match
Domain namesexample.com matches requests to example.com
Wildcard domains.example.com matches requests to example.com and its subdomains
IP addresses127.0.0.1 matches requests to 127.0.0.1
CIDR blocks192.168.0.0./16 matches requests to any IP addresses in this range
Splat* matches all hosts

secret

optional object
Configuration options to retrieve secrets from external backend in order to avoid storing secrets in plaintext in Vector config. Currently, only the exec backend is supported. Multiple backends can be configured. To signify Vector that it should look for a secret to retrieve use the SECRET[<backend_name>.<secret_key>]. This placeholder will then be replaced by the secret retrieved from the relevant backend.

secret.exec

required object

Run a local command to retrieve secrets.

The provided command will be run and provided a list of secrets to fetch, determined from the configuration file, on stdin as JSON in the format:

{"version": "1.0", "secrets": ["secret1", "secret2"]}

The executable is expected to respond with the values of these secrets on stdout, also as JSON, in the format:

{
	"secret1": {"value": "secret_value", "error": null},
	"secret2": {"value": null, "error": "could not fetch the secret"}
}

If an error is returned for any secrets, or if the command exits with a non-zero status code, Vector will log the errors and exit.

Secrets will be loaded when Vector starts or if Vector receives a SIGHUP signal triggering its configuration reload process.

secret.exec.command
required [string]
The command to be run, plus any arguments required.
Examples
[
  "/path/to/get-secret",
  "-s"
]
[
  "/path/to/vault-wrappper"
]
secret.exec.timeout
optional uint
The amount of time Vector will wait for the command to complete.
default: 5 (seconds)

timezone

optional string literal
The name of the time zone to apply to timestamp conversions that do not contain an explicit time zone. The time zone name may be any name in the TZ database, or local to indicate system local time.
Examples
"local"
"America/NewYork"
"EST5EDT"
default: local