Filter Transform

The Vector filter transform filters events based on a set of conditions.

Configuration

• Common
• Advanced

• vector.toml
• vector.yaml
• vector.json

[transforms.my_transform_id]
  type = "filter" # required
  inputs = ["my-source-or-transform-id", "prefix-*"] # required
  condition = '.status_code != 200 && !includes(["info", "debug"], .severity)' # required

• commonrequiredstring

  condition

  The condition to be matched against every input event. Only messages that pass the condition will be forwarded.

  This field accepts a Vector Remap Language (VRL) comparison expression, it must return a boolean.

  Please refer to the Vector Remap Language reference for a list of expressions, functions, and examples.

  • Syntax: remap_boolean_expression
  • View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

• counter

  events_discarded_total

  The total number of events discarded by this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_in_total

  The number of events accepted by this component either from tagged origin like file and uri, or cumulatively from other origins. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • container_name - The name of the container from which the event originates.

  • file - The file from which the event originates.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode - The connection mode used by the component.

  • peer_addr - The IP from which the event originates.

  • peer_path - The pathname from which the event originates.

  • pod_name - The name of the pod from which the event originates.

  • uri - The sanitized uri from which the event originates.

• counter

  processed_events_total

  The total number of events processed by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_out_total

  The total number of events emitted by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  processed_bytes_total

  The number of bytes processed by the component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • container_name - The name of the container from which the bytes originate.

  • file - The file from which the bytes originate.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode - The connection mode used by the component.

  • peer_addr - The IP from which the bytes originate.

  • peer_path - The pathname from which the bytes originate.

  • pod_name - The name of the pod from which the bytes originate.

  • uri - The sanitized uri from which the bytes originate.

Examples

Given the following Vector log event:

[
  {
    "level": "debug",
    "message": "I'm a noisy debug log"
  },
  {
    "level": "info",
    "message": "I'm a normal info log"
  }
]

And the following configuration:

vector.toml

[transforms.filter]
type = "filter"
condition = '.level == "debug"'

The following Vector log event will be output:

[
  {
    "level": "info",
    "message": "I'm a normal info log"
  }
]

How It Works

State

This component is stateless, meaning its behavior is consistent across each input.