Internal Metrics Source

The Vector internal_metrics source exposes Vector's own internal metrics, allowing you to collect, process, and route Vector's internal metrics just like other metrics.

Requirements

• Vector >= 0.11.0 is required.

Configuration

• Common
• Advanced

• vector.toml
• vector.yaml
• vector.json

[sources.my_source_id]
  type = "internal_metrics" # required

Output

This component outputs the following metric events:

• counter

  api_started_total

  The number of times the Vector GraphQL API has been started. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  config_load_errors_total

  The total number of errors loading the Vector configuration. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_errors_total

  The total number of connection errors for this Vector instance. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_established_total

  The total number of times a connection has been established. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_failed_total

  The total number of times a connection has failed. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_send_errors_total

  The total number of errors sending data via the connection. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_shutdown_total

  The total number of times the connection has been shut down. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  quit_total

  The total number of times the Vector instance has quit. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  recover_errors_total

  The total number of errors caused by Vector failing to recover from a failed reload. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  reload_errors_total

  The total number of errors encountered when reloading Vector. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  reloaded_total

  The total number of times the Vector instance has been reloaded. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  started_total

  The total number of times the Vector instance has been started. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  stopped_total

  The total number of times the Vector instance has been stopped. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  adaptive_concurrency_averaged_rtt

  The average round-trip time (RTT) from the HTTP sink across the current window. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  adaptive_concurrency_in_flight

  The number of outbound requests from the HTTP sink currently awaiting a response. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  adaptive_concurrency_limit

  The concurrency limit that the adaptive concurrency feature has decided on for this current window. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  adaptive_concurrency_observed_rtt

  The observed round-trip time (RTT) for requests from this HTTP sink. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  checkpoint_write_errors_total

  The total number of errors writing checkpoints. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  checkpoints_total

  The total number of files checkpointed. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  checksum_errors_total

  The total number of errors identifying files via checksum. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  collect_completed_total

  The total number of metrics collections completed for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  collect_duration_nanoseconds

  The duration spent collecting of metrics for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  communication_errors_total

  The total number of errors stemming from communication with the Docker daemon. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  connection_read_errors_total

  The total number of errors reading datagram. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode -

• counter

  consumer_offset_updates_failed_total

  The total number of failures to update a Kafka consumer offset. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  container_processed_events_total

  The total number of container events processed. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  container_metadata_fetch_errors_total

  The total number of errors encountered when fetching container metadata. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  containers_unwatched_total

  The total number of times Vector stopped watching for container logs. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  containers_watched_total

  The total number of times Vector started watching for container logs. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  k8s_format_picker_edge_cases_total

  The total number of edge cases encountered while picking format of the Kubernetes log message. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  k8s_docker_format_parse_failures_total

  The total number of failures to parse a message as a JSON object. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  k8s_event_annotation_failures_total

  The total number of failures to annotate Vector events with Kubernetes Pod metadata. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  encode_errors_total

  The total number of errors encountered when encoding an event. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_discarded_total

  The total number of events discarded by this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_failed_total

  The total number of failures to read a Kafka message. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  events_in_total

  The number of events accepted by this component either from tagged origin like file and uri, or cumulatively from other origins. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • container_name - The name of the container from which the event originates.

  • file - The file from which the event originates.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode - The connection mode used by the component.

  • peer_addr - The IP from which the event originates.

  • peer_path - The pathname from which the event originates.

  • pod_name - The name of the pod from which the event originates.

  • uri - The sanitized uri from which the event originates.

• counter

  events_out_total

  The total number of events emitted by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  processed_events_total

  The total number of events processed by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  file_delete_errors_total

  The total number of failures to delete a file. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  file_watch_errors_total

  The total number of errors encountered when watching files. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  files_added_total

  The total number of files Vector has found to watch. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  files_deleted_total

  The total number of files deleted. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  files_resumed_total

  The total number of times Vector has resumed watching a file. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  files_unwatched_total

  The total number of times Vector has stopped watching a file. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  fingerprint_read_errors_total

  The total number of times Vector failed to read a file for fingerprinting. This metric includes the following tags:

  • file - The file that produced the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  glob_errors_total

  The total number of errors encountered when globbing paths. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • path - The path that produced the error.

• counter

  http_bad_requests_total

  The total number of HTTP 400 Bad Request errors encountered. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  http_error_response_total

  The total number of HTTP error responses for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  http_request_errors_total

  The total number of HTTP request errors for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  http_requests_total

  The total number of HTTP requests issued by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  invalid_record_total

  The total number of invalid records that have been discarded. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  invalid_record_bytes_total

  The total number of bytes from invalid records that have been discarded. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  logging_driver_errors_total

  The total number of logging driver errors encountered caused by not using either the jsonfile or journald driver. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• gauge

  memory_used_bytes

  The total memory currently being used by Vector (in bytes). This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  metadata_refresh_failed_total

  The total number of failed efforts to refresh AWS EC2 metadata. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  metadata_refresh_successful_total

  The total number of AWS EC2 metadata refreshes. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• gauge

  open_connections

  The number of current open connections to Vector. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  parse_errors_total

  The total number of errors parsing metrics for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  processed_bytes_total

  The number of bytes processed by the component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • container_name - The name of the container from which the bytes originate.

  • file - The file from which the bytes originate.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode - The connection mode used by the component.

  • peer_addr - The IP from which the bytes originate.

  • peer_path - The pathname from which the bytes originate.

  • pod_name - The name of the pod from which the bytes originate.

  • uri - The sanitized uri from which the bytes originate.

• counter

  processing_errors_total

  The total number of processing errors encountered by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • error_type - The type of the error

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  protobuf_decode_errors_total

  The total number of Protocol Buffers errors thrown during communication between Vector instances. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  request_errors_total

  The total number of requests errors for this component. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• histogram

  request_duration_nanoseconds

  The total request duration in nanoseconds. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  request_read_errors_total

  The total number of request read errors for this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  requests_completed_total

  The total number of requests completed by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  requests_received_total

  The total number of requests received by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  send_errors_total

  The total number of errors sending messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_delete_failed_total

  The total number of failures to delete SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_delete_succeeded_total

  The total number of successful deletions of SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_processing_failed_total

  The total number of failures to process SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_processing_succeeded_total

  The total number of SQS messages successfully processed. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_receive_failed_total

  The total number of failures to receive SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_receive_succeeded_total

  The total number of times successfully receiving SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_message_received_messages_total

  The total number of received SQS messages. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  sqs_s3_event_record_ignored_total

  The total number of times an S3 record in an SQS message was ignored (for an event that was not ObjectCreated). This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • ignore_type - The reason for ignoring the S3 record

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  stale_events_flushed_total

  The number of stale events that Vector has flushed. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  stdin_reads_failed_total

  The total number of errors reading from stdin. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  tag_value_limit_exceeded_total

  The total number of events discarded because the tag has been rejected after hitting the configured value_limit. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  timestamp_parse_errors_total

  The total number of errors encountered parsing RFC 3339 timestamps. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• gauge

  uptime_seconds

  The total number of seconds the Vector instance has been up. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  utf8_convert_errors_total

  The total number of errors converting bytes to a UTF-8 string in UDP mode. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

  • mode - The connection mode used by the component.

• counter

  value_limit_reached_total

  The total number of times new values for a key have been rejected because the value limit has been reached. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_does_not_exist_total

  The total number of errors raised due to the Windows service not existing. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_install_total

  The total number of times the Windows service has been installed. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_restart_total

  The total number of times the Windows service has been restarted. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_start_total

  The total number of times the Windows service has been started. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_stop_total

  The total number of times the Windows service has been stopped. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

• counter

  windows_service_uninstall_total

  The total number of times the Windows service has been uninstalled. This metric includes the following tags:

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

• counter

  events_out_total

  The total number of events emitted by this component. This metric includes the following tags:

  • component_kind - The Vector component kind.

  • component_name - The Vector component ID.

  • component_type - The Vector component type.

  • instance - The Vector instance identified by host and port.

  • job - The name of the job producing Vector metrics.

How It Works

Context

By default, the internal_metrics source will augment events with helpful context keys as shown in the "Output" section.

State

This component is stateless, meaning its behavior is consistent across each input.