HTTP Source

The Vector http source receives logs from HTTP.

Configuration

Common
Advanced

vector.toml
vector.yaml
vector.json

[sources.my_source_id]
  type = "http" # required
  address = "0.0.0.0:80" # required
  encoding = "text" # optional, default

9 items

commonrequired

commonrequiredstring
address
The address to accept connections on. The address must include a port.
- Syntax: literal
- View examples
optionaltable
auth
Options for HTTP Basic Authentication.
- commonrequiredstring
  password
  The basic authentication password.
  - Syntax: literal
  - View examples
- commonrequiredstring
  username
  The basic authentication user name.
  - Syntax: literal
  - View examples
enumcommonoptionalstring
encoding
The expected encoding of received data. Note that for json and ndjson encodings, the fields of the JSON objects are output as separate fields.
- Syntax: literal
- Default: "text"
- Enum, must be one of: "text" "ndjson" "json"
- View examples
optional[string]
headers
A list of HTTP headers to include in the log event. These will override any values included in the JSON payload with conflicting names.
- View examples
optionalstring
path
The URL path on which log event POST requests shall be sent.
- Syntax: literal
- Default: "/"
- View examples
optionalstring
path_key
The event key in which the requested URL path used to send the request will be stored.
- Syntax: literal
- Default: "path"
- View examples
optional[string]
query_parameters
A list of URL query parameters to include in the log event. These will override any values included in the body with conflicting names.
- View examples
optionalbool
strict_path
If set to true, only requests using the exact URL path specified in path will be accepted; otherwise requests sent to a URL path that starts with the value of path will be accepted. With strict_path set to false and path set to "", the configured HTTP source will accept requests from any URL path.
- Default: true
- View examples
optionaltable
tls
Configures the TLS options for incoming connections.
- optionalstring
  ca_file
  Absolute path to an additional CA certificate file, in DER or PEM format (X.509), or an in-line CA certificate in PEM format.
  - Syntax: literal
  - View examples
- optionalstring
  crt_file
  Absolute path to a certificate file used to identify this server, in DER or PEM format (X.509) or PKCS#12, or an in-line certificate in PEM format. If this is set, and is not a PKCS#12 archive, key_file must also be set. This is required if enabled is set to true.
  - Syntax: literal
  - View examples
- optionalbool
  enabled
  Require TLS for incoming connections. If this is set, an identity certificate is also required.
  - Default: false
  - View examples
- optionalstring
  key_file
  Absolute path to a private key file used to identify this server, in DER or PEM format (PKCS#8), or an in-line private key in PEM format.
  - Syntax: literal
  - View examples
- optionalstring
  key_pass
  Pass phrase used to unlock the encrypted key file. This has no effect unless key_file is set.
  - Syntax: literal
  - View examples
- optionalbool
  verify_certificate
  If true, Vector will require a TLS certificate from the connecting host and terminate the connection if the certificate is not valid. If false (the default), Vector will not request a certificate from the client.
  - Default: false
  - View examples

Output

This component outputs log events with the following fields:

{
  "message" : "Hello world",
  "path" : "/",
  "timestamp" : "2020-10-10T17:07:36+00:00"
}

commonrequired*string
message
The raw line line from the incoming payload.
- Syntax: literal
- Only required when: encoding == "text"
- View examples
commonrequiredstring
path
The HTTP path the event was received from. The key can be changed using the path_key configuration setting
- Syntax: literal
- View examples
commonrequiredtimestamp
timestamp
The exact time the event was ingested into Vector.
- View examples

Telemetry

This component provides the following metrics that can be retrieved through the internal_metrics source. See the metrics section in the monitoring page for more info.

counter
events_in_total
The number of events accepted by this component either from tagged origin like file and uri, or cumulatively from other origins. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- container_name - The name of the container from which the event originates.
- file - The file from which the event originates.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
- mode - The connection mode used by the component.
- peer_addr - The IP from which the event originates.
- peer_path - The pathname from which the event originates.
- pod_name - The name of the pod from which the event originates.
- uri - The sanitized uri from which the event originates.
counter
http_bad_requests_total
The total number of HTTP 400 Bad Request errors encountered. This metric includes the following tags:
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
counter
events_out_total
The total number of events emitted by this component. This metric includes the following tags:
- component_kind - The Vector component kind.
- component_name - The Vector component ID.
- component_type - The Vector component type.
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.
counter
parse_errors_total
The total number of errors parsing metrics for this component. This metric includes the following tags:
- instance - The Vector instance identified by host and port.
- job - The name of the job producing Vector metrics.

Examples

text/plain
application/json

Given the following input:

POST / HTTP/1.1
Content-Type: text/plain
User-Agent: my-service/v2.1
X-Forwarded-For: my-host.local

Hello world

And the following configuration:

vector.toml

[sources.http]
type = "http"
address = "0.0.0.0:80"
encoding = "text"
headers = ["User-Agent"]

The following Vector log event will be output:

[
  {
    "host": "my-host.local",
    "message": "Hello world",
    "timestamp": "2020-10-10T17:07:36.452332Z",
    "path": "/",
    "User-Agent": "my-service/v2.1"
  }
]

How It Works

Context

By default, the http source will augment events with helpful context keys as shown in the "Output" section.

Decompression

Received body is decompressed according to Content-Encoding header. Supported algorithms are gzip, deflate, and snappy.

State

This component is stateless, meaning its behavior is consistent across each input.

Transport Layer Security (TLS)

Vector uses Openssl for TLS protocols. You can adjust TLS behavior via the tls.* options.

#Configuration

#address

#auth

#password

#username

#encoding

#headers

#path

#path_key

#query_parameters

#strict_path

#tls

#ca_file

#crt_file

#enabled

#key_file

#key_pass

#verify_certificate

#Output

#message

#path

#timestamp

#Telemetry

#events_in_total

#http_bad_requests_total

#events_out_total

#parse_errors_total

#Examples

#How It Works

#Context

#Decompression

#State

#Transport Layer Security (TLS)

Configuration

address

auth

password

username

encoding

headers

path

path_key

query_parameters

strict_path

tls

ca_file

crt_file

enabled

key_file

key_pass

verify_certificate

Output

message

path

timestamp

Telemetry

events_in_total

http_bad_requests_total

events_out_total