Demo Logs
Generate fake log events, which can be useful for testing and demos
Alias
This component was previously called the generator
source. Make sure to update your
Vector configuration to accommodate the name change:
[sources.my_demo_logs_source]
+type = "demo_logs"
-type = "generator"
Configuration
Example configurations
{
"sources": {
"my_source_id": {
"type": "demo_logs",
"format": "apache_common",
"lines": [
"line1"
]
}
}
}
[sources.my_source_id]
type = "demo_logs"
format = "apache_common"
lines = [ "line1" ]
---
sources:
my_source_id:
type: demo_logs
format: apache_common
lines:
- line1
{
"sources": {
"my_source_id": {
"type": "demo_logs",
"count": 9223372036854776000,
"format": "apache_common",
"interval": 1,
"lines": [
"line1"
]
}
}
}
[sources.my_source_id]
type = "demo_logs"
count = 9_223_372_036_854_776_000
format = "apache_common"
interval = 1
lines = [ "line1" ]
---
sources:
my_source_id:
type: demo_logs
count: 9223372036854776000
format: apache_common
interval: 1
lines:
- line1
count
optional uintThe total number of lines to output.
By default, the source continuously prints logs (infinitely).
9.223372036854776e+18
decoding
optional objectdecoding.codec
optional string literal enumOption | Description |
---|---|
bytes | Uses the raw bytes as-is. |
gelf | Decodes the raw bytes as a GELF message. |
json | Decodes the raw bytes as JSON. |
native | Decodes the raw bytes as Vector’s native Protocol Buffers format. This codec is experimental. |
native_json | Decodes the raw bytes as Vector’s native JSON format. This codec is experimental. |
syslog | Decodes the raw bytes as a Syslog message. Decodes either as the RFC 3164-style format (“old” style) or the RFC 5424-style format (“new” style, includes structured data). |
bytes
format
required string literal enumOption | Description |
---|---|
apache_common | Randomly generated logs in Apache common format. |
apache_error | Randomly generated logs in Apache error format. |
bsd_syslog | Randomly generated logs in Syslog format (RFC 3164). |
json | Randomly generated HTTP server logs in JSON format. |
shuffle | Lines are chosen at random from the list specified using lines . |
syslog | Randomly generated logs in Syslog format (RFC 5424). |
framing
optional objectFraming configuration.
Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
framing.character_delimited
required objectmethod = "character_delimited"
framing.character_delimited.delimiter
required uintframing.character_delimited.max_length
optional uintThe maximum length of the byte buffer.
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
framing.method
optional string literal enumOption | Description |
---|---|
bytes | Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). |
character_delimited | Byte frames which are delimited by a chosen character. |
length_delimited | Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. |
newline_delimited | Byte frames which are delimited by a newline character. |
octet_counting | Byte frames according to the octet counting format. |
bytes
framing.newline_delimited
optional objectmethod = "newline_delimited"
framing.newline_delimited.max_length
optional uintThe maximum length of the byte buffer.
This length does not include the trailing delimiter.
By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases.
If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded.
framing.octet_counting
optional objectmethod = "octet_counting"
framing.octet_counting.max_length
optional uintinterval
optional floatThe amount of time, in seconds, to pause between each batch of output lines.
The default is one batch per second. To remove the delay and output batches as quickly as possible, set
interval
to 0.0
.
1
(seconds)sequence
optional booltrue
, each output line starts with an increasing sequence number, beginning with 0.false
format = "shuffle"
Outputs
<component_id>
Output Data
Telemetry
Metrics
linkcomponent_discarded_events_total
countercomponent_id
instead. The value is the same as component_id
.component_errors_total
countercomponent_id
instead. The value is the same as component_id
.component_received_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_events_count
histogramA histogram of the number of events passed in each internal batch in Vector’s internal topology.
Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.
component_id
instead. The value is the same as component_id
.component_received_events_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_events_total
countercomponent_id
instead. The value is the same as component_id
.source_lag_time_seconds
histogramcomponent_id
instead. The value is the same as component_id
.