Lua

Modify event data using the Lua programming language

status: stable egress: stream state: stateful
Page source Edit this page
Transform events with a full embedded Lua 5.4 engine.

Warnings

The lua transform is ~60% slower than the remap transform; we recommend that you use the remap transform whenever possible. The lua transform is designed solely for edge cases not covered by the remap transform and not as a go-to option. If the remap transform doesn’t cover your use case, please open an issue and let us know.

Configuration

Example configurations 

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "1"
    }
  }
}
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "1"

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "1"

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "metric_tag_values": "single",
      "search_dirs": [
        "/etc/vector/lua"
      ],
      "source": "function init()\n\tcount = 0\nend\n\nfunction process()\n\tcount = count + 1\nend\n\nfunction timer_handler(emit)\n\temit(make_counter(counter))\n\tcounter = 0\nend\n\nfunction shutdown(emit)\n\temit(make_counter(counter))\nend\n\nfunction make_counter(value)\n\treturn metric = {\n\t\tname = \"event_counter\",\n\t\tkind = \"incremental\",\n\t\ttimestamp = os.date(\"!*t\"),\n\t\tcounter = {\n\t\t\tvalue = value\n\t\t}\n \t}\nend",
      "timers": {
        "handler": "timer_handler",
        "interval_seconds": null
      },
      "version": "1"
    }
  }
}
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
metric_tag_values = "single"
search_dirs = [ "/etc/vector/lua" ]
source = """
function init()
\tcount = 0
end

function process()
\tcount = count + 1
end

function timer_handler(emit)
\temit(make_counter(counter))
\tcounter = 0
end

function shutdown(emit)
\temit(make_counter(counter))
end

function make_counter(value)
\treturn metric = {
\t\tname = "event_counter",
\t\tkind = "incremental",
\t\ttimestamp = os.date("!*t"),
\t\tcounter = {
\t\t\tvalue = value
\t\t}
 \t}
end"""
version = "1"

  [transforms.my_transform_id.timers]
  handler = "timer_handler"

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    metric_tag_values: single
    search_dirs:
      - /etc/vector/lua
    source: |-
      function init()
      	count = 0
      end

      function process()
      	count = count + 1
      end

      function timer_handler(emit)
      	emit(make_counter(counter))
      	counter = 0
      end

      function shutdown(emit)
      	emit(make_counter(counter))
      end

      function make_counter(value)
      	return metric = {
      		name = "event_counter",
      		kind = "incremental",
      		timestamp = os.date("!*t"),
      		counter = {
      			value = value
      		}
       	}
      end      
    timers:
      handler: timer_handler
      interval_seconds: null
    version: "1"

hooks

required object

Lifecycle hooks.

These hooks can be set to perform additional processing during the lifecycle of the transform.

hooks.init

optional string literal

The function called when the first event comes in, before hooks.process is called.

It can produce new events using the emit function.

This can either be inline Lua that defines a closure to use, or the name of the Lua function to call. In both cases, the closure/function takes a single parameter, emit, which is a reference to a function for emitting events.

Examples 
"function (emit)\n\t-- Custom Lua code here\nend"
"init"

hooks.process

required string literal

The function called for each incoming event.

It can produce new events using the emit function.

This can either be inline Lua that defines a closure to use, or the name of the Lua function to call. In both cases, the closure/function takes two parameters. The first parameter, event, is the event being processed, while the second parameter, emit, is a reference to a function for emitting events.

Examples 
"function (event, emit)\n\tevent.log.field = \"value\" -- set value of a field\n\tevent.log.another_field = nil -- remove field\n\tevent.log.first, event.log.second = nil, event.log.first -- rename field\n\t-- Very important! Emit the processed event.\n\temit(event)\nend"
"process"

hooks.shutdown

optional string literal

The function called when the transform is stopped.

It can produce new events using the emit function.

This can either be inline Lua that defines a closure to use, or the name of the Lua function to call. In both cases, the closure/function takes a single parameter, emit, which is a reference to a function for emitting events.

Examples 
"function (emit)\n\t-- Custom Lua code here\nend"
"shutdown"

inputs

required [string]

A list of upstream source or transform IDs.

Wildcards (*) are supported.

See configuration for more info.

Array string literal
Examples 
[
  "my-source-or-transform-id",
  "prefix-*"
]

metric_tag_values

optional string literal enum

When set to single, metric tag values will be exposed as single strings, the same as they were before this config option. Tags with multiple values will show the last assigned value, and null values will be ignored.

When set to full, all metric tags will be exposed as arrays of either string or null values.

Enum options string literal
OptionDescription
fullAll tags will be exposed as arrays of either string or null values.
singleTag values will be exposed as single strings, the same as they were before this config option. Tags with multiple values will show the last assigned value, and null values will be ignored.
default: single

search_dirs

optional [string]

A list of directories to search when loading a Lua file via the require function.

If not specified, the modules are looked up in the configuration directories.

Array string literal
Examples 
[
  "/etc/vector/lua"
]

source

optional string literal

The Lua program to initialize the transform with.

The program can be used to to import external dependencies, as well as define the functions used for the various lifecycle hooks. However, it’s not strictly required, as the lifecycle hooks can be configured directly with inline Lua source for each respective hook.

Examples 
"function init()\n\tcount = 0\nend\n\nfunction process()\n\tcount = count + 1\nend\n\nfunction timer_handler(emit)\n\temit(make_counter(counter))\n\tcounter = 0\nend\n\nfunction shutdown(emit)\n\temit(make_counter(counter))\nend\n\nfunction make_counter(value)\n\treturn metric = {\n\t\tname = \"event_counter\",\n\t\tkind = \"incremental\",\n\t\ttimestamp = os.date(\"!*t\"),\n\t\tcounter = {\n\t\t\tvalue = value\n\t\t}\n \t}\nend"
"-- external file with hooks and timers defined\nrequire('custom_module')"

timers

optional [object]
A list of timers which should be configured and executed periodically.
Array object

version

required string literal enum

Transform API version.

Specifying this version ensures that backward compatibility is not broken.

Examples 
"1"
"2"
Enum options string literal
OptionDescription
1

Lua transform API version 1.

This version is deprecated and will be removed in a future version.

2Lua transform API version 2.

Outputs

<component_id>

Default output stream of the component. Use this component’s ID as an input to downstream transforms and sinks.

Telemetry

Metrics

link

component_received_event_bytes_total

counter
The number of event bytes accepted by this component either from tagged origins like file and uri, or cumulatively from other origins.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_received_events_count

histogram

A histogram of the number of events passed in each internal batch in Vector’s internal topology.

Note that this is separate than sink-level batching. It is mostly useful for low level debugging performance issues in Vector due to small internal batches.

component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_received_events_total

counter
The number of events accepted by this component either from tagged origins like file and uri, or cumulatively from other origins.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

component_sent_event_bytes_total

counter
The total number of event bytes emitted by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
output optional
The specific output of the component.
pid optional
The process ID of the Vector instance.

component_sent_events_total

counter
The total number of events emitted by this component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
output optional
The specific output of the component.
pid optional
The process ID of the Vector instance.

events_in_total

counter
The number of events accepted by this component either from tagged origins like file and uri, or cumulatively from other origins. This metric is deprecated and will be removed in a future version. Use component_received_events_total instead.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
container_name optional
The name of the container from which the data originated.
file optional
The file from which the data originated.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the data originated.
peer_path optional
The pathname from which the data originated.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the data originated.
uri optional
The sanitized URI from which the data originated.

events_out_total

counter
The total number of events emitted by this component. This metric is deprecated and will be removed in a future version. Use component_sent_events_total instead.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
output optional
The specific output of the component.
pid optional
The process ID of the Vector instance.

lua_memory_used_bytes

gauge
The total memory currently being used by the Lua runtime.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.

processed_bytes_total

counter
The number of bytes processed by the component.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
container_name optional
The name of the container from which the bytes originate.
file optional
The file from which the bytes originate.
host optional
The hostname of the system Vector is running on.
mode optional
The connection mode used by the component.
peer_addr optional
The IP from which the bytes originate.
peer_path optional
The pathname from which the bytes originate.
pid optional
The process ID of the Vector instance.
pod_name optional
The name of the pod from which the bytes originate.
uri optional
The sanitized URI from which the bytes originate.

processed_events_total

counter
The total number of events processed by this component. This metric is deprecated in place of using component_received_events_total and component_sent_events_total metrics.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.

processing_errors_total

counter
The total number of processing errors encountered by this component. This metric is deprecated in favor of component_errors_total.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
error_type
The type of the error
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.

utilization

gauge
A ratio from 0 to 1 of the load on a component. A value of 0 would indicate a completely idle component that is simply waiting for input. A value of 1 would indicate a that is never idle. This value is updated every 5 seconds.
component_id
The Vector component ID.
component_kind
The Vector component kind.
component_name
Deprecated, use component_id instead. The value is the same as component_id.
component_type
The Vector component type.
host optional
The hostname of the system Vector is running on.
pid optional
The process ID of the Vector instance.

Examples

Add, rename, and remove log fields

Given this event...
{
  "log": {
    "field_to_remove": "remove me",
    "field_to_rename": "old value"
  }
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"

  [transforms.my_transform_id.hooks]
  process = """
function (event, emit)
\t-- Add root level field
\tevent.log.field = "new value"
\t-- Add nested field
\tevent.log.nested.field = "nested value"
\t-- Rename field
\tevent.log.renamed_field = event.log.field_to_rename
\tevent.log.field_to_rename = nil
\t-- Remove fields
\tevent.log.field_to_remove = nil
\temit(event)
end"""

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      process: |-
        function (event, emit)
        	-- Add root level field
        	event.log.field = "new value"
        	-- Add nested field
        	event.log.nested.field = "nested value"
        	-- Rename field
        	event.log.renamed_field = event.log.field_to_rename
        	event.log.field_to_rename = nil
        	-- Remove fields
        	event.log.field_to_remove = nil
        	emit(event)
        end        

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "process": "function (event, emit)\n\t-- Add root level field\n\tevent.log.field = \"new value\"\n\t-- Add nested field\n\tevent.log.nested.field = \"nested value\"\n\t-- Rename field\n\tevent.log.renamed_field = event.log.field_to_rename\n\tevent.log.field_to_rename = nil\n\t-- Remove fields\n\tevent.log.field_to_remove = nil\n\temit(event)\nend"
      }
    }
  }
}
...this Vector event is produced:
{
  "field": "new value",
  "nested": {
    "field": "nested value"
  },
  "renamed_field": "old value"
}

Add, rename, remove metric tags

Given this event...
{
  "metric": {
    "counter": {
      "value": 2
    },
    "kind": "incremental",
    "name": "logins",
    "tags": {
      "tag_to_remove": "remove me",
      "tag_to_rename": "old value"
    }
  }
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"

  [transforms.my_transform_id.hooks]
  process = """
function (event, emit)
\t-- Add tag
\tevent.metric.tags.tag = "new value"
\t-- Rename tag
\tevent.metric.tags.renamed_tag = event.log.tag_to_rename
\tevent.metric.tags.tag_to_rename = nil
\t-- Remove tag
\tevent.metric.tags.tag_to_remove = nil
\temit(event)
end"""

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      process: |-
        function (event, emit)
        	-- Add tag
        	event.metric.tags.tag = "new value"
        	-- Rename tag
        	event.metric.tags.renamed_tag = event.log.tag_to_rename
        	event.metric.tags.tag_to_rename = nil
        	-- Remove tag
        	event.metric.tags.tag_to_remove = nil
        	emit(event)
        end        

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "process": "function (event, emit)\n\t-- Add tag\n\tevent.metric.tags.tag = \"new value\"\n\t-- Rename tag\n\tevent.metric.tags.renamed_tag = event.log.tag_to_rename\n\tevent.metric.tags.tag_to_rename = nil\n\t-- Remove tag\n\tevent.metric.tags.tag_to_remove = nil\n\temit(event)\nend"
      }
    }
  }
}
...this Vector event is produced:
{
  "counter": {
    "value": 2
  },
  "kind": "incremental",
  "name": "logins",
  "tags": {
    "renamed_tag": "old value",
    "tag": "new value"
  }
}

Drop an event

Given this event...
{
  "log": {
    "field_to_remove": "remove me",
    "field_to_rename": "old value"
  }
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"

  [transforms.my_transform_id.hooks]
  process = """
function (event, emit)
\t-- Drop event entirely by not calling the `emit` function
end"""

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      process: |-
        function (event, emit)
        	-- Drop event entirely by not calling the `emit` function
        end        

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "process": "function (event, emit)\n\t-- Drop event entirely by not calling the `emit` function\nend"
      }
    }
  }
}
...this Vector event is produced:

Iterate over log fields

Given this event...
{
  "log": {
    "value_to_keep": "keep",
    "value_to_remove": "-"
  }
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"

  [transforms.my_transform_id.hooks]
  process = """
function (event, emit)
\t-- Remove all fields where the value is "-"
\tfor f, v in pairs(event) do
\t\tif v == "-" then
\t\t\tevent[f] = nil
\t\tend
\tend
\temit(event)
end"""

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      process: |-
        function (event, emit)
        	-- Remove all fields where the value is "-"
        	for f, v in pairs(event) do
        		if v == "-" then
        			event[f] = nil
        		end
        	end
        	emit(event)
        end        

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "process": "function (event, emit)\n\t-- Remove all fields where the value is \"-\"\n\tfor f, v in pairs(event) do\n\t\tif v == \"-\" then\n\t\t\tevent[f] = nil\n\t\tend\n\tend\n\temit(event)\nend"
      }
    }
  }
}
...this Vector event is produced:
{
  "value_to_keep": "keep"
}

Parse timestamps

Given this event...
{
  "log": {
    "timestamp_string": "2020-04-07 06:26:02.643"
  }
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"
source = """
  timestamp_pattern = "(%d%d%d%d)[-](%d%d)[-](%d%d) (%d%d):(%d%d):(%d%d).?(%d*)"
  function parse_timestamp(str)
\tlocal year, month, day, hour, min, sec, millis = string.match(str, timestamp_pattern)
\tlocal ms = 0
\tif millis and millis ~= "" then
\t\tms = tonumber(millis)
\tend
\treturn {
\t\tyear    = tonumber(year),
\t\tmonth   = tonumber(month),
\t\tday     = tonumber(day),
\t\thour    = tonumber(hour),
\t\tmin     = tonumber(min),
\t\tsec     = tonumber(sec),
\t\tnanosec = ms * 1000000
\t}
  end
  function process(event, emit)
\tevent.log.timestamp = parse_timestamp(event.log.timestamp_string)
\temit(event)
  end"""

  [transforms.my_transform_id.hooks]
  process = "process"

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      process: process
    source: >-2
        timestamp_pattern = "(%d%d%d%d)[-](%d%d)[-](%d%d) (%d%d):(%d%d):(%d%d).?(%d*)"
        function parse_timestamp(str)
      	local year, month, day, hour, min, sec, millis = string.match(str, timestamp_pattern)
      	local ms = 0
      	if millis and millis ~= "" then
      		ms = tonumber(millis)
      	end
      	return {
      		year    = tonumber(year),
      		month   = tonumber(month),
      		day     = tonumber(day),
      		hour    = tonumber(hour),
      		min     = tonumber(min),
      		sec     = tonumber(sec),
      		nanosec = ms * 1000000
      	}
        end
        function process(event, emit)
      	event.log.timestamp = parse_timestamp(event.log.timestamp_string)
      	emit(event)
        end

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "process": "process"
      },
      "source": "  timestamp_pattern = \"(%d%d%d%d)[-](%d%d)[-](%d%d) (%d%d):(%d%d):(%d%d).?(%d*)\"\n  function parse_timestamp(str)\n\tlocal year, month, day, hour, min, sec, millis = string.match(str, timestamp_pattern)\n\tlocal ms = 0\n\tif millis and millis ~= \"\" then\n\t\tms = tonumber(millis)\n\tend\n\treturn {\n\t\tyear    = tonumber(year),\n\t\tmonth   = tonumber(month),\n\t\tday     = tonumber(day),\n\t\thour    = tonumber(hour),\n\t\tmin     = tonumber(min),\n\t\tsec     = tonumber(sec),\n\t\tnanosec = ms * 1000000\n\t}\n  end\n  function process(event, emit)\n\tevent.log.timestamp = parse_timestamp(event.log.timestamp_string)\n\temit(event)\n  end"
    }
  }
}
...this Vector event is produced:
{
  "timestamp": "2020-04-07 06:26:02.643",
  "timestamp_string": "2020-04-07 06:26:02.643"
}

Count the number of logs

Given this event...
{
  "log": {}
}
...and this configuration...
[transforms.my_transform_id]
type = "lua"
inputs = [ "my-source-or-transform-id" ]
version = "2"
source = """
function init()
\tcount = 0
end
function process()
\tcount = count + 1
end
function timer_handler(emit)
\temit(make_counter(count))
\tcount = 0
end
function shutdown(emit)
\temit(make_counter(count))
end
function make_counter(value)
\treturn metric = {
\t\tname = "event_counter",
\t\tkind = "incremental",
\t\ttimestamp = os.date("!*t"),
\t\tcounter = {
\t\t\tvalue = value
\t\t}
\t}
end"""

  [transforms.my_transform_id.hooks]
  init = "init"
  process = "process"
  shutdown = "shutdown"

  [[transforms.my_transform_id.timers]]
  interval_seconds = 5
  handler = "timer_handler"

---
transforms:
  my_transform_id:
    type: lua
    inputs:
      - my-source-or-transform-id
    version: "2"
    hooks:
      init: init
      process: process
      shutdown: shutdown
    timers:
      - interval_seconds: 5
        handler: timer_handler
    source: |-
      function init()
      	count = 0
      end
      function process()
      	count = count + 1
      end
      function timer_handler(emit)
      	emit(make_counter(count))
      	count = 0
      end
      function shutdown(emit)
      	emit(make_counter(count))
      end
      function make_counter(value)
      	return metric = {
      		name = "event_counter",
      		kind = "incremental",
      		timestamp = os.date("!*t"),
      		counter = {
      			value = value
      		}
      	}
      end      

{
  "transforms": {
    "my_transform_id": {
      "type": "lua",
      "inputs": [
        "my-source-or-transform-id"
      ],
      "version": "2",
      "hooks": {
        "init": "init",
        "process": "process",
        "shutdown": "shutdown"
      },
      "timers": [
        {
          "interval_seconds": 5,
          "handler": "timer_handler"
        }
      ],
      "source": "function init()\n\tcount = 0\nend\nfunction process()\n\tcount = count + 1\nend\nfunction timer_handler(emit)\n\temit(make_counter(count))\n\tcount = 0\nend\nfunction shutdown(emit)\n\temit(make_counter(count))\nend\nfunction make_counter(value)\n\treturn metric = {\n\t\tname = \"event_counter\",\n\t\tkind = \"incremental\",\n\t\ttimestamp = os.date(\"!*t\"),\n\t\tcounter = {\n\t\t\tvalue = value\n\t\t}\n\t}\nend"
    }
  }
}
...this Vector event is produced:
{
  "counter": {
    "value": 1
  },
  "kind": "incremental",
  "name": "event_counter",
  "tags": {
    "renamed_tag": "old value",
    "tag": "new value"
  }
}

How it works

Event Data Model

The process hook takes an event as its first argument. Events are represented as tables in Lua and follow Vector’s data model exactly. Please refer to Vector’s data model reference for the event schema. How Vector’s types map to Lua’s type are covered below.

Type Mappings

The correspondence between Vector’s data types and Lua data type is summarized by the following table:

Vector TypeLua TypeComment
Stringstring
Integerinteger
Floatnumber
Booleanboolean
TimestamptableThere is no dedicated timestamp type in Lua. Timestamps are represented as tables using the convention defined by os.date and os.time. The table representation of a timestamp contains the fields year, month, day, hour, min, sec, nanosec, yday, wday, and isdst. If such a table is passed from Lua to Vector, the fields yday, wday, and isdst can be omitted. In addition to the os.time representation, Vector supports sub-second resolution with a nanosec field in the table.
Nullempty stringIn Lua setting the value of a table field to nil means deletion of this field. In addition, the length operator # does not work in the expected way with sequences containing nulls. Because of that Null values are encoded as empty strings.
Maptable
ArraysequenceSequences are a special case of tables. Indexes start from 1, following the Lua convention.

Learning Lua

In order to write non-trivial transforms in Lua, one has to have basic understanding of Lua. Because Lua is an easy to learn language, reading a few first chapters of the official book or consulting the manual would suffice.

Search Directories

Vector provides a search_dirs option that allows you to specify absolute paths that will be searched when using the Lua require function. If this option is not set, the directories of the configuration files will be used instead.

State

This component is stateful, meaning its behavior changes based on previous inputs (events). State is not preserved across restarts, therefore state-dependent behavior will reset between restarts and depend on the inputs (events) received since the most recent restart.
Sidebar