LTTng Collector Plugin

This plugin implements a collector for LTTng live (opens new window) data streamed from an LTTng relay daemon using babeltrace (opens new window). The format of the data is Common Trace Format (opens new window) data, and follows the same conventions as the CTF importer.

The source code can be found in the plugin's repository (opens new window).

# Getting Started

The following instructions walk through a basic setup for collecting kernel-level LTTng trace data. Note that these events don't come from any one process, they are produced by the kernel itself and can therefore concern any process running on the system. For more help setting up LTTng, including for application-level tracing, see the getting started guide (opens new window) and further documentation.

  1. Install LTTng packages.
sudo apt-get install lttng-tools lttng-modules-dkms
  1. Run an LTTng relayd instance. More information about relayd can be found here. (opens new window)
  1. Create an LTTng tracing session in live mode pointed at the LTTng relayd instance.
sudo lttng create linux --live --set-url net://
  1. Create and enable LTTng recording event rules. This example enables some generally useful events: failing IO system calls, process fork and exec, netdev state changes, and ext4 errors.
# Failing IO system calls
sudo lttng enable-event --kernel --syscall open,write,read,close --filter '_ret != 0'

# Process fork and exec static tracepoints
sudo lttng enable-event --kernel sched_process_fork,sched_process_exec

# netdev state changes and ext4 error function calls via Linux kprobes
# NOTE: requires kprobes support in the kernel (CONFIG_KPROBES=y)
sudo lttng enable-event --kernel --probe netdev_state_change netdev_state_change
sudo lttng enable-event --kernel --probe __ext4_error+0x0 ext4_error
  1. Start the LTTng recording session.
sudo lttng start
  1. Set the plugin configuration to point to the running relayd instance.
# modality-reflector config

url = 'net://'
  1. Run the Modality collector plugin to start collecting data from the LTTng relayd instance.
modality-reflector run --collector lttng-live --config <your-config-file>

# Concepts

The following describes the default mapping between CTF (opens new window) concepts and Modality's concepts. See the configuration section for ways to change the default behavior.

  • CTF streams are represented as separate Modality timelines
  • CTF trace and stream properties are represented as Modality timeline attributes under the timeline.internal.ctf prefix
  • CTF event common, specific, and packet context fields are represented as Modality event attributes under the timeline.internal.ctf prefix
  • CTF event field attributes are at the root level

# Configuration Fields

Configuration fields should be placed in a [plugins.ingest.collectors.lttng-live.metadata] table in a modality-reflector configuration file. All fields are optional.

Use the provided UUID as the run ID instead of generating a random one.
Provide a trace UUID for this data import. This UUID will override a UUID contained in the CTF metadata, if any.
This is useful for constructing deterministic trace UUIDs which form the timeline IDs.
Logging level for libbabeltrace itself, useful for diagnosing import issues. Must be one of trace, debug, info, warning, error, fatal, none. Defaults to none. See the libbabeltrace documentation (opens new window) for more info.
Set the duration of a retry if a component requires it, in microseconds. Corresponds to the retry-duration (opens new window) parameter in babeltrace.
What to do when the message iterator does not find the remote tracing session. Must be one of [continue, fail, end]. Corresponds to the session-not-found-action (opens new window) parameter in babeltrace.
URL to connect to the LTTng relay daemon. Corresponds to the inputs (opens new window) parameter in babeltrace.

# Example Configuration

# config.toml for modality-reflector

log-level = 'info'
retry-duration_us = 10
url = 'net://'