System Under Test (SUT) Definition Files

# Summary

The definition for a System Under Test (SUT) consists of a collection of manifest files. SUT.toml is the manifest for the SUT itself—it contains some identifying information for the SUT and then points to directories containing additional manifest files for each component.

# SUT Manifest File

The SUT manifest is a toml file with the name SUT.toml which contains the following fields:

name
Required
string
The name of the whole System Under Test.
tags
Optional
array
Optional array of 0 or more strings. Each string is an arbitrary piece of metadata or description about the SUT.
component_paths
Optional
array
Optional array of 0 or more strings. Each string must contain a relative file system path, pointing to a component directory containing definition files for a component.

Example SUT Manifest File

# SUT.toml

name = "test-sut"
tags = ["test", "producer", "consumer"]
component_paths = [
    "./consumer-component",
    "./monitor-component",
    "./producer-component"
]

# Component Directories

A SUT is made up of components, logical groupings of functionality. Each component has a set of manifest files which Modality generates based on the instrumentation in your code using the modality-probe manifest-gen command. Generated component manifest files must be together in a directory with the following structure.

<component-name>/
Required
directory
Directory generated with modality-probe manifest-gen and modality-probe mutator-gen.
  • Component.toml
    Required
    file
    The Component Manifest file.
  • events.csv
    Required
    file
    The Events Manifest file.
  • probes.csv
    Required
    file
    The Probes Manifest file.
  • mutators/
    Optional
    directory
    Optional directory that contains 0 or more Mutator Manifest files.
    • <mutator-name>.toml
      Optional
      file
      Mutator Manifest file(s).

A diagram of Modality's system under test (SUT) model, where components of any sort can integrate with Modality for observation and troubleshooting

Example Project with Component Directories, SUT Manifest File, and Modality Configuration File

# /usr/share/doc/modality/examples/c-example
.
├── bin
│   ├── consumer
│   ├── monitor
│   └── producer
├── CMakeLists.txt
├── consumer-component    # Component Directory
│   ├── Component.toml
│   ├── events.csv
│   └── probes.csv
├── include
│   └── common.h
├── Modality.toml         # Modality Configuration File
├── monitor-component     # Component Directory
│   ├── Component.toml
│   ├── events.csv
│   └── probes.csv
├── objectives
│   └── example.toml
├── producer-component    # Component Directory With Mutators
│   ├── Component.toml
│   ├── events.csv
│   ├── mutators
│   │   └── sample_data_mutator.toml
│   └── probes.csv
├── scripts
│   ├── ...
├── src
│   ├── common.c
│   ├── consumer
│   │   └── main.c
│   ├── monitor
│   │   └── main.c
│   └── producer
│       └── main.c
└── SUT.toml              # SUT Manifest File