Deviant Concepts

TIP

Deviant is built on top of Auxon's Modality database and uses many of the same concepts. These are described in detail in the Modality documentation (opens new window).

# Mutations

A mutation is a concrete attempt to change something about the system under test. Mutations come with zero or more parameters that define the exact changes to the system that should be made.

For example, if you wanted to see how your system performs under certain environmental conditions, you could have a mutation which changes the value reported by a temperature sensor. That mutation's parameters could be set to reflect the exact Fahrenheit/Celsius degree value you are interested in.

Mutations are created by users via Deviant, which then pushes the mutation to the appropriate mutator via its mutation-plane protocol.

The main way to create mutations is via Deviant's CLI:

$ deviant mutation create --params 'temp=42' --params 'bar="extra"' \
                          --mutator-id='ed109c30-a5e8-488e-868d-ae2800b4d628'

While mutations can be any interesting change to the system under test, it may be useful to think of them as equivalent to "faults" from a traditional fault-injection testing perspective.

Each mutation is defined in terms of a single, specific, mutator.

# Mutators

A mutator is a well-described mutation actuator. It is responsible for enacting mutations and making them real. Each mutator declares the bounds of mutations that it is capable of injecting.

Every mutator is defined by its metadata attributes, much like modality events (opens new window). These attributes are key-value pairs responsible for describing what kind of change to the system the mutator can cause, as well as where the mutator resides relative to the rest of the system under test.

Attribute keys are strings. Keys have a hierarchical namespace where each segment is separated by the . character. Attribute values can be of a number of types: strings, numbers, etc.

Some example mutator attribute keys and values might be:

  • mutator.name = "message bus delayer"
  • mutator.description = "Introduce delivery delay to some messages passing through the bus"
  • mutator.layer = "operational"
  • mutator.statefulness = "intermittent"
  • mutator.mycorp.build_number = 199
  • mutator.mycorp.deployment.pod.id = 45926
  • mutator.mycorp.safe_for_prod = true

If a mutator has any parameters available for its mutations, the mutator's attributes also describe the expected parameters and their data types.

For example:

  • mutator.params.delay.name = "delay"
  • mutator.params.delay.description = "the amount of time to delay messages"
  • mutator.params.delay.value_type = "Nanoseconds"
  • mutator.params.delay.value_min = 0
  • mutator.params.delay.value_max = 2500
  • mutator.params.delay.default_value = 1000
  • mutator.params.delay.least_effect_value = 0
  • mutator.params.delay.value_distribution.kind = "continuous"
  • mutator.params.delay.value_distribution.scaling = "linear"
  • mutator.params.route.name = "route"
  • mutator.params.route.description = "which bus route to target"
  • mutator.params.route.value_type = "String"
  • mutator.params.route.value_distribution.kind = "discrete"
  • mutator.params.route.value_distribution.option_set.alpha = "a"
  • mutator.params.route.value_distribution.option_set.beta = "b"
  • mutator.params.route.value_distribution.option_set.gamma = "c"

See the tutorials for examples of how to use a premade mutators or host your own custom mutator.

# Mutator Server

Deviant applies mutations by way of requests sent to a mutation server; a long-running process which provides an implementation of the Mutator HTTP API for describing and operating one or more mutators.

Mutator servers can either be directly integrated into your system, for cases where internal system access is required to perform the mutation; or exist standalone, for cases where internal system access is not needed, such as when mutations are performed via external facing API calls.

This API is then used by Auxon's modality-reflector (opens new window) relay process to connect deviant, by way of modalityd, to a given mutator server.

Mutator servers that are integrated within a system can be used by configuring a modality-reflector instance with the mutator server's HTTP API URL, which will query the server for mutators and make them available within Deviant.

Alternatively, mutator servers that are able to operate from outside a system may be structured to operate as a modality-reflector plugin for convenience. Plugins are hosted by the reflector, running as a subprocess and provided with structured configuration.

# Experiments

An experiment is an organizing mechanism for grouping and guiding mutations. Every experiment has a unique name, and may optionally have extra metadata that controls the sorts of mutations suggested within that experiment.

Most experiments will specify a "mutator filter," a logical expression that specifies which mutators shall be used for this experiment. This filter is described using the SpeQTr event predicate (opens new window) syntax:

$ deviant experiment create my-chaos-experiment \
     --mutator-filter '_.group = "foo" AND _.mycorp.safe_for_prod = true'

Having created an experiment, you can ask Deviant to suggest or directly create mutations for it.

$ deviant mutation create --params 'set_to_value=42' \
     --experiment my-chaos-experiment

After your experimentation, you can ask about the experiment's effects on the system under test using Deviant's CLI and characterization JupyterLab notebooks.

← Client →