Modality's Types of Functionality

# Introduction

Modality is a powerful tool for experiment automation for embedded systems. The goal of Modality is to give you insights into what is actually happening in your system and to help you find problems you wouldn't think to look for before they crop up in production. To run an experiment, you must be able to change the state of your system and then observe the results of those changes. Thus, Modality's functionality falls into three broad categories: observation, mutation, and experimentation.

A diagram of Modality's process, introducing mutations into your system, observing many conditions, and returning sophisticated analysis

# Observation

Observation means knowing exactly what your system is doing, down to the lowest level. In Modality, this is the ability to collect detailed information from any part of your system, as well as information about the forces acting on the system from outside, and then synthesize all of this information into a trace that you can query. Importantly, Modality does this in such a way that you can see not only that something happened during execution but can query about what was happening beforehand that could have caused it.

Example

You might add instrumentation to periodically report battery voltage. You could then confirm that the battery was always operating within the required parameters, or, if it wasn't, you could see the downstream effects of battery malfunction.

# Mutation

Mutation means altering the state of your system. To enable this, Modality has instrumentation that lets you specify pieces of state that can be changed. Modality records these mutations in the same way as the data it collects from your system, giving you the ability to explore fine-grained questions about your system's reaction to finding itself in a certain state. Mutations are so powerful because they allow you to manipulate your system's state in extremely specific ways, like upgrading to a scalpel from a hand saw. Then, querying your trace for downstream effects of a mutation lets you explore what happened because of it and why.

Example

Keeping with our example above, if you are concerned about your system's sensitivity to battery malfunction you could add a mutator to knock your primary battery offline. This gives you an easy and repeatable way to validate your system's resilience to battery issues.

# Experimentation

Experimentation means learning about your system by putting it into specific states and observing the results. While this can be done manually, Modality's secret sauce is the intelligence it brings to automatically find weak points in your system through the mutation planner. Simply encode the parameters of your experiment in an objective and then Modality does the rest. Rather than having to guess at where problems could arise in your system you can have Modality rapidly and intelligently explore the space of possible mutations, trying out different combinations of system state. It can then give you a report of potential problems to investigate, including which of your system's invariants it was able to violate.

Example

To take our previous examples to the next level we could create an objective targeting mutators in our battery component. We could then have Modality run through hundreds or thousands of possible mutations to get a complete picture of the system's sensitivity to battery behavior.

These three sets of functionality combine to make Modality an extremely powerful and flexible tool. This flexibility means that integrating Modality into your system will provide value throughout the entire product lifecycle, from R&D to production. The same set of instrumentation can explore potential weaknesses in an initial design as can continuously monitor a production system, and everything in between.