Core Concepts

# Three Pillars of Functionality

Modality is a powerful set of tools for discovering and removing risk in embedded systems. The goal of Modality is to give you insights into what is actually happening in your system and to help you find problems you wouldn't think to look for before they crop up in production. Modality's functionality falls into three broad categories: observation, mutation, and experimentation.

A diagram of Modality's process, introducing mutations into your system, observing many conditions, and returning sophisticated analysis

# Observation

Observation means knowing exactly what your system is doing, down to the lowest level. In Modality, this is the ability to collect detailed information from any part of your system, as well as information about the forces acting on the system from outside, and then synthesize all of this information into a trace that you can query. Importantly, Modality does this in such a way that you can see not only that something happened during execution but can query about what was happening beforehand that could have caused it. When something goes wrong, Modality can analyze your traces and suggest likely culprits, kick-starting your investigations.

Example

You might add instrumentation to periodically report battery voltage. You could then confirm that the battery was always operating within the required parameters, or, if it wasn't, you could see the downstream effects of battery malfunction.

# Mutation

Mutation means altering the state of your system. To enable this, Modality has instrumentation that lets you specify pieces of state that can be changed. Modality records these mutations in the same way as the data it collects from your system, giving you the ability to explore fine-grained questions about your system's reaction to finding itself in a certain state. Mutations are so powerful because they allow you to manipulate your system's state in extremely specific ways, like upgrading to a scalpel from a hand saw. Then, querying your trace for downstream effects of a mutation lets you explore what happened because of it and why.

Example

Keeping with our example above, if you are concerned about your system's sensitivity to battery malfunction you could add a mutator to knock your primary battery offline. This gives you an easy and repeatable way to validate your system's resilience to battery issues.

# Experimentation

Experimentation means learning about your system by putting it into specific states and observing the results. While this can be done manually, Modality's secret sauce is the intelligence it brings to automatically find weak points in your system through the mutation planner. Simply encode the parameters of your experiment in an objective and then Modality does the rest. Rather than having to guess at where problems could arise in your system you can have Modality rapidly and intelligently explore the space of possible mutations, trying out different combinations of system state. It can then give you a report of potential problems to investigate, including which of your system's invariants it was able to violate.

Example

To take our previous examples to the next level we could create an objective targeting mutators in our battery component. We could then have Modality run through hundreds or thousands of possible mutations to get a complete picture of the system's sensitivity to battery behavior.

These three sets of functionality combine to make Modality an extremely powerful and flexible tool. This flexibility means that integrating Modality into your system will provide value throughout the entire product lifecycle, from R&D to production. The same set of instrumentation can explore potential weaknesses in an initial design as can continuously monitor a production system, and everything in between.

# Anatomy of a System Using Modality

Under Modality the world is split into two main pieces: the stimulus and the SUT, or System Under Test. Depending on what you want to verify, the SUT could be the actual system you're developing, a simulator, or your system along with its test harness, to name a few. The stimulus is an umbrella term for everything that interacts with the SUT from the outside. Some examples of things that could be included in the stimulus are your test harness, manual interaction with the SUT (e.g. pushing a button), or queueing a mutation through Modality to change the state of the SUT.

A diagram of Modality's system under test (SUT) model, where components of any sort can integrate with Modality for observation and troubleshooting

The SUT is made up of components, logical groupings of functionality which you define. To use Modality you instrument your components with probes, which are the units that allow Modality to collect data, track communication between components, and make targeted alterations to the state of the SUT.

A diagram of Modality's process, introducing mutations into your system, observing many conditions, and returning sophisticated analysis

Data is collected in the form of events, which indicate that something happened and optionally carry a payload with some information about the system. Expectations are a special type of event that you can use to call attention to meaningful state of the SUT by specifying a pass/fail condition. To take advantage of Modality's full functionality you can create mutators in your components, which specify what piece(s) of state Modality can change.

A diagram of Modality's scope capabilities, defining areas of interest in the complex flow of system events. Modality scope is observing the trace of all events, including several Modality mutations.

Modality collects all the data recorded in events and expectations, along with all mutations and whatever actions you record from the stimulus, and synthesizes them into a trace which gives a holistic picture of everything that happened in your system.

You can then use the SpeQTr Language to write test conditions or validate system requirements against any trace of recorded execution. When you encounter a problem, Modality can automatically analyze what your system was doing leading up to the problem and suggest likely causes. If you instrument your system with mutators you can perform precise fault-injection testing, or have Modality generate hundreds or thousands of test cases for you, helping you find problems you wouldn't otherwise know how to look for.