Coverage, Location, Detection, and Measurement
The correctness and performance of complex engineered systems is typically impacted by numerous factors, each set to a feasible level from a finite set of admissible levels. Behaviour of such systems may be affected not just by levels of individual factors, but also by interactions among them. Typically, a set $\mathcal S$ of interactions specifies the possible interactions impacting behaviour (for example, all $t$-way interactions for some small $t$). Then combinatorial testing concerns the design and analysis of test suites when behaviour is a function of the impacts of interactions in $\mathcal S$. We examine four tasks: \begin{description} \item[{\sl Coverage:}] Determine whether any interaction in $\mathcal S$ significantly impacts behaviour. \item[{\sl Location:}] Determine which interaction(s) in $\mathcal S$ significantly impact behaviour. \item[{\sl Detection:}] {\em Efficiently} determine which
interaction(s) in $\mathcal S$ significantly impact behaviour. \item[{\sl Measurement:}] Measure the impact on behaviour of every interaction in $\mathcal S$. \end{description} Coverage has been well studied in the software testing literature, while measurement is the concern of the statistical design of experiments.
In this talk, we develop the combinatorial requirements for location and detection, under the assumption that at most $d$ interactions in $\mathcal S$ have a significant impact on behaviour. Then we describe how the resulting locating and detecting arrays relate to covering arrays. Finally we outline a practical use of arrays for location and detection in evaluating TCP throughput, a complex system with 74 factors.
This is joint work with Violet R. Syrotiuk (ASU).