Oracles for Distributed Testing

• Published in: IEEE transactions on software engineering Vol. 38; no. 3; pp. 629 - 641
• Main Author: Hierons, R. M.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-05-2012; IEEE Computer Society
• Subjects: Algorithms; Controllability; distributed systems; finite state machine; Interfaces; local observability; Mathematical models; Mathematical problems; nondeterminism; Observability; Polynomials; Software; Software engineering; Software engineering/software/program verification; software engineering/testing and debugging; Studies; systems and software; test oracle; Testing
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2011.45

The problem of deciding whether an observed behavior is acceptable is the oracle problem. When testing from a finite state machine (FSM), it is easy to solve the oracle problem and so it has received relatively little attention for FSMs. However, if the system under test has physically distributed interfaces, called ports, then in distributed testing, we observe a local trace at each port and we compare the set of local traces with the set of allowed behaviors (global traces). This paper investigates the oracle problem for deterministic and nondeterministic FSMs and for two alternative definitions of conformance for distributed testing. We show that the oracle problem can be solved in polynomial time for the weaker notion of conformance (⊆ w ) but is NP-hard for the stronger notion of conformance (⊆), even if the FSM is deterministic. However, when testing from a deterministic FSM with controllable input sequences, the oracle problem can be solved in polynomial time and similar results hold for nondeterministic FSMs. Thus, in some cases, the oracle problem can be efficiently solved when using ⊆ s and where this is not the case, we can use the decision procedure for ⊆ w as a sound approximation.